Commits · d18bab3587e6804aaa74f93fb1d55ab5766f75a3 · OpenDAS / vllm_cscc

"vllm/v1/executor/ray_executor.py" did not exist on "ddeec11ba9d39ee72bb3337f810c6197c96b10e1"

03 Jul, 2024 3 commits

[CI] Fix base url doesn't strip "/" (#6087) · d18bab35
SangBin Cho authored Jul 03, 2024

d18bab35

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

[hardware][misc] introduce platform abstraction (#6080) · 482045ee
youkaichao authored Jul 02, 2024

482045ee

02 Jul, 2024 10 commits
- [Model] Jamba support (#4115) · 9d6a8daa
  Mor Zusman authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  9d6a8daa
- [CORE] Quantized lm-head Framework (#4442) · ee93f4f9
  Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  ee93f4f9
- [ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) · 7c008c51
  Robert Shaw authored Jul 02, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7c008c51
- Update conftest.py (#6076) · 4d26d806
  Robert Shaw authored Jul 02, 2024
  
  4d26d806
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
- [Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
  Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
  15aba081
- [Doc] Reinstate doc dependencies (#6061) · 31354e56
  Cyrus Leung authored Jul 02, 2024
  
  31354e56
- [VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c
  xwjiang2010 authored Jul 02, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  98d6682c
- [Frontend] Add template related params to request (#5709) · 2c37540a
  danieljannai21 authored Jul 02, 2024
  
  2c37540a
- [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
  Alexander Matveev authored Jul 01, 2024
  
  3476ed08
01 Jul, 2024 14 commits
- [Model] Changes to MLPSpeculator to support tie_weights and input_scale (#5965) · 54600709
  Thomas Parnell authored Jul 02, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
```
  54600709
- [Frontend] Relax api url assertion for openai benchmarking (#6046) · e373853e
  James Whedbee authored Jul 01, 2024
  
  e373853e
- [BugFix] Ensure worker model loop is always stopped at the right time (#5987) · c87ebc3e
  Nick Hill authored Jul 01, 2024
  
  c87ebc3e
- [Bugfix] Add explicit `end_forward` calls to flashinfer (#6044) · c4059ea5
  Antoni Baum authored Jul 01, 2024
  
  c4059ea5
- [Bugfix][Doc] Fix Doc Formatting (#6048) · 8e0817c2
  Roger Wang authored Jul 01, 2024
  
  8e0817c2
- add FAQ doc under 'serving' (#5946) · 83bdcb6a
  ning.zhang authored Jul 01, 2024
  
  83bdcb6a
- [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) · 12a59959
  Avshalom Manevich authored Jul 02, 2024
  
  12a59959
- [Bugfix] Use RayActorError for older versions of Ray in RayTokenizerGroupPool (#6039) · dec6fc6f
  Antoni Baum authored Jul 01, 2024
  
  dec6fc6f
- [doc][misc] further lower visibility of simple api server (#6041) · 8893130b
  youkaichao authored Jul 01, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  8893130b
- [Misc] update benchmark backend for scalellm (#6018) · bb603268
  zhyncs authored Jul 02, 2024
  
  bb603268
- [doc][misc] remove deprecated api server in doc (#6037) · 4050d646
  youkaichao authored Jul 01, 2024
  
  4050d646
- [ CI ] Re-enable Large Model LM Eval (#6031) · d76084c1
  Robert Shaw authored Jul 01, 2024
  
  d76084c1
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
30 Jun, 2024 9 commits
- [ Misc ] Refactor w8a8 to use `process_weights_after_load` (Simplify Weight Loading) (#5940) · af9ad46f
  Robert Shaw authored Jun 30, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  af9ad46f
- [Misc] Fix `get_min_capability` (#5971) · 7836fdcc
  Dipika Sikka authored Jun 30, 2024
  
  7836fdcc
- [ CI ] Temporarily Disable Large LM-Eval Tests (#6005) · deacb7ec
  Robert Shaw authored Jun 30, 2024
```
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic>
```
  deacb7ec
- [Lora] Use safetensor keys instead of adapter_config.json to find unexpected modules. (#5909) · f5e73c9f
  SangBin Cho authored Jul 01, 2024
```
Co-authored-by: sang <sangcho@anyscale.com>
```
  f5e73c9f
- [Frontend]: Support base64 embedding (#5935) · c6c240aa
  llmpros authored Jun 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c6c240aa
- [ci][distributed] fix device count call · 2be6955a
  youkaichao authored Jun 30, 2024
```
[ci][distributed] fix some cuda init that makes it necessary to use spawn (#5991)
```
  2be6955a
- [CI/Build] [3/3] Reorganize entrypoints tests (#5966) · 9d47f64e
  Cyrus Leung authored Jun 30, 2024
  
  9d47f64e
- [CI/Build] Reuse code for checking output consistency (#5988) · cff6a1fe
  Cyrus Leung authored Jun 30, 2024
  
  cff6a1fe
- [CI/Build] Temporarily Remove Phi3-Vision from TP Test (#5989) · bcc6a09b
  Roger Wang authored Jun 29, 2024
  
  bcc6a09b
29 Jun, 2024 4 commits
- [Bugfix][CI/Build][Hardware][AMD] Install matching torchvision to fix AMD tests (#5949) · 9def1066
  Matt Wong authored Jun 29, 2024
  
  9def1066
- [ CI/Build ] LM Eval Harness Based CI Testing (#5838) · 75aa1442
  Robert Shaw authored Jun 29, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  75aa1442
- [CI/Build] Add TP test for vision models (#5892) · 99397da5
  Cyrus Leung authored Jun 29, 2024
  
  99397da5
- [ CI/Build ] Added E2E Test For Compressed Tensors (#5839) · 8dbfcd35
  Robert Shaw authored Jun 29, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  8dbfcd35