Commits · 82bfc38d079b1ef5f4b88ac7094a00029d2e99af · OpenDAS / vllm_cscc · GitLab

06 Nov, 2024 5 commits
- [Misc] Sort the list of embedding models (#10037) · 82bfc38d
  Cyrus Leung authored Nov 06, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82bfc38d
- [v1] reduce graph capture time for piecewise cudagraph (#10059) · c4cacbaa
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c4cacbaa
- [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
  Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  0c63c34f
- [Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
  Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  966e3169
- [Bugfix] Properly propagate trust_remote_code settings (#10047) · 43300bd9
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  43300bd9
05 Nov, 2024 15 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
- [CI] Prune tests/models/decoder_only/language/* tests (#9940) · 02462465
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  02462465
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
- [Feature] Update benchmark_throughput.py to support image input (#9851) · d2e80332
  lkchen authored Nov 05, 2024
```
Signed-off-by: Linkun Chen <github+anyscale@lkchen.net>
Co-authored-by: Linkun Chen <github+anyscale@lkchen.net>
```
  d2e80332
- [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) · a53046b1
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a53046b1
- [CI/Build] Limit github CI jobs based on files changed (#9928) · 731aec5b
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  731aec5b
- [Misc] Add logging for CUDA memory (#10027) · 09d35503
  Chenghao (Alan) Yang authored Nov 05, 2024
```
Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  09d35503
- Refactor TPU requirements file and pin build dependencies (#10010) · cd34029e
  Richard Liu authored Nov 05, 2024
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  cd34029e
- [Frontend] Fix tcp port reservation for api server (#10012) · 5952d811
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  5952d811
- [Misc] vllm CLI flags should be ordered for better user readability (#10017) · 93dee88f
  Chauncey authored Nov 05, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  93dee88f
- [BugFix] Lazy import ray (#10021) · 7a83b1ae
  Gene Der Su authored Nov 05, 2024
  
  7a83b1ae
- [Bugfix] Fixup Mamba (#10004) · ad233189
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ad233189
- [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) · bbc3619d
  Cyrus Leung authored Nov 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bbc3619d
- [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) · 04bbf38e
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  04bbf38e
04 Nov, 2024 20 commits
- [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) · 8f0a9ca8
  Michael Goin authored Nov 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8f0a9ca8
- [4.5/N] bugfix for quant config in speculative decode (#10007) · 2094062b
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2094062b
- [Bugfix] Upgrade to pytorch 2.5.1 (#10001) · d93478b3
  bnellnm authored Nov 04, 2024
```
Signed-off-by: Bill Nell <bill@neuralmagic.com>
```
  d93478b3
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [Misc] Refactor benchmark_throughput.py (#9779) · 9a5664d4
  lkchen authored Nov 04, 2024
```
Signed-off-by: Linkun Chen <github+anyscale@lkchen.net>
Co-authored-by: Linkun Chen <lkchen@github.com>
Co-authored-by: Linkun Chen <github+anyscale@lkchen.net>
```
  9a5664d4
- [Bugfix] Fix `MQLLMEngine` hanging (#9973) · 04cef2c6
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  04cef2c6
- [Doc] Update VLM doc about loading from local files (#9999) · 6e056bcf
  Roger Wang authored Nov 04, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  6e056bcf
- [Bugfix][CI/Build][Hardware][AMD] Shard ID parameters in AMD tests running parallel jobs (#9279) · 5208dc7a
  hissu-hyvarinen authored Nov 04, 2024
```
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
```
  5208dc7a
- [CI] Basic Integration Test For TPU (#9968) · 1c45f4c3
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
```
  1c45f4c3
- [Model] factoring out MambaMixer out of Jamba (#8993) · 603a661a
  Mor Zusman authored Nov 04, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  603a661a
- [Misc]Reduce BNB static variable (#9987) · fb2716d6
  Jee Jee Li authored Nov 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  fb2716d6
- [4/N] make quant config first-class citizen (#9978) · 8d72bb20
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d72bb20
- [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) · ac6b8f19
  Chauncey authored Nov 04, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ac6b8f19
- [Bugfix][OpenVINO] Fix circular reference #9939 (#9974) · ccb5376a
  Mengqing Cao authored Nov 04, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  ccb5376a
- [Bugfix] Fix E2EL mean and median stats (#9984) · ea4adedd
  Tran Quang Dai authored Nov 04, 2024
```
Signed-off-by: daitran2k1 <tranquangdai7a@gmail.com>
```
  ea4adedd
- [Misc] Compute query_start_loc/seq_start_loc on CPU (#9447) · 4dbcbbeb
  Yang Zheng authored Nov 04, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  4dbcbbeb
- [Bugfix]Using the correct type hints (#9885) · b67feb12
  Gregory Shtrasberg authored Nov 04, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b67feb12
- [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) · c49f0407
  Jee Jee Li authored Nov 04, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c49f0407
- [V1] Fix Configs (#9971) · 91c9ebbb
  Robert Shaw authored Nov 03, 2024
  
  91c9ebbb
- [Model] Add support for H2OVL-Mississippi models (#9747) · 54597724
  shanshan wang authored Nov 03, 2024
```
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  54597724