Commits · a5fda50a10641e47c0c290907f30ef2add6d4e7a · OpenDAS / vllm_cscc · GitLab

06 Nov, 2024 12 commits
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
- [distributed] add function to create ipc buffers directly (#10064) · 4be3a451
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4be3a451
- [V1] Integrate Piecewise CUDA graphs (#10058) · 40899855
  Woosuk Kwon authored Nov 05, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  40899855
- [Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) · 9d59b755
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  9d59b755
- [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) · ea928f60
  arakowsk-amd authored Nov 05, 2024
```
Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>
```
  ea928f60
- [Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) · 2bcbae70
  Travis Johnson authored Nov 05, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  2bcbae70
- [Model][OpenVINO] Fix regressions from #8346 (#10045) · ffc0f2b4
  Peter Salas authored Nov 05, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  ffc0f2b4
- [Misc] Sort the list of embedding models (#10037) · 82bfc38d
  Cyrus Leung authored Nov 06, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82bfc38d
- [v1] reduce graph capture time for piecewise cudagraph (#10059) · c4cacbaa
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c4cacbaa
- [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
  Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  0c63c34f
- [Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
  Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  966e3169
- [Bugfix] Properly propagate trust_remote_code settings (#10047) · 43300bd9
  zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  43300bd9
05 Nov, 2024 10 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
- [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) · a53046b1
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a53046b1
- [Misc] Add logging for CUDA memory (#10027) · 09d35503
  Chenghao (Alan) Yang authored Nov 05, 2024
```
Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  09d35503
- [Frontend] Fix tcp port reservation for api server (#10012) · 5952d811
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  5952d811
- [Misc] vllm CLI flags should be ordered for better user readability (#10017) · 93dee88f
  Chauncey authored Nov 05, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  93dee88f
- [BugFix] Lazy import ray (#10021) · 7a83b1ae
  Gene Der Su authored Nov 05, 2024
  
  7a83b1ae
- [Bugfix] Fixup Mamba (#10004) · ad233189
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ad233189
- [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) · bbc3619d
  Cyrus Leung authored Nov 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bbc3619d
- [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) · 04bbf38e
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  04bbf38e
04 Nov, 2024 14 commits
- [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) · 8f0a9ca8
  Michael Goin authored Nov 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8f0a9ca8
- [4.5/N] bugfix for quant config in speculative decode (#10007) · 2094062b
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2094062b
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [Bugfix] Fix `MQLLMEngine` hanging (#9973) · 04cef2c6
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  04cef2c6
- [Model] factoring out MambaMixer out of Jamba (#8993) · 603a661a
  Mor Zusman authored Nov 04, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  603a661a
- [Misc]Reduce BNB static variable (#9987) · fb2716d6
  Jee Jee Li authored Nov 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  fb2716d6
- [4/N] make quant config first-class citizen (#9978) · 8d72bb20
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d72bb20
- [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) · ac6b8f19
  Chauncey authored Nov 04, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ac6b8f19
- [Bugfix][OpenVINO] Fix circular reference #9939 (#9974) · ccb5376a
  Mengqing Cao authored Nov 04, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  ccb5376a
- [Misc] Compute query_start_loc/seq_start_loc on CPU (#9447) · 4dbcbbeb
  Yang Zheng authored Nov 04, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  4dbcbbeb
- [Bugfix]Using the correct type hints (#9885) · b67feb12
  Gregory Shtrasberg authored Nov 04, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b67feb12
- [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) · c49f0407
  Jee Jee Li authored Nov 04, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c49f0407
- [V1] Fix Configs (#9971) · 91c9ebbb
  Robert Shaw authored Nov 03, 2024
  
  91c9ebbb
- [Model] Add support for H2OVL-Mississippi models (#9747) · 54597724
  shanshan wang authored Nov 03, 2024
```
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  54597724
03 Nov, 2024 1 commit
- [V1] Support per-request seed (#9945) · 1f1b6d6e
  Nick Hill authored Nov 03, 2024
```
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
```
  1f1b6d6e
02 Nov, 2024 3 commits
- [bugfix] fix tsts (#9959) · 3bb4befe
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3bb4befe
- [torch.compile] Adding torch compile to vision-language models (#9946) · ae5279a1
  Yongzao authored Nov 03, 2024
  
  ae5279a1
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3