Commits · a02a50e6e5bb74f9d48f75942e47197d22ec6444 · OpenDAS / vllm_cscc

"vllm/model_executor/models/step1.py" did not exist on "643ecf7b11a3e74c838f438cfc1b3e59c018853b"

06 Nov, 2024 13 commits

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6

[CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
21063c11
[distributed] add function to create ipc buffers directly (#10064) · 4be3a451
youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
4be3a451
[V1] Integrate Piecewise CUDA graphs (#10058) · 40899855
Woosuk Kwon authored Nov 05, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
40899855
[Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) · 9d59b755
zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
9d59b755

[Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) · ea928f60

arakowsk-amd authored Nov 05, 2024


Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>

ea928f60

[Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) · 2bcbae70
Travis Johnson authored Nov 05, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
2bcbae70
[Model][OpenVINO] Fix regressions from #8346 (#10045) · ffc0f2b4
Peter Salas authored Nov 05, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
ffc0f2b4
[Misc] Sort the list of embedding models (#10037) · 82bfc38d
Cyrus Leung authored Nov 06, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
82bfc38d
[v1] reduce graph capture time for piecewise cudagraph (#10059) · c4cacbaa
youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
c4cacbaa
[Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
0c63c34f
[Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
966e3169
[Bugfix] Properly propagate trust_remote_code settings (#10047) · 43300bd9
zifeitong authored Nov 05, 2024
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
43300bd9

05 Nov, 2024 10 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
- [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) · a53046b1
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  a53046b1
- [Misc] Add logging for CUDA memory (#10027) · 09d35503
  Chenghao (Alan) Yang authored Nov 05, 2024
```
Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  09d35503
- [Frontend] Fix tcp port reservation for api server (#10012) · 5952d811
  Russell Bryant authored Nov 05, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  5952d811
- [Misc] vllm CLI flags should be ordered for better user readability (#10017) · 93dee88f
  Chauncey authored Nov 05, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  93dee88f
- [BugFix] Lazy import ray (#10021) · 7a83b1ae
  Gene Der Su authored Nov 05, 2024
  
  7a83b1ae
- [Bugfix] Fixup Mamba (#10004) · ad233189
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ad233189
- [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) · bbc3619d
  Cyrus Leung authored Nov 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bbc3619d
- [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) · 04bbf38e
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  04bbf38e
04 Nov, 2024 14 commits
- [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) · 8f0a9ca8
  Michael Goin authored Nov 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8f0a9ca8
- [4.5/N] bugfix for quant config in speculative decode (#10007) · 2094062b
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2094062b
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [Bugfix] Fix `MQLLMEngine` hanging (#9973) · 04cef2c6
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  04cef2c6
- [Model] factoring out MambaMixer out of Jamba (#8993) · 603a661a
  Mor Zusman authored Nov 04, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  603a661a
- [Misc]Reduce BNB static variable (#9987) · fb2716d6
  Jee Jee Li authored Nov 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  fb2716d6
- [4/N] make quant config first-class citizen (#9978) · 8d72bb20
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d72bb20
- [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) · ac6b8f19
  Chauncey authored Nov 04, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ac6b8f19
- [Bugfix][OpenVINO] Fix circular reference #9939 (#9974) · ccb5376a
  Mengqing Cao authored Nov 04, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  ccb5376a
- [Misc] Compute query_start_loc/seq_start_loc on CPU (#9447) · 4dbcbbeb
  Yang Zheng authored Nov 04, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  4dbcbbeb
- [Bugfix]Using the correct type hints (#9885) · b67feb12
  Gregory Shtrasberg authored Nov 04, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b67feb12
- [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) · c49f0407
  Jee Jee Li authored Nov 04, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c49f0407
- [V1] Fix Configs (#9971) · 91c9ebbb
  Robert Shaw authored Nov 03, 2024
  
  91c9ebbb
- [Model] Add support for H2OVL-Mississippi models (#9747) · 54597724
  shanshan wang authored Nov 03, 2024
```
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  54597724
03 Nov, 2024 1 commit
- [V1] Support per-request seed (#9945) · 1f1b6d6e
  Nick Hill authored Nov 03, 2024
```
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
```
  1f1b6d6e
02 Nov, 2024 2 commits
- [bugfix] fix tsts (#9959) · 3bb4befe
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3bb4befe
- [torch.compile] Adding torch compile to vision-language models (#9946) · ae5279a1
  Yongzao authored Nov 03, 2024
  
  ae5279a1