Commits · d054da1992175787f936d18aead51bef663a0399 · OpenDAS / vllm_cscc

03 Jun, 2025 1 commit

[Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) · 9112b443

Siyuan Liu authored Jun 02, 2025


Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>

9112b443

02 Jun, 2025 1 commit
- [Core] Support inplace model weights loading (#18745) · 9760fd8f
  22quinn authored Jun 02, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  9760fd8f
01 Jun, 2025 1 commit

[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with... · a35ca765

Isotr0py authored Jun 01, 2025


[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with arbitrary components (#18987)
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>

a35ca765

31 May, 2025 1 commit

[Neuron] Add Multi-Modal model support for Neuron (#18921) · 2a50ef57

Satyajith Chilappagari authored May 31, 2025


Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
Co-authored-by: Rohith Nallamaddi <nalrohit@amazon.com>
Co-authored-by: FeliciaLuo <luof@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>

2a50ef57

29 May, 2025 2 commits
- Fix an error in dummy weight loading for quantization models (#18855) · f8977c23
  Chenyaaang authored May 29, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  f8977c23
- [Neuron] Add multi-LoRA support for Neuron. (#18284) · 972eddf7
  Satyajith Chilappagari authored May 29, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  972eddf7
26 May, 2025 1 commit
- [Doc] Move examples and further reorganize user guide (#18666) · 82e2339b
  Cyrus Leung authored May 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82e2339b
25 May, 2025 3 commits
- [BUGFIX] catch subclass first for try...except (#18672) · 44073a7a
  Ning Xie authored May 25, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  44073a7a
- [Misc][ModelScope] Change to use runtime VLLM_USE_MODELSCOPE (#18655) · 6ab681bc
  Mengqing Cao authored May 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  6ab681bc
- [MISC] correct signature for LoaderFunction (#18670) · 6c6dcd86
  Ning Xie authored May 25, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  6c6dcd86
23 May, 2025 4 commits
- [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) · 4fc1bf81
  Feng XiaoLong authored May 24, 2025
```
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
```
  4fc1bf81
- [Bugfix] Fix transformers model impl ignored for mixtral quant (#18602) · 6220f3c6
  Tristan Leclercq authored May 23, 2025
```
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
```
  6220f3c6
- [V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) · c6b636f9
  Mark McLoughlin authored May 23, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c6b636f9
- [Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization... · c32e249a
  Sanger Steel authored May 22, 2025
```
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926)
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
```
  c32e249a
22 May, 2025 2 commits

Order sequence ids + config update to support specifying custom quantization layers (#18279) · fa72f9a8

aws-elaineyz authored May 22, 2025


Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Tailin Pan <tailinpa@amazon.com>
Co-authored-by: Rishabh Rajesh <rishyraj@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Maxwell Goldberg <mgld@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>

fa72f9a8

Update default neuron config for speculation (#18274) · ebed81fb

aws-elaineyz authored May 22, 2025


Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>

ebed81fb

19 May, 2025 4 commits
- Neuron up mistral (#18222) · dc1440cf
  Satyajith Chilappagari authored May 19, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  dc1440cf
- [Quantization] Pool model support bitsandbytes (#18087) · 6781af56
  Jee Jee Li authored May 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6781af56
- [BugFix] [Vul] Add missing `usedforsecurity=False` in MD5 hashing to enable FIPS (#18319) · d637b960
  Shaoyu Yang authored May 19, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: shaoyuyoung <shaoyuyoung@gmail.com>
Co-authored-by: cascade <cascade812@outlook.com>
```
  d637b960
- fix: Add type specifications for CLI arguments in tensorizer options (#18314) · 275c5dae
  CYJiang authored May 19, 2025
  
  275c5dae
15 May, 2025 1 commit
- Update deprecated type hinting in `model_loader` (#18130) · 07ad2712
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  07ad2712
13 May, 2025 1 commit
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
07 May, 2025 4 commits

[Quantization] Quark MXFP4 format loading (#16943) · db593aa6
Bowen Bao authored May 07, 2025

db593aa6

Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49

Satyajith Chilappagari authored May 07, 2025


Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>

043e4c49

[Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
ba7703e6
[Misc] Split model loader (#17712) · 822de7fb
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
822de7fb

02 May, 2025 1 commit
- Add `pt_load_map_location` to allow loading to cuda (#16869) · 109e15a3
  Jerry Zhang authored May 01, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  109e15a3
30 Apr, 2025 2 commits
- [Fix] Support passing args to logger (#17425) · da4e7687
  Aaron Pham authored Apr 30, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  da4e7687
- Improve configs - `ModelConfig` (#17130) · 13698db6
  Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  13698db6
24 Apr, 2025 1 commit
- More informative error when using Transformers backend (#16988) · 2c8ed8ee
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2c8ed8ee
23 Apr, 2025 1 commit
- Improve Transformers backend model loading QoL (#17039) · 8e630d68
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8e630d68
22 Apr, 2025 2 commits
- [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) · d6da9322
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d6da9322
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
10 Apr, 2025 2 commits
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
- [TPU] Fix dummy loading OOM (#16372) · 1621b252
  Chengji Yao authored Apr 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  1621b252
09 Apr, 2025 1 commit
- update neuron config (#16289) · 24834f48
  ajayvohra2005 authored Apr 09, 2025
```
Signed-off-by: Ajay Vohra <ajayvohr@amazon.com>
```
  24834f48
08 Apr, 2025 1 commit
- [Bugfix] Handle `process_weights_after_loading` for `QKVCrossParallelLinear` (#15328) · 40b4284f
  Isotr0py authored Apr 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  40b4284f
07 Apr, 2025 1 commit

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>