Commits · 02798ecabed36f4c255f5a12ad6c271f95cd8c4e · OpenDAS / vllm_cscc · GitLab

18 Jan, 2025 3 commits
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
- [core] further polish memory profiling (#12126) · da02cb4b
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  da02cb4b
- [misc] fix cross-node TP (#12166) · 2b835032
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2b835032
17 Jan, 2025 9 commits
- [torch.compile] disable logging when cache is disabled (#12043) · 7b98a65a
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7b98a65a
- [AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (#12134) · b5b57e30
  Gregory Shtrasberg authored Jan 17, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b5b57e30
- [Bugfix] Mistral tokenizer encode accept list of str (#12149) · 54cacf00
  Kunshang Ji authored Jan 18, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  54cacf00
- [Bugfix] Fix score api for missing max_model_len validation (#12119) · 58fd57ff
  Wallas Henrique authored Jan 17, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  58fd57ff
- [core] allow callable in collective_rpc (#12151) · 87a0c076
  youkaichao authored Jan 17, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  87a0c076
- [CI/Build][CPU][Bugfix] Fix CPU CI (#12150) · d4e61945
  Li, Jiang authored Jan 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  d4e61945
- [Misc][LoRA] Improve the readability of LoRA error messages (#12102) · 07934cc2
  Jee Jee Li authored Jan 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  07934cc2
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
- [BugFix] add more `is not None` check in VllmConfig.__post_init__ (#12138) · d1adb9b4
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d1adb9b4
16 Jan, 2025 8 commits
- [Bugfix] Set enforce_eager automatically for mllama (#12127) · d06e8240
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d06e8240
- [Model] Add support for deepseek-vl2-tiny model (#12068) · 62b06ba2
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  62b06ba2
- [Bugfix] Fix max image feature size for Llava-one-vision (#12104) · 874f7c29
  Roger Wang authored Jan 16, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  874f7c29
- [core] LLM.collective_rpc interface and RLHF example (#12084) · 92e793d9
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  92e793d9
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
- [Bugfix] Remove hardcoded `head_size=256` for Deepseek v2 and v3 (#12067) · dd7c9ad8
  Isotr0py authored Jan 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dd7c9ad8
- Various cosmetic/comment fixes (#12089) · 9aa1519f
  Michael Goin authored Jan 16, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9aa1519f
- [Core] Default to using per_token quantization for fp8 when cutlass is supported. (#8651) · fa0050db
  Elfie Guo authored Jan 15, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  fa0050db
15 Jan, 2025 17 commits
- [Bugfix] Fix _get_lora_device for HQQ marlin (#12090) · ebd8c669
  Varun Sundar Rabindranath authored Jan 16, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  ebd8c669
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
- [Bugfix] use right truncation for non-generative tasks (#12050) · edce722e
  Joe Runde authored Jan 15, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  edce722e
- [Doc]: Update `OpenAI-Compatible Server` documents (#12082) · 57e729e8
  maang-h authored Jan 16, 2025
  
  57e729e8
- [Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6
  kewang-xlnx authored Jan 16, 2025
```
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  de0526f6
- Misc: allow to use proxy in `HTTPConnection` (#12042) · 5ecf3e0a
  Yuan authored Jan 15, 2025
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  5ecf3e0a
- [Model]: Support internlm3 (#12037) · 97eb97b5
  RunningLeon authored Jan 15, 2025
  
  97eb97b5
- [Platform] Do not raise error if _Backend is not found (#12023) · 3adf0ffd
  wangxiyuan authored Jan 15, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
```
  3adf0ffd
- Type-fix: make execute_model output type optional (#12020) · ad388d25
  Keyun Tong authored Jan 15, 2025
  
  ad388d25
- Fix: cases with empty sparsity config (#12057) · cbe94391
  Rahul Tuli authored Jan 15, 2025
```
Signed-off-by: Rahul Tuli <rahul@neuralmagic.com>
```
  cbe94391
- [V1][Prefix Cache] Move the logic of num_computed_tokens into KVCacheManager (#12003) · 994fc655
  Chen Zhang authored Jan 15, 2025
  
  994fc655
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
- [core] Turn off GPU communication overlap for Ray executor (#12051) · f218f9c2
  Rui Qiao authored Jan 14, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  f218f9c2
- [Misc] Add multipstep chunked-prefill support for FlashInfer (#10467) · 0794e744
  Elfie Guo authored Jan 14, 2025
  
  0794e744
- [V1][BugFix] Fix edge case in VLM scheduling (#12065) · b7ee940a
  Woosuk Kwon authored Jan 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b7ee940a
- [Platform] move current_memory_usage() into platform (#11369) · 9ddac563
  Shanshan Shen authored Jan 15, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  9ddac563
- [Kernel] Support MulAndSilu (#11624) · 42f5e7c5
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  42f5e7c5
14 Jan, 2025 3 commits
- [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) · a3a3ee4e
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a3ee4e
- [Doc]: Update the Json Example of the `Engine Arguments` document (#12045) · 87054a57
  maang-h authored Jan 15, 2025
  
  87054a57
- [Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) · a2d2acb4
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a2d2acb4