Commits · a03ea40792201ac8ff547d37d9f9255b347b9ccd · OpenDAS / vllm_cscc · GitLab

18 Nov, 2024 13 commits
- [3/N][torch.compile] consolidate custom op logging (#10399) · a03ea407
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a03ea407
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [Model][Bugfix] Support TP for PixtralHF ViT (#10405) · 281cc4b3
  Michael Goin authored Nov 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  281cc4b3
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [Model] Remove transformers attention porting in VITs (#10414) · e7ebb662
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e7ebb662
- [Model][LoRA]LoRA support added for glm-4v (#10418) · 5be4e52b
  B-201 authored Nov 18, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  5be4e52b
- [Model] Remove redundant softmax when using PoolingType.STEP (#10415) · 01aae1cc
  Maybewuss authored Nov 18, 2024
  
  01aae1cc
- [VLM] Report multi_modal_placeholders in output (#10407) · c7dec926
  lkchen authored Nov 18, 2024
```
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
```
  c7dec926
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
- [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) · 47826cac
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Nov 18, 2024
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  47826cac
- [Misc] Add uninitialized params tracking for `AutoWeightsLoader` (#10327) · c4e46433
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c4e46433
17 Nov, 2024 6 commits
- [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) · 80d85c5d
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  80d85c5d
- [Hardware] [HPU]add `mark_step` for hpu (#10239) · 76aab90a
  Kunshang Ji authored Nov 17, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  76aab90a
- [platforms] refactor cpu code (#10402) · 8d74b5ae
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d74b5ae
- [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) · cf349c4a
  Isotr0py authored Nov 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  cf349c4a
- [V1] Refactor model executable interface for all text-only language models (#10374) · 643ecf7b
  Roger Wang authored Nov 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  643ecf7b
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
16 Nov, 2024 6 commits
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
- [Misc] Medusa supports custom bias (#10361) · b98d89ef
  Sky Lee authored Nov 17, 2024
  
  b98d89ef
- [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) · 1d754726
  rasmith authored Nov 16, 2024
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  1d754726
- [misc][plugin] improve log messages (#10386) · 2f427c2d
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2f427c2d
- [doc] add doc for the plugin system (#10372) · 755b8535
  youkaichao authored Nov 15, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  755b8535
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
15 Nov, 2024 15 commits
- [Frontend] Add --version flag to CLI (#10369) · a067f85e
  Russell Bryant authored Nov 15, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a067f85e
- [Bugfix] Ensure special tokens are properly filtered out for guided structured... · 691a3ec0
  Guillaume Calmettes authored Nov 15, 2024
```
[Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (#10363)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  691a3ec0
- [core][misc] keep compatibility for old-style classes (#10356) · 3a763ba0
  youkaichao authored Nov 15, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3a763ba0
- [Misc] Fix some help info of arg_utils to improve readability (#10362) · f2056f72
  shangmingc authored Nov 15, 2024
  
  f2056f72
- [Bugfix] Fix fully sharded LoRA bug (#10352) · 1d65ec7e
  Jee Jee Li authored Nov 15, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1d65ec7e
- [Doc] Remove float32 choice from --lora-dtype (#10348) · 26908554
  Xin Yang authored Nov 15, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  26908554
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
- Add default value to avoid Falcon crash (#5363) (#10347) · 3d158cdc
  wchen61 authored Nov 15, 2024
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  3d158cdc
- [Misc] Consolidate pooler config overrides (#10351) · 2ac6d0e7
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2ac6d0e7
- [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) · 2ec88272
  Sky Lee authored Nov 15, 2024
  
  2ec88272
- [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) · b40cf640
  Cyrus Leung authored Nov 15, 2024
  
  b40cf640
- [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308) · 2885ba0e
  Tyler Michael Smith authored Nov 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2885ba0e
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
- [Bugfix] Fix unable to load some models (#10312) · 972112d8
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  972112d8
- [Tool parsing] Improve / correct mistral tool parsing (#10333) · 11cd1ae6
  Patrick von Platen authored Nov 15, 2024
  
  11cd1ae6