Commits · 8b0fe06c890a202eba24d517cc77562e4a8b0d0c · OpenDAS / vllm_cscc · GitLab

21 Nov, 2024 4 commits
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
- [Platforms] Add `device_type` in `Platform` (#10508) · 9d827170
  Mengqing Cao authored Nov 21, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  9d827170
- [Core] Add Sliding Window Support with Flashinfer (#10462) · 6c1208d0
  Pavani Majety authored Nov 20, 2024
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  6c1208d0
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
20 Nov, 2024 11 commits
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
- [Bugfix]: allow extra fields in requests to openai compatible server (#10463) · c68f7ede
  Guillaume Calmettes authored Nov 20, 2024
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c68f7ede
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [platforms] restore xpu check for parallel config (#10479) · 772a6673
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  772a6673
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
- [Platforms] Refactor xpu code (#10468) · d5b28447
  Mengqing Cao authored Nov 20, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  d5b28447
- [Bugfix] Handle conflicts between modern and legacy fields (#10471) · 09dbf9ff
  Cyrus Leung authored Nov 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  09dbf9ff
- [model] Reduce medusa weight (#10454) · 343041c4
  Sky Lee authored Nov 20, 2024
```
Signed-off-by: skylee-01 <497627264@qq.com>
```
  343041c4
- [Bugfix] Enforce no chunked prefill for embedding models (#10470) · b4be5a8a
  Cyrus Leung authored Nov 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b4be5a8a
- [Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456) · ad44437b
  Isotr0py authored Nov 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ad44437b
- [Misc] Add __setitem__ for LazyDict (#10469) · 9e05252b
  Yanyi Liu authored Nov 20, 2024
```
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
```
  9e05252b
19 Nov, 2024 12 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [Core] Avoid metrics log noise when idle (#8868) · efa90846
  Russell Bryant authored Nov 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  efa90846
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
- [Doc] fix link for page that was renamed (#10455) · fd9f1249
  Russell Bryant authored Nov 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  fd9f1249
- [Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) · 11fd7ea6
  Patrick von Platen authored Nov 19, 2024
  
  11fd7ea6
- [BugFix] Fix hermes tool parser output error stream arguments in some cases (#10395) (#10398) · f028dff3
  COSMOPlat authored Nov 19, 2024
```
Signed-off-by: xiyuan lee <lixiyuan@haier.com>
```
  f028dff3
- [misc][plugin] improve plugin loading (#10443) · 25f9c789
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25f9c789
- [Doc] Add the start of an arch overview page (#10368) · 5390d666
  Russell Bryant authored Nov 19, 2024
  
  5390d666
- [Misc] Avoid misleading warning messages (#10438) · 382b6a48
  Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  382b6a48
- [Bugfix] Guard for negative counter metrics to prevent crash (#10430) · 272e31c0
  Travis Johnson authored Nov 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  272e31c0
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
- [Bugfix]Fix Phi-3 BNB online quantization (#10417) · 7eb719df
  Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7eb719df
18 Nov, 2024 13 commits
- [ci][bugfix] fix kernel tests (#10431) · 2298e69b
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2298e69b
- [3/N][torch.compile] consolidate custom op logging (#10399) · a03ea407
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a03ea407
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [Model][Bugfix] Support TP for PixtralHF ViT (#10405) · 281cc4b3
  Michael Goin authored Nov 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  281cc4b3
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [Model] Remove transformers attention porting in VITs (#10414) · e7ebb662
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e7ebb662
- [Model][LoRA]LoRA support added for glm-4v (#10418) · 5be4e52b
  B-201 authored Nov 18, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  5be4e52b
- [Model] Remove redundant softmax when using PoolingType.STEP (#10415) · 01aae1cc
  Maybewuss authored Nov 18, 2024
  
  01aae1cc
- [VLM] Report multi_modal_placeholders in output (#10407) · c7dec926
  lkchen authored Nov 18, 2024
```
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
```
  c7dec926
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
- [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) · 47826cac
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Nov 18, 2024
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  47826cac