Commits · a324d3a1a74ab0a3fafc0f2d19860bd1d1301a85 · OpenDAS / vllm_cscc

20 Nov, 2024 1 commit
- Change granite chat template to keep json list formatting for tool calls (#10452) · a324d3a1
  Maximilien de Bayser authored Nov 19, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
```
  a324d3a1
19 Nov, 2024 16 commits

[Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
b00b33d7
[Core] Avoid metrics log noise when idle (#8868) · efa90846
Russell Bryant authored Nov 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
efa90846
[6/N] torch.compile rollout to users (#10437) · 803f37ea
youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
803f37ea
[Doc] fix link for page that was renamed (#10455) · fd9f1249
Russell Bryant authored Nov 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
fd9f1249

Fix: Build error seen on Power Architecture (#10421) · 1ea291a4

Manjul Mohan authored Nov 19, 2024


Signed-off-by: Manjul Mohan <manjul.mohan@ibm.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: ismael-dm <ismaeldm99@gmail.com>
Signed-off-by: Andrew Nesbitt <andrewnez@gmail.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Manjul Mohan manjul.mohan@ibm.com <manjulmohan@ltcd97-lp2.aus.stglabs.ibm.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: ismael-dm <ismaeldm99@gmail.com>
Co-authored-by: Andrew Nesbitt <andrewnez@gmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Angus Wang <wangjadehao@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Ricky Xu <rickyx@anyscale.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1ea291a4

[Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) · 11fd7ea6
Patrick von Platen authored Nov 19, 2024

11fd7ea6
[BugFix] Fix hermes tool parser output error stream arguments in some cases (#10395) (#10398) · f028dff3
COSMOPlat authored Nov 19, 2024
```
Signed-off-by: xiyuan lee <lixiyuan@haier.com>
```
f028dff3
[CI][CPU] adding numa node number as container name suffix (#10441) · b4614656
Yuan authored Nov 19, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
b4614656
[misc][plugin] improve plugin loading (#10443) · 25f9c789
youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
25f9c789
[Doc] Add the start of an arch overview page (#10368) · 5390d666
Russell Bryant authored Nov 19, 2024

5390d666
[Misc] Avoid misleading warning messages (#10438) · 382b6a48
Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
382b6a48
[Bugfix] Guard for negative counter metrics to prevent crash (#10430) · 272e31c0
Travis Johnson authored Nov 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
272e31c0
Add openai.beta.chat.completions.parse example to structured_outputs.rst (#10433) · 74f8c2cf
Michael Goin authored Nov 18, 2024

74f8c2cf
[Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
8c1fb507
[Bugfix]Fix Phi-3 BNB online quantization (#10417) · 7eb719df
Jee Jee Li authored Nov 19, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
7eb719df

[ci/build] Have dependabot ignore all patch update (#10436) · 284203f1

Kevin H. Luu authored Nov 18, 2024

We have too many dependencies and all patch updates can be a little noisy. This is to have dependabot ignore all patch version updates.

284203f1

18 Nov, 2024 18 commits
- [misc] partial prefix & random input generation benchmark (#9929) · 90a6c759
  Ricky Xu authored Nov 18, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  90a6c759
- [ci][bugfix] fix kernel tests (#10431) · 2298e69b
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2298e69b
- [3/N][torch.compile] consolidate custom op logging (#10399) · a03ea407
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a03ea407
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [Model][Bugfix] Support TP for PixtralHF ViT (#10405) · 281cc4b3
  Michael Goin authored Nov 18, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  281cc4b3
- Fix open_collective value in FUNDING.yml (#10426) · 4f686d13
  Andrew Nesbitt authored Nov 18, 2024
```
Signed-off-by: Andrew Nesbitt <andrewnez@gmail.com>
```
  4f686d13
- [Doc] Add documentation for Structured Outputs (#9943) · 31894a21
  ismael-dm authored Nov 18, 2024
```
Signed-off-by: ismael-dm <ismaeldm99@gmail.com>
```
  31894a21
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [Doc] Update doc for LoRA support in GLM-4V (#10425) · 4186be81
  B-201 authored Nov 18, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  4186be81
- [Model] Remove transformers attention porting in VITs (#10414) · e7ebb662
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e7ebb662
- [Model][LoRA]LoRA support added for glm-4v (#10418) · 5be4e52b
  B-201 authored Nov 18, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  5be4e52b
- [Model] Remove redundant softmax when using PoolingType.STEP (#10415) · 01aae1cc
  Maybewuss authored Nov 18, 2024
  
  01aae1cc
- [VLM] Report multi_modal_placeholders in output (#10407) · c7dec926
  lkchen authored Nov 18, 2024
```
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
```
  c7dec926
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
- [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) · 47826cac
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Nov 18, 2024
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  47826cac
- [Misc] Add uninitialized params tracking for `AutoWeightsLoader` (#10327) · c4e46433
  Isotr0py authored Nov 18, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c4e46433
17 Nov, 2024 5 commits
- [Misc] Enhance offline_inference to support user-configurable paramet… (#10392) · d1557e66
  wchen61 authored Nov 17, 2024
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  d1557e66
- [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) · 80d85c5d
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  80d85c5d
- [Hardware] [HPU]add `mark_step` for hpu (#10239) · 76aab90a
  Kunshang Ji authored Nov 17, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  76aab90a
- [platforms] refactor cpu code (#10402) · 8d74b5ae
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d74b5ae
- [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) · cf349c4a
  Isotr0py authored Nov 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  cf349c4a