Commits · 35bae114a89e03e3dc6a6d2f758378e58938bffa · OpenDAS / vllm_cscc

17 Dec, 2024 1 commit
- fix gh200 tests on main (#11246) · 35bae114
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  35bae114
16 Dec, 2024 4 commits
- [Docs] hint to enable use of GPU performance counters in profiling tools for... · 35ffa682
  bk-TurbaAI authored Dec 16, 2024
```
[Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (#11235)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  35ffa682
- [Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) · bddbbcb1
  Jani Monoses authored Dec 16, 2024
  
  bddbbcb1
- WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (#11212) · b3b1526f
  cennn authored Dec 16, 2024
```
Signed-off-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
```
  b3b1526f
- Update deploying_with_k8s.rst (#10922) · da6f4092
  AlexHe99 authored Dec 16, 2024
  
  da6f4092
15 Dec, 2024 2 commits
- [Doc] add documentation for disaggregated prefilling (#11197) · 38e599d6
  Kuntai Du authored Dec 15, 2024
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  38e599d6
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
14 Dec, 2024 1 commit
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
13 Dec, 2024 3 commits
- [Doc] Reorganize online pooling APIs (#11172) · 0920ab91
  Cyrus Leung authored Dec 14, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0920ab91
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- PaliGemma 2 support (#11142) · 7cd74091
  Jani Monoses authored Dec 13, 2024
  
  7cd74091
12 Dec, 2024 3 commits
- fix(docs): typo in helm install instructions (#11141) · d4d5291c
  Ramon Ziai authored Dec 12, 2024
```
Signed-off-by: Ramon Ziai <ramon.ziai@bettermarks.com>
```
  d4d5291c
- [Model] Add support for embedding model GritLM (#10816) · 1da8f0e1
  Pooya Davoodi authored Dec 11, 2024
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  1da8f0e1
- Update link to LlamaStack remote vLLM guide in serving_with_llamastack.rst (#11112) · 24a36d6d
  Yuan Tang authored Dec 11, 2024
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  24a36d6d
11 Dec, 2024 4 commits
- [Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103) · fd222206
  bingps authored Dec 11, 2024
```
Signed-off-by: Guangda Liu <bingps@users.noreply.github.com>
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
```
  fd222206
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
10 Dec, 2024 2 commits
- Add example of helm chart for vllm deployment on k8s (#9199) · fe2e10c7
  Maxime Fournioux authored Dec 10, 2024
```
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
```
  fe2e10c7
- [Docs] Add dedicated tool calling page to docs (#10554) · 6d525288
  Michael Goin authored Dec 09, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  6d525288
09 Dec, 2024 1 commit
- [Doc][V1] Add V1 support column for multimodal models (#10998) · af7c4a92
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  af7c4a92
07 Dec, 2024 3 commits
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [Doc] Explicitly state that InternVL 2.5 is supported (#10978) · 1c768fe5
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1c768fe5
06 Dec, 2024 1 commit
- [Doc] add KubeAI to serving integrations (#10837) · 74062740
  Sam Stoelinga authored Dec 06, 2024
```
Signed-off-by: Sam Stoelinga <sammiestoel@gmail.com>
```
  74062740
05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
04 Dec, 2024 2 commits
- [CI/Build] improve python-only dev setup (#9621) · e4c34c23
  Daniele authored Dec 04, 2024
```
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  e4c34c23
- [ci/build] Update vLLM postmerge ECR repo (#10887) · c92acb96
  Kevin H. Luu authored Dec 04, 2024
  
  c92acb96
03 Dec, 2024 2 commits

[Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315

Aaron Pham authored Dec 03, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

9323a315

[Doc] Add github links for source code references (#10672) · ef51831e

Russell Bryant authored Dec 03, 2024


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

ef51831e

01 Dec, 2024 2 commits
- [doc] add warning about comparing hf and vllm outputs (#10805) · 169a0ff9
  youkaichao authored Dec 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  169a0ff9
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
30 Nov, 2024 1 commit
- [doc] format fix (#10789) · 7e4bbda5
  wangxiyuan authored Nov 30, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  7e4bbda5
28 Nov, 2024 3 commits
- [Model] Add Internlm2 LoRA support (#5064) · c83919c7
  Isotr0py authored Nov 29, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c83919c7
- [Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561) · 5fc5ce0f
  sixgod authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5fc5ce0f
- [Doc] Update model in arch_overview.rst to match comment (#10701) · 278be671
  罗泽轩 authored Nov 28, 2024
```
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
```
  278be671
27 Nov, 2024 1 commit

[Model] Support telechat2 (#10311) · 1209261e

shunxing12345 authored Nov 27, 2024


Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: xiangw2 <xiangw2@chinatelecom.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>

1209261e

26 Nov, 2024 3 commits

[Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018

Murali Andoorveedu authored Nov 26, 2024


Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>

db66e018

custom allreduce + torch.compile (#10121) · 9a88f897

Sage Moore authored Nov 26, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

9a88f897

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) · a6760f64

Sanket Kale authored Nov 26, 2024


Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

a6760f64