Commits · 4be3a45158a7fb707973d4b00410e0d2981e6825 · OpenDAS / vllm_cscc

06 Nov, 2024 4 commits
- [distributed] add function to create ipc buffers directly (#10064) · 4be3a451
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4be3a451
- [Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) · 2bcbae70
  Travis Johnson authored Nov 05, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  2bcbae70
- [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
  Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  0c63c34f
- [Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
  Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  966e3169
05 Nov, 2024 4 commits
- [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) · ca9844b3
  youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ca9844b3
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
- [CI] Prune tests/models/decoder_only/language/* tests (#9940) · 02462465
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  02462465
- [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) · bbc3619d
  Cyrus Leung authored Nov 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bbc3619d
04 Nov, 2024 5 commits
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
- [Bugfix][CI/Build][Hardware][AMD] Shard ID parameters in AMD tests running parallel jobs (#9279) · 5208dc7a
  hissu-hyvarinen authored Nov 04, 2024
```
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
```
  5208dc7a
- [CI] Basic Integration Test For TPU (#9968) · 1c45f4c3
  Robert Shaw authored Nov 04, 2024
```
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
```
  1c45f4c3
- [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) · ac6b8f19
  Chauncey authored Nov 04, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ac6b8f19
- [Model] Add support for H2OVL-Mississippi models (#9747) · 54597724
  shanshan wang authored Nov 03, 2024
```
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  54597724
02 Nov, 2024 3 commits
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 10 commits
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
- [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) · 598b6d7b
  Pavani Majety authored Nov 01, 2024
  
  598b6d7b
- [Bugfix] Fix edge cases for MistralTokenizer (#9625) · 1dd4cb29
  Travis Johnson authored Nov 01, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
```
  1dd4cb29
- [Frontend] Use a proper chat template for VLM2Vec (#9912) · ba0d8920
  Cyrus Leung authored Nov 01, 2024
  
  ba0d8920
- [CI/Build] Add Model Tests for PixtralHF (#9813) · 30a2e807
  Michael Goin authored Nov 01, 2024
  
  30a2e807
- [Frontend] Chat-based Embeddings API (#9759) · 06386a64
  Cyrus Leung authored Nov 01, 2024
  
  06386a64
- [torch.compile] Adding torch compile annotations to some models (#9876) · 2b5bf209
  Yongzao authored Nov 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  2b5bf209
- [torch.compile] rework test plans (#9866) · 566cd277
  youkaichao authored Oct 31, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  566cd277
- [torch.compile] directly register custom op (#9896) · 96e0c9cb
  youkaichao authored Oct 31, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  96e0c9cb
- [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) · 031a7995
  Joe Runde authored Oct 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  031a7995
31 Oct, 2024 4 commits

[BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) · 9fb12f78
Mor Zusman authored Oct 31, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
9fb12f78

[Bugfix] Fix `illegal memory access` error with chunked prefill, prefix... · 55650c83

sasha0552 authored Oct 31, 2024


[Bugfix] Fix `illegal memory access` error with chunked prefill, prefix caching, block manager v2 and xformers enabled together (#9532)
Signed-off-by: sasha0552 <admin@sasha0552.org>

55650c83

[CI/Build] Add Model Tests for Qwen2-VL (#9846) · 16b8f7a8

Alex Brooks authored Oct 31, 2024


Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

16b8f7a8

[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens... · abbfb613
Guillaume Calmettes authored Oct 31, 2024
```
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
```
abbfb613

30 Oct, 2024 6 commits
- [torch.compile] upgrade tests (#9858) · 64384bbc
  youkaichao authored Oct 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  64384bbc
- [CI/Build] Simplify exception trace in api server tests (#9787) · 00d91c8a
  Yongzao authored Oct 31, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  00d91c8a
- [Bugfix][core] replace heartbeat with pid check (#9818) · 3b3f1e74
  Joe Runde authored Oct 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3b3f1e74
- [Misc] Add chunked-prefill support on FlashInfer. (#9781) · 9ff4511e
  Elfie Guo authored Oct 30, 2024
  
  9ff4511e
- [CI/Build] VLM Test Consolidation (#9372) · cc98f1e0
  Alex Brooks authored Oct 30, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  cc98f1e0
- [torch.compile] rework compile control with piecewise cudagraph (#9715) · ff5ed6e1
  youkaichao authored Oct 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ff5ed6e1
29 Oct, 2024 4 commits
- [Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339) · 882a1ad0
  Will Eaton authored Oct 29, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
```
  882a1ad0
- [Bugfix][Frontend] Guard against bad token ids (#9634) · 67bdf8e5
  Joe Runde authored Oct 29, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  67bdf8e5
- [CI][Bugfix] Skip chameleon for transformers 4.46.1 (#9808) · ab6f9816
  Michael Goin authored Oct 29, 2024
  
  ab6f9816
- [Hardware] using current_platform.seed_everything (#9785) · 622b7ab9
  wangshuai09 authored Oct 29, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  622b7ab9