Commits · 551603feffd9b4ba98ccdd34e02e403e04db88c1 · OpenDAS / vllm_cscc

16 Dec, 2024 1 commit
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
05 Dec, 2024 1 commit
- [Bugfix] Fallback to outlines for complex json schemas (#10899) · 8d370e91
  Michael Goin authored Dec 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8d370e91
03 Dec, 2024 1 commit

[Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315

Aaron Pham authored Dec 03, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

9323a315

01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
27 Nov, 2024 1 commit
- [ci] fix slow tests (#10698) · 308cc5e2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  308cc5e2
26 Nov, 2024 1 commit
- [ci] add vllm_test_utils (#10659) · 334d64d1
  youkaichao authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  334d64d1
11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

06 Nov, 2024 1 commit
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
29 Oct, 2024 1 commit
- [Bugfix][Frontend] Guard against bad token ids (#9634) · 67bdf8e5
  Joe Runde authored Oct 29, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  67bdf8e5
22 Oct, 2024 1 commit
- [CI/Build] Replaced some models on tests for smaller ones (#9570) · c0292211
  Wallas Henrique authored Oct 22, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  c0292211
18 Oct, 2024 3 commits
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d
- [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) · de4008e2
  Joe Runde authored Oct 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  de4008e2
01 Oct, 2024 1 commit
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
27 Sep, 2024 1 commit
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
25 Sep, 2024 2 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
24 Sep, 2024 1 commit

[Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b

Andy authored Sep 24, 2024


Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>