Commits · b5cbe8eeb30e86c8477d91c66f5c7a10e4ee754b · OpenDAS / vllm_cscc · GitLab

28 Dec, 2024 1 commit

[Bugfix] Last token measurement fix (#11376) · b5cbe8ee

Rajveer Bachkaniwala authored Dec 27, 2024


Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

b5cbe8ee

23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
19 Dec, 2024 1 commit

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

13 Dec, 2024 1 commit
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
12 Dec, 2024 1 commit
- Fix logging of the vLLM Config (#11143) · 9f3974a3
  Jeremy Arnold authored Dec 12, 2024
  
  9f3974a3
11 Dec, 2024 1 commit
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
10 Dec, 2024 2 commits
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [Bugfix] Fix usage of `deprecated` decorator (#11025) · 391d7b27
  Cyrus Leung authored Dec 10, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  391d7b27
09 Dec, 2024 1 commit
- [misc] clean up and unify logging (#10999) · 46004e83
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  46004e83
06 Dec, 2024 1 commit

[torch.compile] add logging for compilation time (#10941) · b031a455

youkaichao authored Dec 06, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

b031a455

04 Dec, 2024 1 commit
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
03 Dec, 2024 2 commits
- [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315
  Aaron Pham authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  9323a315
- [Misc] Remove deprecated names (#10817) · 3257d449
  Cyrus Leung authored Dec 03, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3257d449
01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
27 Nov, 2024 1 commit
- [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) · 197b4484
  Mor Zusman authored Nov 27, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  197b4484
26 Nov, 2024 1 commit
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
23 Nov, 2024 1 commit
- [Core] remove temporary local variables in LLMEngine.__init__ (#10577) · 28598f39
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  28598f39
19 Nov, 2024 2 commits
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
- [Bugfix] Guard for negative counter metrics to prevent crash (#10430) · 272e31c0
  Travis Johnson authored Nov 18, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  272e31c0
16 Nov, 2024 1 commit
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
15 Nov, 2024 1 commit
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
11 Nov, 2024 1 commit

[Metrics] add more metrics (#4464) · 874f551b

harrywu authored Nov 12, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

874f551b

09 Nov, 2024 1 commit

[Frontend][Core] Override HF `config.json` via CLI (#5836) · b09895a6

Krishna Mandal authored Nov 09, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

b09895a6

06 Nov, 2024 2 commits

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6

[CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
21063c11

05 Nov, 2024 1 commit
- [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) · bbc3619d
  Cyrus Leung authored Nov 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bbc3619d
04 Nov, 2024 1 commit
- [Frontend] Add max_tokens prometheus metric (#9881) · ac04a97a
  tomeras91 authored Nov 05, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  ac04a97a
02 Nov, 2024 1 commit
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
01 Nov, 2024 2 commits
- [1/N] pass the complete config from engine to executor (#9933) · 18bd7587
  youkaichao authored Nov 01, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  18bd7587
- [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) · 031a7995
  Joe Runde authored Oct 31, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  031a7995
30 Oct, 2024 1 commit
- [Model] Support math-shepherd-mistral-7b-prm model (#9697) · 81f09cfd
  Went-Liang authored Oct 31, 2024
```
Signed-off-by: Went-Liang <wenteng_liang@163.com>
```
  81f09cfd
29 Oct, 2024 2 commits
- [Bugfix][Frontend] Guard against bad token ids (#9634) · 67bdf8e5
  Joe Runde authored Oct 29, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  67bdf8e5
- [Misc] Add metrics for request queue time, forward time, and execute time (#9659) · 74fc2d77
  科英 authored Oct 30, 2024
  
  74fc2d77
26 Oct, 2024 1 commit
- [Frontend] Bad words sampling parameter (#9717) · 07e981fd
  Vasiliy Alekseev authored Oct 26, 2024
```
Signed-off-by: Vasily Alexeev <alvasian@yandex.ru>
```
  07e981fd
24 Oct, 2024 2 commits
- [core] simplify seq group code (#9569) · 4fdc581f
  youkaichao authored Oct 24, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  4fdc581f
- [Bugfix]: Make chat content text allow type content (#9358) · 33bab410
  Vinay R Damodaran authored Oct 24, 2024
```
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
```
  33bab410
23 Oct, 2024 1 commit
- [Bugfix] Fix divide by zero when serving Mamba models (#9617) · e5ac6a41
  Tyler Michael Smith authored Oct 23, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  e5ac6a41
22 Oct, 2024 2 commits
- [BugFix] Fix metrics error for --num-scheduler-steps > 1 (#8234) · b17046e2
  yulei authored Oct 23, 2024
  
  b17046e2
- [BugFix] Prevent exporting duplicate OpenTelemetry spans (#9017) · cd5601ac
  Ronen Schaffer authored Oct 22, 2024
  
  cd5601ac