Commits · 8017c8db7f7831b997a772da89ff8a134376260c · OpenDAS / vllm_cscc

"docs/serving/offline_inference.md" did not exist on "90bd2ae172007b7e84e15eea45b37826d5da1a67"

07 Apr, 2025 12 commits
- [Doc]Update image to latest version (#16186) · 8017c8db
  Robin authored Apr 07, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  8017c8db
- [Misc] improve example mlpspeculator and llm_engine_example (#16175) · dc3529db
  Reid authored Apr 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  dc3529db
- [Model] Add Qwen3 and Qwen3MoE (#15289) · 7699258e
  YamPengLi authored Apr 07, 2025
```
Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  7699258e
- [V1][Structured Output] Add `supports_structured_output()` method to Platform (#16148) · e9ba99f2
  Shanshan Shen authored Apr 07, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  e9ba99f2
- [VLM] Florence-2 supports online serving (#16164) · 7c803687
  Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  7c803687
- doc: fix some typos in doc (#16154) · 95d63f38
  yihong authored Apr 07, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  95d63f38
- [CI] Set max transformers version for Ultravox model test (#16149) · bb8dab82
  Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  bb8dab82
- [Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) · fc0f8776
  Isotr0py authored Apr 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  fc0f8776
- [Misc] Update Mistral-3.1 example (#16147) · 0a573867
  Cyrus Leung authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0a573867
- [V1][Minor] Minor simplification for get_computed_blocks (#16139) · 3749e287
  Woosuk Kwon authored Apr 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3749e287
- [Metrics] Add bucket for `request_latency`, `time_to_first_token` and... · 86fc2321
  Kay Yan authored Apr 07, 2025
```
[Metrics] Add bucket for `request_latency`, `time_to_first_token` and `time_per_output_token` (#15202)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
```
  86fc2321
- Fix requires-python (#16132) · 2549c0df
  Martin Hoyer authored Apr 07, 2025
  
  2549c0df
06 Apr, 2025 17 commits
- [V1][Minor] Optimize get_cached_block (#16135) · b10e5198
  Woosuk Kwon authored Apr 06, 2025
  
  b10e5198
- [TPU] Update PyTorch/XLA (#16130) · 9bde5ba1
  Chengji Yao authored Apr 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  9bde5ba1
- [Misc] update requires-python in pyproject.toml (#16116) · 72c8f1ad
  Reid authored Apr 06, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  72c8f1ad
- [Bugfix] add hf_token to EngineArgs (#16093) · da224daa
  paolovic authored Apr 06, 2025
```
Signed-off-by: paolovic <paul-philipp.luley@uzh.ch>
Co-authored-by: paolovic <paul-philipp.luley@uzh.ch>
```
  da224daa
- [Bugfix] LoRA : Fix the order in which the kernels process LoRAs (#16040) · 3a100b92
  Varun Sundar Rabindranath authored Apr 06, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  3a100b92
- [Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103) · 242a637a
  rongfu.leng authored Apr 06, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  242a637a
- [Misc] Improve model redirect to accept json dictionary (#16119) · c2a96715
  Isotr0py authored Apr 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c2a96715
- [Doc][Bugfix] Add missing EOF in k8s deploy doc (#16025) · d5ae4f7f
  Paul Schweigert authored Apr 06, 2025
  
  d5ae4f7f
- [Misc] refactor example eagle (#16100) · b6c502a1
  Reid authored Apr 06, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  b6c502a1
- [CI][V1] Fix passing `tokenizer` as kwarg to `validate_guidance_grammar` (#16117) · 9ca710e5
  Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9ca710e5
- [Frontend] Fix typo in tool chat templates for llama3.2 and toolace (#14501) · eb07c8cb
  Ben Jackson authored Apr 06, 2025
```
Signed-off-by: Ben Jackson <ben@ben.com>
```
  eb07c8cb
- [Benchmark] Add sampling parameters to benchmark_serving. (#16022) · ba108019
  Hyesoo Yang authored Apr 05, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  ba108019
- [Model] fix model testing for TeleChat2ForCausalLM and V0 llama4 (#16112) · 620fc2d0
  Lucia Fang authored Apr 05, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  620fc2d0
- [Model] use AutoWeightsLoader for phi, gemma, deepseek (#16088) · 29283eaa
  Jonghyun Choe authored Apr 06, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  29283eaa
- [Bugfix] fix use_atomic_add support of marlin kernel when using v1 engine (#15946) · 2fa66ef7
  Jinzhen Lin authored Apr 06, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  2fa66ef7
- [Misc] Remove redundant code (#16098) · 13affc43
  Chauncey authored Apr 06, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  13affc43
- [Misc] format output for encoder_decoder.py (#16095) · d8f094a9
  Reid authored Apr 06, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d8f094a9
05 Apr, 2025 6 commits
- Fix some capitalisations in generated examples doc titles (#16094) · 97ae6d77
  Harry Mellor authored Apr 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  97ae6d77
- Revert "doc: add info for macos clang errors (#16049)" (#16091) · 6baeee70
  yihong authored Apr 05, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  6baeee70
- [doc] fix 404 (#16082) · d2517a49
  Reid authored Apr 05, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d2517a49
- fix: support clang17 for macos and fix the real libomp (#16086) · 6342adc4
  yihong authored Apr 05, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  6342adc4
- [CI] Fix benchmark script level (#16089) · 0adba915
  Kevin H. Luu authored Apr 05, 2025
  
  0adba915
- [Misc] Auto detect bitsandbytes pre-quantized models (#16027) · 4285e423
  Tristan Leclercq authored Apr 05, 2025
```
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
```
  4285e423
04 Apr, 2025 5 commits
- [V1][Spec Decode] Update N-gram Proposer Interface (#15750) · 63375f0c
  Woosuk Kwon authored Apr 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  63375f0c
- [Bugfix][TPU] Fix V1 TPU worker for sliding window (#16059) · 70ad3f9e
  Michael Goin authored Apr 04, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  70ad3f9e
- [Kernel][Minor] Re-fuse triton moe weight application (#16071) · d6fc629f
  bnellnm authored Apr 04, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  d6fc629f
- Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
  Roger Wang authored Apr 04, 2025
  
  af51d80f
- [V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50
  Cyrus Leung authored Apr 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f5722a50