Commits · 7b203b76947de241659d0f1a8b77183bc48a6dd2 · OpenDAS / vllm_cscc

"examples/backends/vllm/launch/agg_tracing.sh" did not exist on "6bccf099f8372d17cae50f8050d42dd8a2c14d6d"

18 Feb, 2025 3 commits
- [Bugfix] Fix invalid rotary embedding unit test (#13431) · 38094584
  Liangfu Chen authored Feb 18, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  38094584
- Add outlines fallback when JSON schema has enum (#13449) · b53d7998
  Michael Goin authored Feb 18, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b53d7998
- [Model] Enable quantization support for `transformers` backend (#12960) · 67ef8f66
  Isotr0py authored Feb 18, 2025
  
  67ef8f66
17 Feb, 2025 4 commits
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2
- [V1] Get input tokens from scheduler (#13339) · 4c21ce9e
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c21ce9e
- [Model] Support Mamba2 (Codestral Mamba) (#9292) · 1f69c4a8
  Tyler Michael Smith authored Feb 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  1f69c4a8
- [Feature][Spec Decode] Simplify the use of Eagle Spec Decode (#12304) · 46cdd595
  shangmingc authored Feb 17, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  46cdd595
16 Feb, 2025 4 commits
- [Bugfix] Fix 2 Node and Spec Decode tests (#13341) · 5d2965b7
  Cyrus Leung authored Feb 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5d2965b7
- [ci] skip failed tests for flashinfer (#13352) · 124776eb
  youkaichao authored Feb 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  124776eb
- [BugFix] Enhance test_pos_encoding to support execution on multi-devices (#13187) · dc0f7ccf
  wchen61 authored Feb 16, 2025
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  dc0f7ccf
- [V1][Spec Decode] Ngram Spec Decode (#12193) · 80f63a39
  Lily Liu authored Feb 15, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  80f63a39
15 Feb, 2025 3 commits
- [V1][PP] Run engine busy loop with batch queue (#13064) · 9206b3d7
  Cody Yu authored Feb 15, 2025
  
  9206b3d7
- [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) · 2ad1bc7a
  Mark McLoughlin authored Feb 15, 2025
  
  2ad1bc7a
- [V1][CI] Fix failed v1-test because of min_p (#13316) · e7eea5a5
  Woosuk Kwon authored Feb 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e7eea5a5
14 Feb, 2025 10 commits
- [V1][Core] min_p sampling support (#13191) · a12934d3
  Aoyu authored Feb 15, 2025
```
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
```
  a12934d3
- [Core] Reduce TTFT with concurrent partial prefills (#10235) · 3bcb8c75
  Joe Runde authored Feb 14, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3bcb8c75
- [Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236) · 5e5c8e09
  Michael Goin authored Feb 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  5e5c8e09
- Support logit_bias in v1 Sampler (#13079) · 6224a9f6
  Lu Fang authored Feb 14, 2025
  
  6224a9f6
- [WIP] TPU V1 Support Refactored (#13049) · 45f90bcb
  Alexander Matveev authored Feb 14, 2025
  
  45f90bcb
- [Bugfix][V1] GPUModelRunner._update_states should return True when there is a... · b0ccfc56
  Kero Liang authored Feb 14, 2025
```
[Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126)
```
  b0ccfc56
- [V1] LoRA - Enable Serving Usecase (#12883) · cbc40128
  Varun Sundar Rabindranath authored Feb 14, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cbc40128
- Consolidate Llama model usage in tests (#13094) · f2b20fe4
  Harry Mellor authored Feb 14, 2025
  
  f2b20fe4
- [Bugfix/CI] Turn test_compressed_tensors_2of4_sparse back on (#13250) · 09545c0a
  Tyler Michael Smith authored Feb 13, 2025
  
  09545c0a
- [Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198) · c1e37bf7
  Tyler Michael Smith authored Feb 13, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c1e37bf7
13 Feb, 2025 8 commits
- [Frontend] Add `/v1/audio/transcriptions` OpenAI API endpoint (#12909) · d84cef76
  Nicolò Lucchesi authored Feb 13, 2025
  
  d84cef76
- [Bugfix] Missing Content Type returns 500 Internal Server Error (#13193) · 37dfa600
  Vaibhav Jain authored Feb 13, 2025
  
  37dfa600
- [VLM] Separate text-only and vision variants of the same model architecture (#13157) · 1bc3b5e7
  Cyrus Leung authored Feb 13, 2025
  
  1bc3b5e7
- [VLM] Merged multi-modal processor for Molmo (#12966) · c9d3ecf0
  Cyrus Leung authored Feb 13, 2025
  
  c9d3ecf0
- [V1][core] Implement pipeline parallel on Ray (#12996) · 9605c125
  Rui Qiao authored Feb 13, 2025
  
  9605c125
- [Bugfix] deepseek_r1_reasoning_parser put reason content in wrong field in... · 04f50ad9
  LikeSundayLikeRain authored Feb 13, 2025
```
[Bugfix] deepseek_r1_reasoning_parser put reason content in wrong field in certain edge case (#13097)
```
  04f50ad9
- [VLM] Implement merged multimodal processor for Mllama (#11427) · bc55d130
  Isotr0py authored Feb 13, 2025
  
  bc55d130
- [NVIDIA] Support nvfp4 quantization (#12784) · 4fc5c23b
  Kaixi Hou authored Feb 12, 2025
  
  4fc5c23b
12 Feb, 2025 7 commits
- [CI] Fix failing FP8 cpu offload test (#13170) · 14b7899d
  Michael Goin authored Feb 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  14b7899d
- [CORE] [QUANT] Support for GPTQModel's `dynamic` quantization per module override/control (#7086) · 36a08630
  Qubitium-ModelCloud authored Feb 13, 2025
  
  36a08630
- [Misc] Delete unused LoRA modules (#13151) · 82cabf53
  Jee Jee Li authored Feb 13, 2025
  
  82cabf53
- [Frontend] Generate valid tool call IDs when using `tokenizer-mode=mistral` (#12332) · 314cfade
  Rafael Vasquez authored Feb 12, 2025
  
  314cfade
- [Neuron][Kernel] Support Longer Sequences in NKI-based Flash PagedAttention... · e92694b6
  Lingfan Yu authored Feb 11, 2025
```
[Neuron][Kernel] Support Longer Sequences in NKI-based Flash PagedAttention and Improve Efficiency (#12921)
Signed-off-by: Lingfan Yu <lingfany@amazon.com>
```
  e92694b6
- [Model] IBM/NASA Prithvi Geospatial model (#12830) · 974dfd49
  Christian Pinto authored Feb 12, 2025
  
  974dfd49
- [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) · 3ee696a6
  Keyun Tong authored Feb 11, 2025
```
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
```
  3ee696a6
11 Feb, 2025 1 commit
- [BugFix] Pop instead of del CUDA_VISIBLE_DEVICES (#12962) · 6c4dbe23
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Feb 11, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  6c4dbe23