Commits · 1e3e76b6cc75c90da6c738951b0b47835de0b6be · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "ad44437ba33e8d31962d272be238eeed4a1b4f84"

05 Mar, 2025 17 commits
- [Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due... · 1e3e76b6
  pyc96 authored Mar 05, 2025
```
[Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due to shape mismatch (#14237)
Signed-off-by: pyc96 <pychen96@gmail.com>
```
  1e3e76b6
- [misc] Add FlashMLA as a new option of VLLM_ATTENTION_BACKEND env (#14267) · 1b7624bf
  Serena authored Mar 06, 2025
  
  1b7624bf
- [V1][BugFix] Fix for mixed top_k batch (#14301) · ac60dc7f
  Nick Hill authored Mar 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Ye Cao <caoye.cao@alibaba-inc.com>
```
  ac60dc7f
- Deprecate `best_of` Sampling Parameter in anticipation for vLLM V1 (#13997) · a4f1ee35
  Vincent authored Mar 05, 2025
```
Signed-off-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a4f1ee35
- [V1][Minor] Remove obsolete FIXME comment (#14304) · a32c8669
  Nick Hill authored Mar 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  a32c8669
- [LoRA] Remove linear hack outside transformers backend (#14177) · e17e4488
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e17e4488
- [V1][Frontend] Add Testing For V1 Runtime Parameters (#14159) · 257e200a
  Robert Shaw authored Mar 05, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  257e200a
- Small update for external_launcher backend docs (#14288) · 47d4a7e0
  Zhe Zhang authored Mar 05, 2025
  
  47d4a7e0
- [Bugfix][V1] Fix allowed_token_ids for v1 Sampler (#14169) · 8d6cd32b
  Lu Fang authored Mar 05, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  8d6cd32b
- [Misc][V1] Avoid using `envs.VLLM_USE_V1` in mm processing (#14256) · ec79b67c
  Roger Wang authored Mar 04, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  ec79b67c
- [Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066) · 32985bed
  Benjamin Chislett authored Mar 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  32985bed
- [platforms] improve rocm debugging info (#14257) · 6eaf9302
  youkaichao authored Mar 05, 2025
  
  6eaf9302
- [V1] EP/TP MoE + DP Attention (#13931) · 72c62eae
  Tyler Michael Smith authored Mar 05, 2025
  
  72c62eae
- [Model] New model support for Phi-4-multimodal-instruct (#14119) · 0a995d54
  Congcong Chen authored Mar 04, 2025
  
  0a995d54
- [V1][Bugfix] Do not reset prefix caching metrics (#14235) · ade3f7d9
  Cody Yu authored Mar 04, 2025
  
  ade3f7d9
- [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) · 0df25101
  rainkert authored Mar 05, 2025
```
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
```
  0df25101
- [V1][TPU] TPU multimodal model support for ragged attention (#14158) · fbfc3ee3
  Michael Goin authored Mar 04, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  fbfc3ee3
04 Mar, 2025 16 commits
- Clean up unused padding_idx variables across many model definitions (#13240) · 4f5b059f
  Tyler Michael Smith authored Mar 04, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4f5b059f
- [Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228) · 288ca110
  Kuntai Du authored Mar 04, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  288ca110
- [Frontend] Do `prompt_logprobs` clamping for chat as well as completions (#14225) · e5b2f160
  Harry Mellor authored Mar 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e5b2f160
- Fix performance when `--generation-config` is not `None` (#14223) · 9badee53
  Harry Mellor authored Mar 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9badee53
- [TPU][Profiler] Support start_profile/stop_profile in TPU worker (#13988) · beebf474
  Siyuan Liu authored Mar 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  beebf474
- [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) · b3cf368d
  lkchen authored Mar 04, 2025
  
  b3cf368d
- [V0][Metrics] Deprecate some questionable request time metrics (#14135) · c8525f06
  Mark McLoughlin authored Mar 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c8525f06
- [V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869) · 5db6b2c9
  Nick Hill authored Mar 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5db6b2c9
- [Bugfix] Restrict MacOS CPU detection (#14210) · 6247bae6
  Michael Goin authored Mar 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6247bae6
- [sleep mode] error out with expandable_segments (#14189) · 71c4b405
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  71c4b405
- [platform] add debug logging during inferring the device type (#14195) · ac65bc92
  youkaichao authored Mar 04, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ac65bc92
- Use math.prod instead of np.prod for trivial ops (#14142) · 66233af7
  Zhanwen Chen authored Mar 04, 2025
  
  66233af7
- [core] Pass all driver env vars to ray workers unless excluded (#14099) · bf13d409
  Rui Qiao authored Mar 03, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  bf13d409
- [Misc] Remove lru_cache in NvmlCudaPlatform (#14156) · 989f4f43
  Cody Yu authored Mar 03, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  989f4f43
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
- [Model] Add support for GraniteMoeShared models (#13313) · c060b714
  Travis Johnson authored Mar 03, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c060b714
03 Mar, 2025 7 commits
- [v1] Add comments to the new ragged paged attention Pallas kernel (#14155) · 79e4937c
  iefgnoix authored Mar 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  79e4937c
- [Kernel] Optimize moe intermediate_cache usage (#13625) · 19d98e0c
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  19d98e0c
- [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) · 2b04c209
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  2b04c209
- [WIP][[V1][Metrics] Implement max_num_generation_tokens, request_params_n,... · ae122b1c
  Mark McLoughlin authored Mar 03, 2025
```
[WIP][[V1][Metrics] Implement max_num_generation_tokens,  request_params_n, and request_params_max_tokens metrics (#14055)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  ae122b1c
- [V1] Simplify stats logging (#14082) · 872db2be
  Nick Hill authored Mar 03, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  872db2be
- [V0][Metrics] Deprecate some KV/prefix cache metrics (#14136) · 2dfdfed8
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  2dfdfed8
- [V0][Metrics] Remove unimplemented `vllm:tokens_total` (#14134) · c41d2715
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c41d2715