Commits · 994fc655b71f59f61b82cc44e868091dae493a84 · OpenDAS / vllm_cscc

15 Jan, 2025 9 commits
- [V1][Prefix Cache] Move the logic of num_computed_tokens into KVCacheManager (#12003) · 994fc655
  Chen Zhang authored Jan 15, 2025
  
  994fc655
- [Doc] Update examples to remove SparseAutoModelForCausalLM (#12062) · 3f9b7ab9
  Kyle Sayers authored Jan 15, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  3f9b7ab9
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
- [core] Turn off GPU communication overlap for Ray executor (#12051) · f218f9c2
  Rui Qiao authored Jan 14, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  f218f9c2
- [Misc] Add multipstep chunked-prefill support for FlashInfer (#10467) · 0794e744
  Elfie Guo authored Jan 14, 2025
  
  0794e744
- [V1][BugFix] Fix edge case in VLM scheduling (#12065) · b7ee940a
  Woosuk Kwon authored Jan 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b7ee940a
- [Platform] move current_memory_usage() into platform (#11369) · 9ddac563
  Shanshan Shen authored Jan 15, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  9ddac563
- [HPU][Bugfix] Don't use /dev/accel/accel0 for HPU autodetection in setup.py (#12046) · 1a51b9f8
  Konrad Zawora authored Jan 15, 2025
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  1a51b9f8
- [Kernel] Support MulAndSilu (#11624) · 42f5e7c5
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  42f5e7c5
14 Jan, 2025 11 commits
- [Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924) · a3a3ee4e
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a3a3ee4e
- [Doc]: Update the Json Example of the `Engine Arguments` document (#12045) · 87054a57
  maang-h authored Jan 15, 2025
  
  87054a57
- Explain where the engine args go when using Docker (#12041) · c9d6ff53
  Harry Mellor authored Jan 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c9d6ff53
- [Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) · a2d2acb4
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a2d2acb4
- [Platform] Add output for Attention Backend (#11981) · 2e0e0176
  wangxiyuan authored Jan 14, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  2e0e0176
- [Kernel] Revert the API change of Attention.forward (#12038) · 1f18adb2
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  1f18adb2
- [Bugfix] Fix various bugs in multi-modal processor (#12031) · bb354e6b
  Cyrus Leung authored Jan 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  bb354e6b
- [HPU][misc] add comments for explanation (#12034) · ff39141a
  youkaichao authored Jan 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ff39141a
- [Doc] Update Quantization Hardware Support Documentation (#12025) · 8a1f938e
  TJian authored Jan 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  8a1f938e
- [HPU][Bugfix] set_forward_context and CI test execution (#12014) · 078da319
  Konrad Zawora authored Jan 14, 2025
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  078da319
- [Docs] Add Sky Computing Lab to project intro (#12019) · 1a401252
  Woosuk Kwon authored Jan 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1a401252
13 Jan, 2025 17 commits
- [Bugfix] Fix deepseekv3 gate bias error (#12002) · f35ec461
  Steve Luo authored Jan 14, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  f35ec461
- [Doc] Fix build from source and installation link in README.md (#12013) · 289b5191
  Yikun Jiang authored Jan 14, 2025
```
Signed-off-by: Yikun <yikunkero@gmail.com>
```
  289b5191
- bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (#11982) · c6db2131
  elijah authored Jan 13, 2025
```
Signed-off-by: elijah <f1renze.142857@gmail.com>
```
  c6db2131
- [Platform] Move get_punica_wrapper() function to Platform (#11516) · a7d59688
  Shanshan Shen authored Jan 13, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  a7d59688
- [platform] add device_control env var (#12009) · 458e63a2
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  458e63a2
- [Doc] Organise installation documentation into categories and tabs (#11935) · e8c23ff9
  Harry Mellor authored Jan 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e8c23ff9
- [Doc][V1] Update model implementation guide for V1 support (#11998) · cd824990
  Roger Wang authored Jan 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  cd824990
- [Kernel] unified_attention for Attention.forward (#11967) · 0f8cafe2
  Chen Zhang authored Jan 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  0f8cafe2
- Fix Max Token ID for Qwen-VL-Chat (#11980) · 5340a30d
  Alex Brooks authored Jan 13, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  5340a30d
- [platform] add ray_device_key (#11948) · 89ce62a3
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  89ce62a3
- [Misc]Minor Changes about Worker (#11555) · c3f05b09
  Chenguang Li authored Jan 13, 2025
```
Signed-off-by: Chenguang Li <757486878@qq.com>
```
  c3f05b09
- [Misc] Fix Deepseek V2 fp8 kv-scale remapping (#11947) · cf6bbcb4
  Concurrensee authored Jan 13, 2025
```
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
```
  cf6bbcb4
- [CI][Spec Decode] fix: broken test for EAGLE model (#11972) · 80ea3af1
  Sungjae Lee authored Jan 13, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
```
  80ea3af1
- [Bug] Fix usage of `.transpose()` and `.view()` consecutively. (#11979) · 9dd02d85
  Siyuan Li authored Jan 13, 2025
  
  9dd02d85
- [MISC] fix typo in kv transfer send recv test (#11983) · f7b3ba82
  Yangcheng Li authored Jan 13, 2025
  
  f7b3ba82
- [V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (#11973) · 619ae268
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  619ae268
- [Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685) · d14e98d9
  Isotr0py authored Jan 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d14e98d9
12 Jan, 2025 3 commits

[V1][Core][1/n] Logging and Metrics (#11962) · 9597a095
Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
9597a095
[Hardware][TPU] workaround fix for MoE on TPU (#11764) · 263a870e
Avshalom Manevich authored Jan 12, 2025

263a870e

[Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) · 8bddb735

Akshat Tripathi authored Jan 12, 2025


Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

8bddb735