Commits · 36e76700453924c8d421db99af70a88a1df835cd · OpenDAS / vllm_cscc · GitLab

30 Dec, 2024 8 commits
- [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631) · 36e76700
  whyiug authored Dec 31, 2024
  
  36e76700
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
- [VLM] Abstract out multi-modal data parsing in merged processor (#11620) · 8d9b6721
  Cyrus Leung authored Dec 30, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8d9b6721
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
- [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) · 5dbf8545
  Li, Jiang authored Dec 30, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  5dbf8545
- [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) · 970d6d07
  Tyler Michael Smith authored Dec 30, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  970d6d07
- [Docker] bump up neuron sdk v2.21 (#11593) · 628ec6c1
  Liangfu Chen authored Dec 29, 2024
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  628ec6c1
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
29 Dec, 2024 4 commits
- Remove print statement in DeepseekScalingRotaryEmbedding (#11604) · 0aa38d16
  Michael Goin authored Dec 29, 2024
  
  0aa38d16
- [Misc] KV cache transfer connector registry (#11481) · faef77c0
  Kuntai Du authored Dec 29, 2024
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  faef77c0
- [v1][bugfix] fix cudagraph with inplace buffer assignment (#11596) · dba4d9de
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dba4d9de
- [Doc] Convert list tables to MyST (#11594) · 32b4c63f
  Cyrus Leung authored Dec 29, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32b4c63f
28 Dec, 2024 10 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [bugfix] interleaving sliding window for cohere2 model (#11583) · 328841d0
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  328841d0
- [Doc] Minor documentation fixes (#11580) · d427e5cf
  Cyrus Leung authored Dec 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d427e5cf
- [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) · 42bb201f
  Woosuk Kwon authored Dec 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42bb201f
- [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (#11515) · 59d6bb4c
  hj-wei authored Dec 28, 2024
```
Signed-off-by: hjwei <hjwei_xd@163.com>
```
  59d6bb4c
- [Model] Remove hardcoded image tokens ids from Pixtral (#11582) · b7dcc003
  Roger Wang authored Dec 28, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b7dcc003
- [Model] Support InternLM2 Reward models (#11571) · d34be24b
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d34be24b
- [Bugfix] Last token measurement fix (#11376) · b5cbe8ee
  Rajveer Bachkaniwala authored Dec 27, 2024
```
Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b5cbe8ee
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
- [Doc] Update mllama example based on official doc (#11567) · a6073124
  Chen Zhang authored Dec 28, 2024
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a6073124
27 Dec, 2024 18 commits
- [Bugfix] Fix for ROCM compressed tensor support (#11561) · ac797994
  Selali authored Dec 27, 2024
  
  ac797994
- [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566) · dde1fa18
  Isotr0py authored Dec 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dde1fa18
- [Misc]Add BNB quantization for MolmoForCausalLM (#11551) · 0240402c
  Jee Jee Li authored Dec 28, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  0240402c
- [MODEL] LoRA support for Jamba model (#11209) · 55509c21
  ErezSC42 authored Dec 27, 2024
```
Signed-off-by: Erez Schwartz <erezs@ai21.com>
```
  55509c21
- [VLM] Support caching in merged multi-modal processor (#11396) · 10141809
  Cyrus Leung authored Dec 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  10141809
- [Doc] Add xgrammar in doc (#11549) · 5ce4627a
  Chen1022 authored Dec 27, 2024
```
Signed-off-by: ccjincong <chenjincong11@gmail.com>
```
  5ce4627a
- [Misc] Abstract the logic for reading and writing media content (#11527) · 7af553ea
  Cyrus Leung authored Dec 27, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7af553ea
- [Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546) · 2c9b8ea2
  Jee Jee Li authored Dec 27, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2c9b8ea2
- Update deploying_with_k8s.md with AMD ROCm GPU example (#11465) · d003f3ea
  AlexHe99 authored Dec 27, 2024
```
Signed-off-by: Alex He <alehe@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d003f3ea
- [Platform] Move model arch check to platform (#11503) · 6c6f7fe8
  Mengqing Cao authored Dec 27, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  6c6f7fe8
- [BugFix] Fix quantization for all other methods (#11547) · 2339d59f
  Robert Shaw authored Dec 27, 2024
  
  2339d59f
- [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) · 1b875a0e
  Robert Shaw authored Dec 27, 2024
  
  1b875a0e
- [misc] fix typing (#11540) · eb881ed0
  youkaichao authored Dec 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eb881ed0
- [CI] Fix broken CI (#11543) · 46d43594
  Robert Shaw authored Dec 26, 2024
  
  46d43594
- [V1] Fix yapf (#11538) · 81b979f2
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  81b979f2
- [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) · 371d04d3
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  371d04d3
- Update openai_compatible_server.md (#11536) · 0c0c2015
  Robert Shaw authored Dec 26, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  0c0c2015
- [Docs] Document Deepseek V3 support (#11535) · 82d24f7a
  Simon Mo authored Dec 26, 2024
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  82d24f7a