Commits · 263a870ee18bd6a90e25dbfa342be32c6b92c33e · OpenDAS / vllm_cscc · GitLab

12 Jan, 2025 5 commits
- [Hardware][TPU] workaround fix for MoE on TPU (#11764) · 263a870e
  Avshalom Manevich authored Jan 12, 2025
  
  263a870e
- [Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) · 8bddb735
  Akshat Tripathi authored Jan 12, 2025
```
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  8bddb735
- [Model] Initialize support for Deepseek-VL2 models (#11578) · f967e51f
  Isotr0py authored Jan 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  f967e51f
- [CI/Build] Add markdown linter (#11857) · 43f3d9e6
  Rafael Vasquez authored Jan 12, 2025
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  43f3d9e6
- [V1] Avoid sending text prompt to core engine (#11963) · b25cfab9
  Roger Wang authored Jan 11, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b25cfab9
11 Jan, 2025 8 commits
- [Model] Add cogagent model support vLLM (#11742) · 4b657d32
  sixgod authored Jan 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  4b657d32
- [Bugfix] Fix RobertaModel loading (#11940) · d697dc01
  Nicolò Lucchesi authored Jan 11, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d697dc01
- [Doc] Basic guide for writing unit tests for new models (#11951) · a991f7d5
  Cyrus Leung authored Jan 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a991f7d5
- [CI/Build] Move model-specific multi-modal processing tests (#11934) · 7a3a83e3
  Cyrus Leung authored Jan 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a3a83e3
- [Bugfix] fused_experts_impl wrong compute type for float32 (#11921) · c32a7c7c
  shaochangxu authored Jan 11, 2025
```
Signed-off-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
Co-authored-by: shaochangxu.scx <shaochangxu.scx@antgroup.com>
```
  c32a7c7c
- [Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (#11672) · 2118d056
  Sungjae Lee authored Jan 11, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
```
  2118d056
- [ci] fix broken distributed-tests-4-gpus (#11937) · 899136b8
  youkaichao authored Jan 11, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  899136b8
- [mypy] Fix mypy warnings in api_server.py (#11941) · c9f09a4f
  Fred Reiss authored Jan 10, 2025
```
Signed-off-by: Fred Reiss <frreiss@us.ibm.com>
```
  c9f09a4f
10 Jan, 2025 20 commits
- [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#11939) · d45cbe70
  Travis Johnson authored Jan 10, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  d45cbe70
- [Misc] Update benchmark_prefix_caching.py fixed example usage (#11920) · 8a579408
  minmin authored Jan 11, 2025
```
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
```
  8a579408
- [Misc] Clean up debug code in Deepseek-V3 (#11930) · 46fa98cc
  Isotr0py authored Jan 11, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  46fa98cc
- [Hardware][CPU] Support MOE models on x86 CPU (#11831) · aa1e77a1
  Li, Jiang authored Jan 11, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  aa1e77a1
- Doc fix in `benchmark_long_document_qa_throughput.py` (#11933) · 5959564f
  Kuntai Du authored Jan 10, 2025
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  5959564f
- [Docs] Fix docstring in `get_ip` function (#11932) · f33e033e
  Kuntai Du authored Jan 10, 2025
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  f33e033e
- [Doc] Rename offline inference examples (#11927) · 482cdc49
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  482cdc49
- [platform] support custom torch.compile backend key (#11318) · 20410b2f
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  20410b2f
- [Doc] [1/N] Initial guide for merged multi-modal processor (#11925) · 12664ddd
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  12664ddd
- [ci] Fix sampler tests (#11922) · 241ad7b3
  youkaichao authored Jan 10, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  241ad7b3
- Replace "online inference" with "online serving" (#11923) · d85c47d6
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d85c47d6
- [platform] support pytorch custom op pluggable (#11328) · ef725fea
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  ef725fea
- [misc] remove python function call for custom activation op (#11885) · d907be7d
  cennn authored Jan 10, 2025
```
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  d907be7d
- [ci] fix gh200 tests (#11919) · d53575a5
  youkaichao authored Jan 10, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d53575a5
- [BUGFIX] Fix `UnspecifiedPlatform` package name (#11916) · 61af6332
  Kunshang Ji authored Jan 10, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  61af6332
- [Bugfix] Validate lora adapters to avoid crashing server (#11727) · ac2f3f7f
  Joe Runde authored Jan 10, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ac2f3f7f
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
- [Doc] Show default pooling method in a table (#11904) · 3de2b1ea
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3de2b1ea
- [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) · b844b99a
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b844b99a
- [Doc][5/N] Move Community and API Reference to the bottom (#11896) · c3cf54dd
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  c3cf54dd
09 Jan, 2025 7 commits
- [Docs] Add Modal to deployment frameworks (#11907) · 36f53035
  Charles Frye authored Jan 09, 2025
  
  36f53035
- [Misc] Provide correct Pixtral-HF chat template (#11891) · 9a228348
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9a228348
- [ci]try to fix flaky multi-step tests (#11894) · bd828722
  youkaichao authored Jan 09, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bd828722
- [platform] Allow platform specify attention backend (#11609) · 405eb8e3
  wangxiyuan authored Jan 09, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
```
  405eb8e3
- [Doc] Add model development API Reference (#11884) · 65097ca0
  Cyrus Leung authored Jan 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  65097ca0
- [Bugfix] fix beam search input errors and latency benchmark script (#11875) · 1d967acb
  Ye (Charlotte) Qi authored Jan 09, 2025
```
Signed-off-by: Ye Qi <yeq@meta.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>
```
  1d967acb
- [Bugfix] Override dunder methods of placeholder modules (#11882) · 0bd1ff43
  Cyrus Leung authored Jan 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0bd1ff43