Commits · b61bd98f907180c70f65e21505b3af6d1cc2bf36 · OpenDAS / vllm_cscc

"fern/pages/components/router/README.md" did not exist on "0d3ff44000b70666300eaa37a0340ffbc25a0984"

12 Sep, 2024 19 commits
- [Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for... · c1636945
  Roger Wang authored Sep 12, 2024
```
[Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425)
```
  c1636945
- [Bugfix] multi-step + flashinfer: ensure cuda graph compatible (#8427) · 01987725
  Alexander Matveev authored Sep 12, 2024
  
  01987725
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [Bugfix] Offline mode fix (#8376) · f2e263b8
  Joe Runde authored Sep 12, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  f2e263b8
- [BugFix] Fix Duplicate Assignment in Hermes2ProToolParser (#8423) · 1f0c75af
  Luis Vega authored Sep 12, 2024
  
  1f0c75af
- [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance (#8403) · 8a23e933
  WANGWEI authored Sep 13, 2024
  
  8a23e933
- [Model] Support multiple images for qwen-vl (#8247) · c6202dae
  Alex Brooks authored Sep 12, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6202dae
- [Bugfix] Fix InternVL2 inference with various num_patches (#8375) · e56bf277
  Isotr0py authored Sep 13, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e56bf277
- [Hotfix][VLM] Fixing max position embeddings for Pixtral (#8399) · 520ca380
  Roger Wang authored Sep 12, 2024
  
  520ca380
- [torch.compile] hide slicing under custom op for inductor (#8384) · 7de49aa8
  youkaichao authored Sep 12, 2024
  
  7de49aa8
- [Misc] Use RoPE cache for MRoPE (#8396) · 42ffba11
  Woosuk Kwon authored Sep 11, 2024
  
  42ffba11
- [Misc] Raise error when using encoder/decoder model with cpu backend (#8355) · 295c4730
  Kevin Lin authored Sep 12, 2024
  
  295c4730
- [Gemma2] add bitsandbytes support for Gemma2 (#8338) · 1bf2dd9d
  Blueyo0 authored Sep 12, 2024
  
  1bf2dd9d
- [Bugfix]: Fix the logic for deciding if tool parsing is used (#8366) · 5a60699c
  tomeras91 authored Sep 12, 2024
  
  5a60699c
- Fix the AMD weight loading tests (#8390) · b6c75e1c
  Michael Goin authored Sep 11, 2024
  
  b6c75e1c
- [TPU] Use Ray for default distributed backend (#8389) · b71c956d
  Woosuk Kwon authored Sep 11, 2024
  
  b71c956d
- [misc] remove engine_use_ray (#8126) · f842a7af
  youkaichao authored Sep 11, 2024
  
  f842a7af
- [MISC] Dump model runner inputs when crashing (#8305) · a65cb160
  Cody Yu authored Sep 11, 2024
  
  a65cb160
11 Sep, 2024 12 commits
- Bump version to v0.6.1 (#8379) · 3fd2b0d2
  Simon Mo authored Sep 11, 2024
  
  3fd2b0d2
- Pixtral (#8377) · d394787e
  Patrick von Platen authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d394787e
- [Misc] Move device options to a single place (#8322) · 8baa4549
  Aarni Koskela authored Sep 11, 2024
  
  8baa4549
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Bugfix] Add missing attributes in mistral tokenizer (#8364) · 7015417f
  Cyrus Leung authored Sep 12, 2024
  
  7015417f
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347) · cea95dfb
  Pooya Davoodi authored Sep 10, 2024
  
  cea95dfb
- [model] Support for Llava-Next-Video model (#7559) · 6a512a00
  Yangshen⚡Deng authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6a512a00
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
- [Bugfix] Fix InternVL2 vision embeddings process with pipeline parallel (#8299) · 1230263e
  Isotr0py authored Sep 11, 2024
  
  1230263e
- [Misc] Skip loading extra bias for Qwen2-MOE GPTQ models (#8329) · e497b8ae
  Jee Jee Li authored Sep 11, 2024
  
  e497b8ae
10 Sep, 2024 8 commits
- [Core/Bugfix] pass VLLM_ATTENTION_BACKEND to ray workers (#8172) · 1d5e397a
  William Lin authored Sep 10, 2024
  
  1d5e397a
- [Bugfix] lookahead block table with cuda graph max capture (#8340) · 22f3a4bc
  Alexander Matveev authored Sep 10, 2024
```
[Bugfix] Ensure multistep lookahead allocation is compatible with cuda graph max capture (#8340)
```
  22f3a4bc
- [MISC] Keep chunked prefill enabled by default with long context when prefix... · b1f3e189
  Cody Yu authored Sep 10, 2024
```
[MISC] Keep chunked prefill enabled by default with long context when prefix caching is enabled (#8342)
```
  b1f3e189
- [Misc] remove peft as dependency for prompt models (#8162) · 04e7c4e7
  Prashant Gupta authored Sep 10, 2024
  
  04e7c4e7
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
- [Frontend] Clean up type annotations for mistral tokenizer (#8314) · 8c054b7a
  Cyrus Leung authored Sep 11, 2024
  
  8c054b7a
- [Bugfix] Fix missing `post_layernorm` in CLIP (#8155) · da1a844e
  Cyrus Leung authored Sep 10, 2024
  
  da1a844e
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
09 Sep, 2024 1 commit
- [Misc] GPTQ Activation Ordering (#8135) · c7cb5c33
  Kyle Sayers authored Sep 09, 2024
  
  c7cb5c33