Commits · 16a1cc9bb2b4bba82d78f329e5a89b44a5523ac8 · OpenDAS / vllm_cscc

"tests/vscode:/vscode.git/clone" did not exist on "2d4733ba2d5dee17c42df84d5f5b21a360e5a323"

04 Aug, 2024 5 commits
- [misc][distributed] improve libcudart.so finding (#7127) · 16a1cc9b
  youkaichao authored Aug 04, 2024
  
  16a1cc9b
- [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when... · b1c9aa3d
  Thomas Parnell authored Aug 04, 2024
```
[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  b1c9aa3d
- [Model]Refactor MiniCPMV (#7020) · 179a6a36
  Jee Jee Li authored Aug 04, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  179a6a36
- [core][misc] simply output processing with shortcut code path (#7117) · 83c644fe
  youkaichao authored Aug 04, 2024
  
  83c644fe
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 6 commits
- [Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080) · 825b0448
  Jeff Fialho authored Aug 03, 2024
```
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  825b0448
- [LoRA] ReplicatedLinear support LoRA (#7081) · 99d7cabd
  Jee Jee Li authored Aug 03, 2024
  
  99d7cabd
- [Bugfix] Fix block table for seqs that have prefix cache hits (#7018) · fb2c1c86
  Zach Zheng authored Aug 02, 2024
  
  fb2c1c86
- [Model] Refactor and decouple weight loading logic for InternVL2 model (#7067) · 0c25435d
  Isotr0py authored Aug 03, 2024
  
  0c25435d
- [Frontend] Factor out chat message parsing (#7055) · 8c025fa7
  Cyrus Leung authored Aug 03, 2024
  
  8c025fa7
- [ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73
  Robert Shaw authored Aug 02, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ed812a73
02 Aug, 2024 11 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Misc] Revive to use loopback address for driver IP (#7091) · 22e718ff
  Rui Qiao authored Aug 02, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  22e718ff
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
- [ci] set timeout for test_oot_registration.py (#7082) · 80694951
  youkaichao authored Aug 02, 2024
  
  80694951
- [Core] Comment out unused code in sampler (#7023) · db351863
  Peng Guanwen authored Aug 02, 2024
  
  db351863
- [cuda][misc] remove error_on_invalid_device_count_status (#7069) · 660dea12
  youkaichao authored Aug 02, 2024
  
  660dea12
- Fix tracing.py (#7065) · cf2a1a4d
  Bongwon Jang authored Aug 02, 2024
  
  cf2a1a4d
- [ci][distributed] try to fix pp test (#7054) · 25235779
  youkaichao authored Aug 01, 2024
  
  25235779
- [Kernel] Fix input for flashinfer prefill wrapper. (#7008) · 954f7305
  Lily Liu authored Aug 01, 2024
  
  954f7305
- [Performance] Optimize `get_seqs` (#7051) · 6ce01f30
  Woosuk Kwon authored Aug 01, 2024
  
  6ce01f30
01 Aug, 2024 12 commits
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
- [Models] Support Qwen model with PP (#6974) · fc912e08
  Murali Andoorveedu authored Aug 01, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  fc912e08
- [Bugfix] Lower gemma's unloaded_params exception to warning (#7002) · f4fd390f
  Michael Goin authored Aug 01, 2024
  
  f4fd390f
- [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992) · 2dd34371
  Isotr0py authored Aug 02, 2024
  
  2dd34371
- [CI/Build] Update PyTorch to 2.4.0 (#6951) · 7e0861bd
  Sage Moore authored Aug 01, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7e0861bd
- [core][scheduler] simplify and improve scheduler (#6867) · c8a7e932
  youkaichao authored Jul 31, 2024
  
  c8a7e932
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
- PP comm optimization: replace send with partial send + allgather (#6695) · 0437492e
  Aurick Qiao authored Jul 31, 2024
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  0437492e
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
- [Model] Pipeline parallel support for Qwen2 (#6924) · 1d2e7fb7
  xuyi authored Aug 01, 2024
  
  1d2e7fb7
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 6 commits
- Revert "[Frontend] Factor out code for running uvicorn" (#7012) · 7eb0cb4a
  Simon Mo authored Jul 31, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  7eb0cb4a
- [Misc] Add compressed-tensors to optimized quant list (#7006) · a0dce938
  Michael Goin authored Jul 31, 2024
  
  a0dce938
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
- [MISC] Introduce pipeline parallelism partition strategies (#6920) · bd700134
  Cody Yu authored Jul 31, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  bd700134
- [Model] use FusedMoE layer in Jamba (#6935) · 2ee8d3ba
  Avshalom Manevich authored Jul 31, 2024
  
  2ee8d3ba
- [Bugfix] Fix feature size calculation for LLaVA-NeXT (#6982) · daed30c4
  Cyrus Leung authored Jul 31, 2024
  
  daed30c4