Commits · ed812a73fae77bb520b739cfeaad36dbd61e2b03 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "d6fa1be3a8ef71fa16f74afdc5d07d27cbf725b1"

03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

02 Aug, 2024 11 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Misc] Revive to use loopback address for driver IP (#7091) · 22e718ff
  Rui Qiao authored Aug 02, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  22e718ff
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
- [ci] set timeout for test_oot_registration.py (#7082) · 80694951
  youkaichao authored Aug 02, 2024
  
  80694951
- [Core] Comment out unused code in sampler (#7023) · db351863
  Peng Guanwen authored Aug 02, 2024
  
  db351863
- [cuda][misc] remove error_on_invalid_device_count_status (#7069) · 660dea12
  youkaichao authored Aug 02, 2024
  
  660dea12
- Fix tracing.py (#7065) · cf2a1a4d
  Bongwon Jang authored Aug 02, 2024
  
  cf2a1a4d
- [ci][distributed] try to fix pp test (#7054) · 25235779
  youkaichao authored Aug 01, 2024
  
  25235779
- [Kernel] Fix input for flashinfer prefill wrapper. (#7008) · 954f7305
  Lily Liu authored Aug 01, 2024
  
  954f7305
- [Performance] Optimize `get_seqs` (#7051) · 6ce01f30
  Woosuk Kwon authored Aug 01, 2024
  
  6ce01f30
01 Aug, 2024 12 commits
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
- [Models] Support Qwen model with PP (#6974) · fc912e08
  Murali Andoorveedu authored Aug 01, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  fc912e08
- [Bugfix] Lower gemma's unloaded_params exception to warning (#7002) · f4fd390f
  Michael Goin authored Aug 01, 2024
  
  f4fd390f
- [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992) · 2dd34371
  Isotr0py authored Aug 02, 2024
  
  2dd34371
- [CI/Build] Update PyTorch to 2.4.0 (#6951) · 7e0861bd
  Sage Moore authored Aug 01, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7e0861bd
- [core][scheduler] simplify and improve scheduler (#6867) · c8a7e932
  youkaichao authored Jul 31, 2024
  
  c8a7e932
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
- PP comm optimization: replace send with partial send + allgather (#6695) · 0437492e
  Aurick Qiao authored Jul 31, 2024
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  0437492e
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
- [Model] Pipeline parallel support for Qwen2 (#6924) · 1d2e7fb7
  xuyi authored Aug 01, 2024
  
  1d2e7fb7
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 13 commits
- Revert "[Frontend] Factor out code for running uvicorn" (#7012) · 7eb0cb4a
  Simon Mo authored Jul 31, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  7eb0cb4a
- [Misc] Add compressed-tensors to optimized quant list (#7006) · a0dce938
  Michael Goin authored Jul 31, 2024
  
  a0dce938
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
- [MISC] Introduce pipeline parallelism partition strategies (#6920) · bd700134
  Cody Yu authored Jul 31, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  bd700134
- [Model] use FusedMoE layer in Jamba (#6935) · 2ee8d3ba
  Avshalom Manevich authored Jul 31, 2024
  
  2ee8d3ba
- [Bugfix] Fix feature size calculation for LLaVA-NeXT (#6982) · daed30c4
  Cyrus Leung authored Jul 31, 2024
  
  daed30c4
- [Bugfix] Clean up MiniCPM-V (#6939) · 2f4e108f
  Alphi authored Jul 31, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  2f4e108f
- Support W4A8 quantization for vllm (#5218) · 6512937d
  HandH1998 authored Jul 31, 2024
  
  6512937d
- [Bugfix] fix logit processor excceed vocab size issue (#6927) · c0644cf9
  Fei authored Jul 31, 2024
  
  c0644cf9
- [Bugfix][TPU] Set readonly=True for non-root devices (#6980) · 533d1932
  Woosuk Kwon authored Jul 31, 2024
  
  533d1932
- [CI/Build] Fix mypy errors (#6968) · 9f0e69b6
  Cyrus Leung authored Jul 31, 2024
  
  9f0e69b6
- [Bugfix] Fix broadcasting logic for `multi_modal_kwargs` (#6836) · f230cc2c
  Cyrus Leung authored Jul 31, 2024
  
  f230cc2c
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
30 Jul, 2024 3 commits
- [core][misc] improve free_finished_seq_groups (#6865) · 6ca8031e
  youkaichao authored Jul 30, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6ca8031e
- [Kernel] Remove scaled_fp8_quant kernel padding footgun (#6842) · d7a299ed
  Tyler Michael Smith authored Jul 30, 2024
  
  d7a299ed
- [BugFix] Fix use of per-request seed with pipeline parallel (#6698) · 5cf9254a
  Nick Hill authored Jul 30, 2024
  
  5cf9254a