Commits · 44dcb52e39ee6b2c9ef9e6497525e1e183c9d24b · OpenDAS / vllm_cscc

"cacheflow/model_executor/layers/attention.py" did not exist on "04e5acc08ed5b878225491bf62540ea10274fb29"

03 Aug, 2024 10 commits
- [ci][test] finalize fork_new_process_for_each_test (#7114) · 44dcb52e
  youkaichao authored Aug 03, 2024
  
  44dcb52e
- [CI] Temporarily turn off H100 performance benchmark (#7104) · 67d745cc
  Kuntai Du authored Aug 02, 2024
  
  67d745cc
- [LoRA] ReplicatedLinear support LoRA (#7081) · 99d7cabd
  Jee Jee Li authored Aug 03, 2024
  
  99d7cabd
- [Bugfix] Fix block table for seqs that have prefix cache hits (#7018) · fb2c1c86
  Zach Zheng authored Aug 02, 2024
  
  fb2c1c86
- [Model] Refactor and decouple weight loading logic for InternVL2 model (#7067) · 0c25435d
  Isotr0py authored Aug 03, 2024
  
  0c25435d
- [ci][distributed] disable ray dag tests (#7099) · a0d16456
  youkaichao authored Aug 02, 2024
  
  a0d16456
- [ci][distributed] merge distributed test commands (#7097) · 04e55834
  youkaichao authored Aug 02, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  04e55834
- [Frontend] Factor out chat message parsing (#7055) · 8c025fa7
  Cyrus Leung authored Aug 03, 2024
  
  8c025fa7
- [ci][distributed] shorten wait time if server hangs (#7098) · 69ea15e5
  youkaichao authored Aug 02, 2024
  
  69ea15e5
- [ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73
  Robert Shaw authored Aug 02, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ed812a73
02 Aug, 2024 14 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Misc] Revive to use loopback address for driver IP (#7091) · 22e718ff
  Rui Qiao authored Aug 02, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  22e718ff
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
- [CI/Build] Add support for Python 3.12 (#7035) · b482b9a5
  Michael Goin authored Aug 02, 2024
  
  b482b9a5
- [ci] set timeout for test_oot_registration.py (#7082) · 80694951
  youkaichao authored Aug 02, 2024
  
  80694951
- [Hardware][Intel CPU] Update torch 2.4.0 for CPU backend (#6931) · c16eaac5
  Jie Fu (傅杰) authored Aug 02, 2024
  
  c16eaac5
- [Core] Comment out unused code in sampler (#7023) · db351863
  Peng Guanwen authored Aug 02, 2024
  
  db351863
- [cuda][misc] remove error_on_invalid_device_count_status (#7069) · 660dea12
  youkaichao authored Aug 02, 2024
  
  660dea12
- Fix tracing.py (#7065) · cf2a1a4d
  Bongwon Jang authored Aug 02, 2024
  
  cf2a1a4d
- [ci][distributed] try to fix pp test (#7054) · 25235779
  youkaichao authored Aug 01, 2024
  
  25235779
- [mypy] Speed up mypy checking (#7056) · 3bb4b1e4
  Cyrus Leung authored Aug 02, 2024
  
  3bb4b1e4
- [Kernel] Fix input for flashinfer prefill wrapper. (#7008) · 954f7305
  Lily Liu authored Aug 01, 2024
  
  954f7305
- [Performance] Optimize `get_seqs` (#7051) · 6ce01f30
  Woosuk Kwon authored Aug 01, 2024
  
  6ce01f30
01 Aug, 2024 16 commits
- [CI/Build][Bugfix] Fix CUTLASS header-only line (#7034) · 6a11fdfb
  Tyler Michael Smith authored Aug 01, 2024
  
  6a11fdfb
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
- Update run-amd-test.sh (#7044) · 562e580a
  omkar kakarparthi authored Aug 01, 2024
  
  562e580a
- [Models] Support Qwen model with PP (#6974) · fc912e08
  Murali Andoorveedu authored Aug 01, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  fc912e08
- [Bugfix] Lower gemma's unloaded_params exception to warning (#7002) · f4fd390f
  Michael Goin authored Aug 01, 2024
  
  f4fd390f
- [CI/Build] Remove sparseml requirement from testing (#7037) · fb3db616
  Michael Goin authored Aug 01, 2024
  
  fb3db616
- [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992) · 2dd34371
  Isotr0py authored Aug 02, 2024
  
  2dd34371
- [CI/Build] Update PyTorch to 2.4.0 (#6951) · 7e0861bd
  Sage Moore authored Aug 01, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7e0861bd
- [Build/CI] Fixing Docker Hub quota issue. (#7043) · a72a424b
  Alexei-V-Ivanov-AMD authored Aug 01, 2024
  
  a72a424b
- [core][scheduler] simplify and improve scheduler (#6867) · c8a7e932
  youkaichao authored Jul 31, 2024
  
  c8a7e932
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
- PP comm optimization: replace send with partial send + allgather (#6695) · 0437492e
  Aurick Qiao authored Jul 31, 2024
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  0437492e
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
- [Model] Pipeline parallel support for Qwen2 (#6924) · 1d2e7fb7
  xuyi authored Aug 01, 2024
  
  1d2e7fb7
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343