Commits · 26422e477b2c485b6dc05c6515ad4250404a3e53 · OpenDAS / vllm_cscc

"vllm/perf/benchmark_serving.py" did not exist on "a62bc0109c3864b9dc770dc637e3acd332c730ea"

29 Mar, 2024 1 commit
- [Test] Make model tests run again and remove --forked from pytest (#3631) · 26422e47
  SangBin Cho authored Mar 29, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  26422e47
28 Mar, 2024 3 commits
- [Core] Support multi-node inference(eager and cuda graph) (#3686) · 515386ef
  Roy authored Mar 29, 2024
  
  515386ef
- [2/N] Chunked prefill data update (#3538) · b51c1cc9
  SangBin Cho authored Mar 29, 2024
  
  b51c1cc9
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c
27 Mar, 2024 3 commits
- feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) · 45b6ef65
  Roger Wang authored Mar 27, 2024
  
  45b6ef65
- [Core] remove cupy dependency (#3625) · 8f44facd
  youkaichao authored Mar 27, 2024
  
  8f44facd
- [Kernel] support non-zero cuda devices in punica kernels (#3636) · 566b57c5
  Jee Li authored Mar 27, 2024
  
  566b57c5
26 Mar, 2024 2 commits
- Enable more models to inference based on LoRA (#3382) · 8af890a8
  Jee Li authored Mar 26, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  8af890a8
- [Misc] Include matched stop string/token in responses (#2976) · dfeb2ecc
  Nick Hill authored Mar 25, 2024
```
Co-authored-by: Sahil Suneja <sahilsuneja@gmail.com>
```
  dfeb2ecc
25 Mar, 2024 8 commits
- [Feature] Add vision language model support. (#3042) · 64172a97
  xwjiang2010 authored Mar 25, 2024
  
  64172a97
- hotfix isort on logprobs ranks pr (#3622) · f408d05c
  Simon Mo authored Mar 25, 2024
  
  f408d05c
- [Bugfix] API stream returning two stops (#3450) · 0b4997e0
  Dylan Hawk authored Mar 25, 2024
```
Co-authored-by: Dylan Hawk <dylanwawk@gmail.com>
```
  0b4997e0
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [Core] Adding token ranks along with logprobs (#3516) · 819924e7
  Swapnil Parekh authored Mar 25, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
```
  819924e7
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Core] Refactor Attention Take 2 (#3462) · 925f3332
  Woosuk Kwon authored Mar 24, 2024
  
  925f3332
- [CI/Build] fix flaky test (#3602) · 837e1851
  youkaichao authored Mar 24, 2024
  
  837e1851
24 Mar, 2024 2 commits
- [CI] typo fix: is_hip --> is_hip() (#3595) · 8b268a46
  youkaichao authored Mar 24, 2024
  
  8b268a46
- [BugFix] 1D query fix for MoE models (#3597) · 41deac4a
  Nick Hill authored Mar 24, 2024
  
  41deac4a
22 Mar, 2024 4 commits
- [Core] Improve detokenization performance for prefill (#3469) · bfdb1ba5
  Antoni Baum authored Mar 22, 2024
```
Co-authored-by: MeloYang <meloyang05@gmail.com>
```
  bfdb1ba5
- Dynamic scheduler delay to improve ITL performance (#3279) · cf2f084d
  Thomas Parnell authored Mar 22, 2024
```
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  cf2f084d
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f
- [Bugfix][Model] Fix Qwen2 (#3554) · ea5f14e6
  Roy authored Mar 22, 2024
  
  ea5f14e6
20 Mar, 2024 5 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
- [Misc] Remove cache stream and cache events (#3461) · 5ee14494
  Woosuk Kwon authored Mar 20, 2024
  
  5ee14494
- [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b
  ElizaWszola authored Mar 20, 2024
```
[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9474e89b
18 Mar, 2024 1 commit
- [CI/Build] Fix Bad Import In Test (#3473) · 097aa0ea
  Robert Shaw authored Mar 18, 2024
  
  097aa0ea
16 Mar, 2024 3 commits
- Support arbitrary json_object in OpenAI and Context Free Grammar (#3211) · 120157fd
  Simon Mo authored Mar 16, 2024
  
  120157fd
- fix lint · ad50bf4b
  simon-mo authored Mar 15, 2024
  
  ad50bf4b
- Fixes the incorrect argument in the prefix-prefill test cases (#3246) · 3123f151
  Tao He authored Mar 16, 2024
  
  3123f151
15 Mar, 2024 1 commit
- Asynchronous tokenization (#2879) · fb96c1e9
  Antoni Baum authored Mar 15, 2024
  
  fb96c1e9
14 Mar, 2024 1 commit
- Fix assertion failure in Qwen 1.5 with prefix caching enabled (#3373) · 54be8a0b
  陈序 authored Mar 15, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  54be8a0b
13 Mar, 2024 4 commits
- Add batched RoPE kernel (#3095) · 7e9bd08f
  Terry authored Mar 13, 2024
  
  7e9bd08f
- Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when... · ae0ccb40
  Or Sharir authored Mar 13, 2024
```
Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when using Multi-LoRA. (#3350)
```
  ae0ccb40
- Add kernel for GeGLU with approximate GELU (#3337) · 602358f8
  Woosuk Kwon authored Mar 12, 2024
  
  602358f8
- Fixes #1556 double free (#3347) · 49a3c866
  Breno Faria authored Mar 13, 2024
  
  49a3c866
11 Mar, 2024 2 commits
- Add distributed model executor abstraction (#3191) · 4c922709
  Zhuohan Li authored Mar 11, 2024
  
  4c922709
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba