Commits · 0b4997e05cf5aefcf648cae4d91c8e1d651008b9 · OpenDAS / vllm_cscc

25 Mar, 2024 6 commits
- [Bugfix] API stream returning two stops (#3450) · 0b4997e0
  Dylan Hawk authored Mar 25, 2024
```
Co-authored-by: Dylan Hawk <dylanwawk@gmail.com>
```
  0b4997e0
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [Core] Adding token ranks along with logprobs (#3516) · 819924e7
  Swapnil Parekh authored Mar 25, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
```
  819924e7
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Core] Refactor Attention Take 2 (#3462) · 925f3332
  Woosuk Kwon authored Mar 24, 2024
  
  925f3332
- [CI/Build] fix flaky test (#3602) · 837e1851
  youkaichao authored Mar 24, 2024
  
  837e1851
24 Mar, 2024 2 commits
- [CI] typo fix: is_hip --> is_hip() (#3595) · 8b268a46
  youkaichao authored Mar 24, 2024
  
  8b268a46
- [BugFix] 1D query fix for MoE models (#3597) · 41deac4a
  Nick Hill authored Mar 24, 2024
  
  41deac4a
22 Mar, 2024 4 commits
- [Core] Improve detokenization performance for prefill (#3469) · bfdb1ba5
  Antoni Baum authored Mar 22, 2024
```
Co-authored-by: MeloYang <meloyang05@gmail.com>
```
  bfdb1ba5
- Dynamic scheduler delay to improve ITL performance (#3279) · cf2f084d
  Thomas Parnell authored Mar 22, 2024
```
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  cf2f084d
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f
- [Bugfix][Model] Fix Qwen2 (#3554) · ea5f14e6
  Roy authored Mar 22, 2024
  
  ea5f14e6
20 Mar, 2024 5 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
- [Misc] Remove cache stream and cache events (#3461) · 5ee14494
  Woosuk Kwon authored Mar 20, 2024
  
  5ee14494
- [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b
  ElizaWszola authored Mar 20, 2024
```
[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9474e89b
18 Mar, 2024 1 commit
- [CI/Build] Fix Bad Import In Test (#3473) · 097aa0ea
  Robert Shaw authored Mar 18, 2024
  
  097aa0ea
16 Mar, 2024 3 commits
- Support arbitrary json_object in OpenAI and Context Free Grammar (#3211) · 120157fd
  Simon Mo authored Mar 16, 2024
  
  120157fd
- fix lint · ad50bf4b
  simon-mo authored Mar 15, 2024
  
  ad50bf4b
- Fixes the incorrect argument in the prefix-prefill test cases (#3246) · 3123f151
  Tao He authored Mar 16, 2024
  
  3123f151
15 Mar, 2024 1 commit
- Asynchronous tokenization (#2879) · fb96c1e9
  Antoni Baum authored Mar 15, 2024
  
  fb96c1e9
14 Mar, 2024 1 commit
- Fix assertion failure in Qwen 1.5 with prefix caching enabled (#3373) · 54be8a0b
  陈序 authored Mar 15, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  54be8a0b
13 Mar, 2024 4 commits
- Add batched RoPE kernel (#3095) · 7e9bd08f
  Terry authored Mar 13, 2024
  
  7e9bd08f
- Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when... · ae0ccb40
  Or Sharir authored Mar 13, 2024
```
Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when using Multi-LoRA. (#3350)
```
  ae0ccb40
- Add kernel for GeGLU with approximate GELU (#3337) · 602358f8
  Woosuk Kwon authored Mar 12, 2024
  
  602358f8
- Fixes #1556 double free (#3347) · 49a3c866
  Breno Faria authored Mar 13, 2024
  
  49a3c866
11 Mar, 2024 3 commits
- Add distributed model executor abstraction (#3191) · 4c922709
  Zhuohan Li authored Mar 11, 2024
  
  4c922709
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
- [BugFix] Fix get tokenizer when using ray (#3301) · 9e8744a5
  Roy authored Mar 11, 2024
  
  9e8744a5
10 Mar, 2024 1 commit
- Enhance lora tests with more layer and rank variations (#3243) · 0bba88df
  Terry authored Mar 09, 2024
  
  0bba88df
09 Mar, 2024 1 commit
- [Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103) · 8437bae6
  Cade Daniel authored Mar 08, 2024
  
  8437bae6
08 Mar, 2024 1 commit
- Fix auto prefix bug (#3239) · b35cc934
  ElizaWszola authored Mar 08, 2024
  
  b35cc934
07 Mar, 2024 2 commits
- Possible fix for conflict between Automated Prefix Caching (#2762) and... · 8cbba462
  jacobthebanana authored Mar 07, 2024
```
Possible fix for conflict between Automated Prefix Caching (#2762) and multi-LoRA support (#1804) (#3263)
```
  8cbba462
- Separate attention backends (#3005) · 2daf23ab
  Woosuk Kwon authored Mar 07, 2024
  
  2daf23ab
06 Mar, 2024 2 commits
- [Testing] Fix core tests (#3224) · a33ce60c
  Cade Daniel authored Mar 06, 2024
  
  a33ce60c
- [Tests] Add block manager and scheduler tests (#3108) · 24aecf42
  SangBin Cho authored Mar 06, 2024
  
  24aecf42
05 Mar, 2024 1 commit
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
04 Mar, 2024 2 commits
- Add health check, make async Engine more robust (#3015) · ff578cae
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ff578cae
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523