Commits · 0258b7a94b08321ca01cf170f867b67c1920af87 · OpenDAS / vllm_cscc

10 Apr, 2024 1 commit
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
29 Mar, 2024 1 commit
- [BugFix][Frontend] Fix completion logprobs=0 error (#3731) · f510395b
  Roy authored Mar 30, 2024
  
  f510395b
25 Mar, 2024 3 commits
- Optimize `_get_ranks` in Sampler (#3623) · 3a243095
  Antoni Baum authored Mar 25, 2024
  
  3a243095
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [Core] Adding token ranks along with logprobs (#3516) · 819924e7
  Swapnil Parekh authored Mar 25, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
```
  819924e7
22 Mar, 2024 1 commit
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f
20 Mar, 2024 3 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
09 Mar, 2024 1 commit
- [Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103) · 8437bae6
  Cade Daniel authored Mar 08, 2024
  
  8437bae6
05 Mar, 2024 1 commit
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
04 Mar, 2024 1 commit
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523
28 Feb, 2024 1 commit
- [Neuron] Support inference with transformers-neuronx (#2569) · 3b7178cf
  Liangfu Chen authored Feb 28, 2024
  
  3b7178cf
21 Feb, 2024 1 commit
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

12 Jan, 2024 1 commit

Aligning `top_p` and `top_k` Sampling (#1885) · 218dc2cc

陈序 authored Jan 13, 2024

* Align top_p and top_k with huggingface

* remove _get_prompt_and_output_tokens

* rename _apply_top_p_top_k

* compare top_p top_k with hf

* fix test errors

218dc2cc

03 Jan, 2024 2 commits
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
- [Minor] Fix typo and remove unused code (#2305) · 91405610
  Roy authored Jan 03, 2024
  
  91405610
20 Dec, 2023 1 commit
- Remove Sampler copy stream (#2209) · bd29cf3d
  Antoni Baum authored Dec 20, 2023
  
  bd29cf3d
17 Dec, 2023 1 commit
- Make sampler less blocking (#1889) · a7347d9a
  Antoni Baum authored Dec 17, 2023
  
  a7347d9a
30 Nov, 2023 1 commit
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2
28 Nov, 2023 1 commit
- [FIX] Fix class naming (#1803) · 708e6c18
  Zhuohan Li authored Nov 28, 2023
  
  708e6c18
22 Nov, 2023 1 commit
- Fix repetition penalty aligned with huggingface (#1577) · de23687d
  ljss authored Nov 23, 2023
  
  de23687d
18 Nov, 2023 1 commit
- Support Min P Sampler (#1642) · e87557b0
  Roy authored Nov 18, 2023
  
  e87557b0
03 Nov, 2023 1 commit
- Added logits processor API to sampling params (#1469) · 555bdcc5
  Noam Gat authored Nov 03, 2023
  
  555bdcc5
30 Oct, 2023 1 commit
- Delay GPU->CPU sync in sampling (#1337) · 15f56323
  Antoni Baum authored Oct 30, 2023
  
  15f56323
29 Oct, 2023 1 commit
- Support repetition_penalty (#1424) · 69be658b
  ljss authored Oct 30, 2023
  
  69be658b
17 Oct, 2023 1 commit
- Change scheduler & input tensor shape (#1381) · c1376e0f
  Woosuk Kwon authored Oct 16, 2023
  
  c1376e0f
16 Oct, 2023 1 commit
- Implement prompt logprobs & Batched topk for computing logprobs (#1328) · 9d9072a0
  Zhuohan Li authored Oct 16, 2023
```
Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>
```
  9d9072a0
11 Oct, 2023 1 commit
- change the timing of sorting logits (#1309) · 91fce82c
  yhlskt23 authored Oct 11, 2023
  
  91fce82c
02 Oct, 2023 2 commits
- TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) · ba0bfd40
  Zhuohan Li authored Oct 02, 2023
  
  ba0bfd40
- [Minor] Fix type annotations (#1238) · 84e4e37d
  Woosuk Kwon authored Oct 02, 2023
  
  84e4e37d
24 Sep, 2023 1 commit
- [FIX] Simplify sampler logic (#1156) · f1878779
  Zhuohan Li authored Sep 23, 2023
  
  f1878779
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941
13 Sep, 2023 1 commit
- [FIX] Minor bug fixes (#1035) · f04908ca
  Zhuohan Li authored Sep 13, 2023
```
* [FIX] Minor bug fixes

* Address review comments
```
  f04908ca
05 Sep, 2023 1 commit
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
31 Aug, 2023 2 commits
- fix: bug fix when penalties are negative (#913) · e1122233
  Dong-Yong Lee authored Sep 01, 2023
```
Co-authored-by: dongyong-lee <dongyong.lee@navercorp.com>
```
  e1122233
- Improve _prune_hidden_states micro-benchmark (#707) · 28873a27
  Aman Gupta Karmani authored Aug 31, 2023
  
  28873a27
25 Aug, 2023 1 commit
- Set replacement=True in torch.multinomial (#858) · 94d2f598
  Woosuk Kwon authored Aug 25, 2023
  
  94d2f598