Commits · d4bf085ad064ba68a77862e2022f37c33a66e94a · OpenDAS / vllm_cscc

28 Aug, 2024 1 commit
- [mypy][CI/Build] Fix mypy errors (#7929) · 51f86bf4
  Cyrus Leung authored Aug 28, 2024
  
  51f86bf4
19 Aug, 2024 2 commits
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
16 Aug, 2024 1 commit
- [Misc/Testing] Use `torch.testing.assert_close` (#7324) · 50b8d08d
  jon-chuang authored Aug 15, 2024
  
  50b8d08d
09 Aug, 2024 1 commit
- [Speculative decoding] [Multi-Step] decouple should_modify_greedy_probs_inplace (#6971) · 57b7be0e
  William Lin authored Aug 08, 2024
  
  57b7be0e
30 Jul, 2024 1 commit
- [BugFix] Fix use of per-request seed with pipeline parallel (#6698) · 5cf9254a
  Nick Hill authored Jul 30, 2024
  
  5cf9254a
27 Jun, 2024 1 commit
- [CI/Build] Fix Args for `_get_logits_warper` in Sampler Test (#5922) · 736ed388
  Roger Wang authored Jun 27, 2024
  
  736ed388
19 Jun, 2024 1 commit
- [Bugfix] Added test for sampling repetition penalty bug. (#5659) · e5150f2c
  Thomas Parnell authored Jun 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  e5150f2c
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
09 May, 2024 1 commit
- [Misc] Remove unnecessary ModelRunner imports (#4703) · 190bc838
  Woosuk Kwon authored May 09, 2024
  
  190bc838
03 May, 2024 1 commit
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
27 Apr, 2024 1 commit
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
26 Apr, 2024 1 commit
- [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) · 603ad848
  SangBin Cho authored Apr 26, 2024
  
  603ad848
23 Apr, 2024 1 commit
- [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) · 62b8aebc
  Cade Daniel authored Apr 23, 2024
  
  62b8aebc
16 Apr, 2024 1 commit
- [Core] Refactor model loading code (#4097) · 69e1d2fb
  Antoni Baum authored Apr 16, 2024
  
  69e1d2fb
10 Apr, 2024 1 commit
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
25 Mar, 2024 3 commits
- hotfix isort on logprobs ranks pr (#3622) · f408d05c
  Simon Mo authored Mar 25, 2024
  
  f408d05c
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
20 Mar, 2024 2 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
21 Feb, 2024 1 commit
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
01 Feb, 2024 1 commit
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

14 Jan, 2024 1 commit
- [CI] Add Buildkite (#2355) · 6e01e8c1
  Simon Mo authored Jan 14, 2024
  
  6e01e8c1
12 Jan, 2024 1 commit

Aligning `top_p` and `top_k` Sampling (#1885) · 218dc2cc

陈序 authored Jan 13, 2024

* Align top_p and top_k with huggingface

* remove _get_prompt_and_output_tokens

* rename _apply_top_p_top_k

* compare top_p top_k with hf

* fix test errors

218dc2cc

03 Dec, 2023 1 commit
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
03 Nov, 2023 1 commit
- Added logits processor API to sampling params (#1469) · 555bdcc5
  Noam Gat authored Nov 03, 2023
  
  555bdcc5
16 Oct, 2023 1 commit
- Fix sampler test (#1379) · d3a5bd9f
  Woosuk Kwon authored Oct 16, 2023
  
  d3a5bd9f
02 Oct, 2023 1 commit
- TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) · ba0bfd40
  Zhuohan Li authored Oct 02, 2023
  
  ba0bfd40
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941