Commits · 201768d595388bb3cf24c2bf9f2519c136068f85 · OpenDAS / vllm_cscc

27 Nov, 2024 1 commit
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
25 Sep, 2024 1 commit
- [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) · 01b6f9e1
  Travis Johnson authored Sep 24, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  01b6f9e1
19 Sep, 2024 1 commit
- [Core] simplify logits resort in _apply_top_k_top_p (#8619) · e42c634a
  盏一 authored Sep 20, 2024
  
  e42c634a
17 Sep, 2024 1 commit
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
19 Aug, 2024 1 commit
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
14 Aug, 2024 1 commit
- Fix empty output when temp is too low (#2937) · c134a464
  Chang Su authored Aug 13, 2024
```
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c134a464
09 Aug, 2024 1 commit
- [Speculative decoding] [Multi-Step] decouple should_modify_greedy_probs_inplace (#6971) · 57b7be0e
  William Lin authored Aug 08, 2024
  
  57b7be0e
29 Jul, 2024 2 commits
- [Bugfix] Allow vllm to still work if triton is not installed. (#6786) · 9a7e2d05
  Thomas Parnell authored Jul 29, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  9a7e2d05
- [Core] Reduce unnecessary compute when logprobs=None (#6532) · db9e5708
  Peng Guanwen authored Jul 30, 2024
  
  db9e5708
26 Jul, 2024 1 commit
- [Core] Use array to speedup padding (#6779) · 89a84b0b
  Peng Guanwen authored Jul 26, 2024
  
  89a84b0b
17 Jul, 2024 1 commit
- [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) · e76466dd
  Alexander Matveev authored Jul 17, 2024
  
  e76466dd
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
13 May, 2024 1 commit
- [CORE] Improvement in ranks code (#4718) · a7be4d00
  Swapnil Parekh authored May 12, 2024
  
  a7be4d00
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
08 May, 2024 1 commit
- [Core] Optimize sampler get_logprobs (#4594) · d7740ea4
  SangBin Cho authored May 09, 2024
  
  d7740ea4
03 May, 2024 2 commits
- [Speculative decoding] Support target-model logprobs (#4378) · ab502751
  Cade Daniel authored May 03, 2024
  
  ab502751
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
29 Apr, 2024 1 commit
- [mypy][5/N] Support all typing on model executor (#4427) · df29793d
  SangBin Cho authored Apr 29, 2024
  
  df29793d
27 Apr, 2024 1 commit
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
26 Apr, 2024 1 commit
- [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) · 603ad848
  SangBin Cho authored Apr 26, 2024
  
  603ad848
23 Apr, 2024 1 commit
- [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) · 62b8aebc
  Cade Daniel authored Apr 23, 2024
  
  62b8aebc
10 Apr, 2024 1 commit
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
29 Mar, 2024 1 commit
- [BugFix][Frontend] Fix completion logprobs=0 error (#3731) · f510395b
  Roy authored Mar 30, 2024
  
  f510395b
25 Mar, 2024 3 commits
- Optimize `_get_ranks` in Sampler (#3623) · 3a243095
  Antoni Baum authored Mar 25, 2024
  
  3a243095
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [Core] Adding token ranks along with logprobs (#3516) · 819924e7
  Swapnil Parekh authored Mar 25, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
```
  819924e7
22 Mar, 2024 1 commit
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f
20 Mar, 2024 3 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
09 Mar, 2024 1 commit
- [Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103) · 8437bae6
  Cade Daniel authored Mar 08, 2024
  
  8437bae6
05 Mar, 2024 1 commit
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
04 Mar, 2024 1 commit
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523
28 Feb, 2024 1 commit
- [Neuron] Support inference with transformers-neuronx (#2569) · 3b7178cf
  Liangfu Chen authored Feb 28, 2024
  
  3b7178cf
21 Feb, 2024 1 commit
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa