Commits · 4851c202bbfb5dadc5a3b3b4409d3f083f4e8e90 · OpenDAS / vllm_cscc · GitLab

11 Sep, 2024 1 commit
- [Speculative Decoding] Test refactor (#8317) · 775f00f8
  Lily Liu authored Sep 11, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  775f00f8
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 1 commit
- [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) · f205c098
  Jonas M. Kübler authored Aug 29, 2024
  
  f205c098
25 Aug, 2024 1 commit
- [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) · 1856aff4
  Nick Hill authored Aug 25, 2024
  
  1856aff4
22 Aug, 2024 2 commits
- [Bugfix] spec decode handle None entries in topk args in create_sequence_group_output (#7232) · cc0eaf12
  Travis Johnson authored Aug 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  cc0eaf12
- [Speculative Decoding] EAGLE Implementation with Top-1 proposer (#6830) · a3fce56b
  Abhinav Goyal authored Aug 22, 2024
  
  a3fce56b
20 Aug, 2024 1 commit
- [Speculative Decoding] Fixing hidden states handling in batch expansion (#7508) · 312f7612
  Abhinav Goyal authored Aug 20, 2024
  
  312f7612
19 Aug, 2024 1 commit
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
16 Aug, 2024 2 commits
- [Misc/Testing] Use `torch.testing.assert_close` (#7324) · 50b8d08d
  jon-chuang authored Aug 15, 2024
  
  50b8d08d
- [Misc] Add quantization config support for speculative model. (#7343) · b67ae00c
  shangmingc authored Aug 16, 2024
  
  b67ae00c
14 Aug, 2024 1 commit

[Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef

Wallas Henrique authored Aug 14, 2024


Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>

70b746ef

09 Aug, 2024 1 commit
- [Bugfix] Fix speculative decoding with MLPSpeculator with padded vocabulary (#7218) · 99b4cf5f
  Travis Johnson authored Aug 08, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  99b4cf5f
05 Aug, 2024 1 commit
- [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) · 82a1b1a8
  Cade Daniel authored Aug 05, 2024
  
  82a1b1a8
30 Jul, 2024 1 commit
- [BugFix] Fix use of per-request seed with pipeline parallel (#6698) · 5cf9254a
  Nick Hill authored Jul 30, 2024
  
  5cf9254a
24 Jul, 2024 2 commits
- [Bugfix] Fix speculative decode seeded test (#6743) · 2cf0df33
  Nick Hill authored Jul 24, 2024
  
  2cf0df33
- [SpecDecoding] Update MLPSpeculator CI tests to use smaller model (#6714) · c882a7f5
  Nick Hill authored Jul 24, 2024
  
  c882a7f5
21 Jul, 2024 1 commit
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
19 Jul, 2024 4 commits
- [Bugfix] [SpecDecode] AsyncMetricsCollector: update time since last collection (#6578) · f0bbfaf9
  Thomas Parnell authored Jul 19, 2024
  
  f0bbfaf9
- [Model] RowParallelLinear: pass bias to quant_method.apply (#6327) · a5314e86
  Thomas Parnell authored Jul 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a5314e86
- [BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369) · a921e863
  Woo-Yeon Lee authored Jul 19, 2024
  
  a921e863
- [Bugfix] Make spec. decode respect per-request seed. (#6034) · d4201e06
  Thomas Parnell authored Jul 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  d4201e06
17 Jul, 2024 1 commit
- [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) · e76466dd
  Alexander Matveev authored Jul 17, 2024
  
  e76466dd
16 Jul, 2024 1 commit
- [Misc] Log spec decode metrics (#6454) · 160e1d8c
  Cody Yu authored Jul 16, 2024
  
  160e1d8c
10 Jul, 2024 2 commits
- [Speculative Decoding] Enabling bonus token in speculative decoding for KV... · ae151d73
  sroy745 authored Jul 10, 2024
```
[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)
```
  ae151d73
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

02 Jul, 2024 5 commits
- [CORE] Quantized lm-head Framework (#4442) · ee93f4f9
  Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  ee93f4f9
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
- [Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) · 15aba081
  Sirej Dua authored Jul 02, 2024
```
Co-authored-by: Sirej Dua <sirej.dua@databricks.com>
Co-authored-by: Sirej Dua <Sirej Dua>
```
  15aba081
- [VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c
  xwjiang2010 authored Jul 02, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  98d6682c
- [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
  Alexander Matveev authored Jul 01, 2024
  
  3476ed08
01 Jul, 2024 1 commit
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
28 Jun, 2024 1 commit
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
26 Jun, 2024 1 commit
- [CI/Build] Add E2E tests for MLPSpeculator (#5791) · c2a8ac75
  Thomas Parnell authored Jun 26, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  c2a8ac75
25 Jun, 2024 1 commit
- [Speculative Decoding] Support draft model on different tensor-parallel size... · 2ce5d668
  Woo-Yeon Lee authored Jun 25, 2024
```
 [Speculative Decoding] Support draft model on different tensor-parallel size than target model (#5414)
```
  2ce5d668
21 Jun, 2024 1 commit

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

19 Jun, 2024 1 commit
- [Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654) · 78687504
  zifeitong authored Jun 19, 2024
  
  78687504
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
11 Jun, 2024 1 commit
- [Core][Doc] Default to multiprocessing for single-node distributed case (#5230) · 99dac099
  Nick Hill authored Jun 11, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  99dac099