Commits · d619ae2d19c41d9aa8f68fa0e5e32cc410dc2522 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "aa3868ecfe65036349412ce54fa5b07cb545d836"

15 Apr, 2024 3 commits
- [Doc] Add better clarity for tensorizer usage (#4090) · d619ae2d
  Sanger Steel authored Apr 15, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  d619ae2d
- [Core] Simplifications to executor classes (#4071) · eb46fbfd
  Nick Hill authored Apr 15, 2024
  
  eb46fbfd
- [Misc][Minor] Fix CPU block num log in CPUExecutor. (#4088) · 0003e915
  Li, Jiang authored Apr 15, 2024
  
  0003e915
14 Apr, 2024 2 commits
- [Core] avoid too many cuda context by caching p2p test (#4021) · 2cd6b4f3
  youkaichao authored Apr 13, 2024
  
  2cd6b4f3
- [Frontend] [Core] feat: Add model loading using `tensorizer` (#3476) · 711a0002
  Sanger Steel authored Apr 13, 2024
  
  711a0002
13 Apr, 2024 4 commits
- [Bugfix] fix_small_bug_in_neuron_executor (#4051) · 0a430b4a
  zspo authored Apr 13, 2024
  
  0a430b4a
- [Bugfix] fix_log_time_in_metrics (#4050) · ec8e3c69
  zspo authored Apr 13, 2024
  
  ec8e3c69
- [Core][Distributed] improve logging for init dist (#4042) · 98afde19
  youkaichao authored Apr 13, 2024
  
  98afde19
- [Bugfix] More type hint fixes for py 3.8 (#4039) · 5c2e66e4
  Dylan Hawk authored Apr 12, 2024
  
  5c2e66e4
12 Apr, 2024 10 commits
- [Bugfix] Fix LoRA bug (#4032) · b8aacac3
  Jee Li authored Apr 13, 2024
  
  b8aacac3
- Fix triton compilation issue (#3984) · d04973ad
  Bellk17 authored Apr 12, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d04973ad
- [Core] fix custom allreduce default value (#4040) · fbb9d9ee
  youkaichao authored Apr 12, 2024
  
  fbb9d9ee
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
- [Misc] Fix typo in scheduler.py (#4022) · d4ec9ffb
  Zhuohan Li authored Apr 12, 2024
  
  d4ec9ffb
- [Bugfix] fix type hint for py 3.8 (#4036) · 96b6a6d7
  youkaichao authored Apr 12, 2024
  
  96b6a6d7
- [Test] Test multiple attn backend for chunked prefill. (#4023) · 36729bac
  SangBin Cho authored Apr 13, 2024
  
  36729bac
- [Frontend][Core] Move `merge_async_iterators` to utils (#4026) · 7fd3949a
  Cyrus Leung authored Apr 12, 2024
  
  7fd3949a
- [Core] Support LoRA on quantized models (#4012) · 1096717a
  Jee Li authored Apr 12, 2024
  
  1096717a
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
11 Apr, 2024 11 commits
- [BugFix] Fix handling of stop strings and stop token ids (#3672) · e46a60aa
  Nick Hill authored Apr 11, 2024
  
  e46a60aa
- Add extra punica sizes to support bigger vocabs (#4015) · 1e96c334
  Antoni Baum authored Apr 11, 2024
  
  1e96c334
- Fix echo/logprob OpenAI completion bug (#3441) · 95e7d4a9
  Dylan Hawk authored Apr 11, 2024
```
Co-authored-by: Dylan Hawk <dylanwawk@gmail.com>
```
  95e7d4a9
- [Core] init_distributed_environment align with init_process_group(#4014) · 559eb852
  youkaichao authored Apr 11, 2024
```
[Core][Distributed] make init_distributed_environment compatible with init_process_group (#4014)
```
  559eb852
- [Core] Set `linear_weights` directly on the layer (#3977) · a10d3056
  Antoni Baum authored Apr 11, 2024
  
  a10d3056
- [Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better maintenance (#3824) · 8afca508
  bigPYJ1151 authored Apr 12, 2024
  
  8afca508
- [Kernel] Fused MoE Config for Mixtral 8x22 (#4002) · c1dc5471
  Roger Wang authored Apr 11, 2024
  
  c1dc5471
- [Misc] Add indirection layer for custom ops (#3913) · e9da5a40
  Kunshang Ji authored Apr 11, 2024
  
  e9da5a40
- [Test] Add xformer and flash attn tests (#3961) · e42df722
  SangBin Cho authored Apr 11, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  e42df722
- [Core][Model] torch.compile for layernorm in commandr (#3985) · caada5e5
  youkaichao authored Apr 10, 2024
```
[Core][Model] Use torch.compile to accelerate layernorm in commandr (#3985)
```
  caada5e5
- [Core][5/N] Fully working chunked prefill e2e (#3884) · 67b4221a
  SangBin Cho authored Apr 11, 2024
  
  67b4221a
10 Apr, 2024 8 commits
- [Core][Refactor] move parallel_utils into vllm/distributed (#3950) · 63e7176f
  youkaichao authored Apr 10, 2024
```
[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)
```
  63e7176f
- [Bugfix] handle hf_config with architectures == None (#3982) · 934d3662
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  934d3662
- [Bugfix] Remove key sorting for `guided_json` parameter in OpenAi compatible Server (#3945) · e4c4072c
  Daniel E Marasco authored Apr 10, 2024
  
  e4c4072c
- [Model][AMD] ROCm support for 256 head dims for Gemma (#3972) · 8b317c6d
  James Whedbee authored Apr 10, 2024
  
  8b317c6d
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
- [Bugfix] Fix logits processor when prompt_logprobs is not None (#3899) · b3104b2a
  胡译文 authored Apr 10, 2024
  
  b3104b2a
- [Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable (#3955) · c2e00af5
  zhaotyer authored Apr 10, 2024
```
Co-authored-by: tianyi_zhao <tianyi.zhao@transwarp.io>
```
  c2e00af5
- [Misc] Avoid loading incorrect LoRA config (#3777) · 11dd6ebb
  Jee Li authored Apr 10, 2024
  
  11dd6ebb
09 Apr, 2024 2 commits
- [ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (#3643) · 6c0b0451
  Juan Villamizar authored Apr 09, 2024
```
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6c0b0451
- [Bugfix] Fix KeyError on loading GPT-NeoX (#3925) · e23a43ae
  Junichi Sato authored Apr 10, 2024
  
  e23a43ae