Commits · 09473ee41c0a22c4d18936ea7eb2328071c19308 · myrfy001 / vllm_dsv4

"vscode:/vscode.git/clone" did not exist on "f91723cb9042a6fce9fc6ba7252b7856c9f506b9"

12 Apr, 2024 7 commits
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
- [Misc] Fix typo in scheduler.py (#4022) · d4ec9ffb
  Zhuohan Li authored Apr 12, 2024
  
  d4ec9ffb
- [Bugfix] fix type hint for py 3.8 (#4036) · 96b6a6d7
  youkaichao authored Apr 12, 2024
  
  96b6a6d7
- [Test] Test multiple attn backend for chunked prefill. (#4023) · 36729bac
  SangBin Cho authored Apr 13, 2024
  
  36729bac
- [Frontend][Core] Move `merge_async_iterators` to utils (#4026) · 7fd3949a
  Cyrus Leung authored Apr 12, 2024
  
  7fd3949a
- [Core] Support LoRA on quantized models (#4012) · 1096717a
  Jee Li authored Apr 12, 2024
  
  1096717a
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
11 Apr, 2024 13 commits
- [BugFix] Fix handling of stop strings and stop token ids (#3672) · e46a60aa
  Nick Hill authored Apr 11, 2024
  
  e46a60aa
- Add extra punica sizes to support bigger vocabs (#4015) · 1e96c334
  Antoni Baum authored Apr 11, 2024
  
  1e96c334
- Fix echo/logprob OpenAI completion bug (#3441) · 95e7d4a9
  Dylan Hawk authored Apr 11, 2024
```
Co-authored-by: Dylan Hawk <dylanwawk@gmail.com>
```
  95e7d4a9
- [Core] init_distributed_environment align with init_process_group(#4014) · 559eb852
  youkaichao authored Apr 11, 2024
```
[Core][Distributed] make init_distributed_environment compatible with init_process_group (#4014)
```
  559eb852
- [Core] Set `linear_weights` directly on the layer (#3977) · a10d3056
  Antoni Baum authored Apr 11, 2024
  
  a10d3056
- [Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better maintenance (#3824) · 8afca508
  bigPYJ1151 authored Apr 12, 2024
  
  8afca508
- punica fix-bgmv-kernel-640 (#4007) · 08ccee1e
  fuchen.ljl authored Apr 11, 2024
  
  08ccee1e
- [Kernel] Fused MoE Config for Mixtral 8x22 (#4002) · c1dc5471
  Roger Wang authored Apr 11, 2024
  
  c1dc5471
- [Doc][Installation] delete python setup.py develop (#3989) · f3d0bf75
  youkaichao authored Apr 10, 2024
  
  f3d0bf75
- [Misc] Add indirection layer for custom ops (#3913) · e9da5a40
  Kunshang Ji authored Apr 11, 2024
  
  e9da5a40
- [Test] Add xformer and flash attn tests (#3961) · e42df722
  SangBin Cho authored Apr 11, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  e42df722
- [Core][Model] torch.compile for layernorm in commandr (#3985) · caada5e5
  youkaichao authored Apr 10, 2024
```
[Core][Model] Use torch.compile to accelerate layernorm in commandr (#3985)
```
  caada5e5
- [Core][5/N] Fully working chunked prefill e2e (#3884) · 67b4221a
  SangBin Cho authored Apr 11, 2024
  
  67b4221a
10 Apr, 2024 12 commits
- [Core][Refactor] move parallel_utils into vllm/distributed (#3950) · 63e7176f
  youkaichao authored Apr 10, 2024
```
[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)
```
  63e7176f
- [Bugfix] handle hf_config with architectures == None (#3982) · 934d3662
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  934d3662
- [Doc] Fix getting stared to use publicly available model (#3963) · 92cd2e2f
  Frαnçois authored Apr 10, 2024
  
  92cd2e2f
- [Bugfix] Remove key sorting for `guided_json` parameter in OpenAi compatible Server (#3945) · e4c4072c
  Daniel E Marasco authored Apr 10, 2024
  
  e4c4072c
- [Doc] Add doc to state our model support policy (#3948) · e3539746
  youkaichao authored Apr 10, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  e3539746
- [Model][AMD] ROCm support for 256 head dims for Gemma (#3972) · 8b317c6d
  James Whedbee authored Apr 10, 2024
  
  8b317c6d
- [Bugfix][ROCm] Add numba to Dockerfile.rocm (#3962) · bd3c144e
  Woosuk Kwon authored Apr 10, 2024
  
  bd3c144e
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
- [Bugfix] Fix logits processor when prompt_logprobs is not None (#3899) · b3104b2a
  胡译文 authored Apr 10, 2024
  
  b3104b2a
- [Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable (#3955) · c2e00af5
  zhaotyer authored Apr 10, 2024
```
Co-authored-by: tianyi_zhao <tianyi.zhao@transwarp.io>
```
  c2e00af5
- [Benchmark] Add cpu options to bench scripts (#3915) · c013d32c
  Zedong Peng authored Apr 10, 2024
  
  c013d32c
- [Misc] Avoid loading incorrect LoRA config (#3777) · 11dd6ebb
  Jee Li authored Apr 10, 2024
  
  11dd6ebb
09 Apr, 2024 4 commits
- [ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (#3643) · 6c0b0451
  Juan Villamizar authored Apr 09, 2024
```
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6c0b0451
- [Bugfix] Fix KeyError on loading GPT-NeoX (#3925) · e23a43ae
  Junichi Sato authored Apr 10, 2024
  
  e23a43ae
- [Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable... · e7c7067b
  Cade Daniel authored Apr 09, 2024
```
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)
```
  e7c7067b
- [Core] separate distributed_init from worker (#3904) · 6d592eb4
  youkaichao authored Apr 09, 2024
  
  6d592eb4
08 Apr, 2024 4 commits
- [BugFix][Model] Fix commandr RoPE max_position_embeddings (#3919) · d036198e
  Roy authored Apr 09, 2024
  
  d036198e
- [Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations (#3782) · 59a6abf3
  Matt Wong authored Apr 08, 2024
  
  59a6abf3
- [Bugfix] Enable Proper `attention_bias` Usage in Llama Model Configuration (#3767) · bc0c0192
  Kiran R authored Apr 09, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  bc0c0192
- [Bugfix] Added Command-R GPTQ support (#3849) · f46864d6
  egortolmachev authored Apr 08, 2024
```
Co-authored-by: Egor Tolmachev <t333ga@gmail.com>
```
  f46864d6