Commits · e35397468f36a857b8d2b7d92a472265e1c500cc · OpenDAS / vllm_cscc

"examples/offline_inference/offline_inference.py" did not exist on "655a5e48df3937bf793add53aa95ce0c992a24c6"

10 Apr, 2024 8 commits
- [Doc] Add doc to state our model support policy (#3948) · e3539746
  youkaichao authored Apr 10, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  e3539746
- [Model][AMD] ROCm support for 256 head dims for Gemma (#3972) · 8b317c6d
  James Whedbee authored Apr 10, 2024
  
  8b317c6d
- [Bugfix][ROCm] Add numba to Dockerfile.rocm (#3962) · bd3c144e
  Woosuk Kwon authored Apr 10, 2024
  
  bd3c144e
- [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876) · 0258b7a9
  Travis Johnson authored Apr 10, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0258b7a9
- [Bugfix] Fix logits processor when prompt_logprobs is not None (#3899) · b3104b2a
  胡译文 authored Apr 10, 2024
  
  b3104b2a
- [Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable (#3955) · c2e00af5
  zhaotyer authored Apr 10, 2024
```
Co-authored-by: tianyi_zhao <tianyi.zhao@transwarp.io>
```
  c2e00af5
- [Benchmark] Add cpu options to bench scripts (#3915) · c013d32c
  Zedong Peng authored Apr 10, 2024
  
  c013d32c
- [Misc] Avoid loading incorrect LoRA config (#3777) · 11dd6ebb
  Jee Li authored Apr 10, 2024
  
  11dd6ebb
09 Apr, 2024 4 commits
- [ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (#3643) · 6c0b0451
  Juan Villamizar authored Apr 09, 2024
```
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6c0b0451
- [Bugfix] Fix KeyError on loading GPT-NeoX (#3925) · e23a43ae
  Junichi Sato authored Apr 10, 2024
  
  e23a43ae
- [Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable... · e7c7067b
  Cade Daniel authored Apr 09, 2024
```
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)
```
  e7c7067b
- [Core] separate distributed_init from worker (#3904) · 6d592eb4
  youkaichao authored Apr 09, 2024
  
  6d592eb4
08 Apr, 2024 5 commits
- [BugFix][Model] Fix commandr RoPE max_position_embeddings (#3919) · d036198e
  Roy authored Apr 09, 2024
  
  d036198e
- [Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations (#3782) · 59a6abf3
  Matt Wong authored Apr 08, 2024
  
  59a6abf3
- [Bugfix] Enable Proper `attention_bias` Usage in Llama Model Configuration (#3767) · bc0c0192
  Kiran R authored Apr 09, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  bc0c0192
- [Bugfix] Added Command-R GPTQ support (#3849) · f46864d6
  egortolmachev authored Apr 08, 2024
```
Co-authored-by: Egor Tolmachev <t333ga@gmail.com>
```
  f46864d6
- [Model] add minicpm (#3893) · b4543c8f
  ywfang authored Apr 08, 2024
  
  b4543c8f
07 Apr, 2024 3 commits
- [Bugfix] Fix Llava inference with Tensor Parallelism. (#3883) · 0ce0539d
  Isotr0py authored Apr 07, 2024
  
  0ce0539d
- [Core] latency optimization (#3890) · 2f192835
  youkaichao authored Apr 06, 2024
  
  2f192835
- [Core] enable out-of-tree model register (#3871) · 95baec82
  youkaichao authored Apr 06, 2024
  
  95baec82
06 Apr, 2024 1 commit
- [CI/Benchmark] add more iteration and use median for robust latency benchmark (#3889) · e4be7d70
  youkaichao authored Apr 06, 2024
  
  e4be7d70
05 Apr, 2024 9 commits
- [Bugfix] Fix incorrect output on OLMo models in Tensor Parallelism (#3869) · 54951ac4
  Isotr0py authored Apr 06, 2024
  
  54951ac4
- [Chunked Prefill][4/n] Chunked prefill scheduler. (#3853) · 18de8834
  SangBin Cho authored Apr 06, 2024
  
  18de8834
- Add option to completion API to truncate prompt tokens (#3144) · 1d7c940d
  Thomas Parnell authored Apr 05, 2024
  
  1d7c940d
- [Misc] Define common requirements (#3841) · cfaf49a1
  Woosuk Kwon authored Apr 05, 2024
  
  cfaf49a1
- [Bugfix] Fixing requirements.txt (#3865) · 9edec652
  Noam Gat authored Apr 05, 2024
  
  9edec652
- [Misc] Fix linter issues in examples/fp8/quantizer/quantize.py (#3864) · e0dd4d35
  Cade Daniel authored Apr 04, 2024
  
  e0dd4d35
- [Misc] Add pytest marker to opt-out of global test cleanup (#3863) · e5043a3e
  Cade Daniel authored Apr 04, 2024
  
  e5043a3e
- [CI/Build] refactor dockerfile & fix pip cache · d03d64fd
  youkaichao authored Apr 04, 2024
```
[CI/Build] fix pip cache with vllm_nccl & refactor dockerfile to build wheels (#3859)
```
  d03d64fd
- [Doc]Add asynchronous engine arguments to documentation. (#3810) · 78107fa0
  Sean Gallen authored Apr 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  78107fa0
04 Apr, 2024 10 commits
- [Core] improve robustness of pynccl (#3860) · c391e4b6
  youkaichao authored Apr 04, 2024
  
  c391e4b6
- [Model] Cohere CommandR+ (#3829) · 9117f892
  Saurabh Dash authored Apr 05, 2024
  
  9117f892
- [Hardware][CPU] Update cpu torch to match default of 2.2.1 (#3854) · db2a6a41
  Michael Goin authored Apr 04, 2024
  
  db2a6a41
- [Core] manage nccl via a pypi package & upgrade to pt 2.2.1 (#3805) · ca81ff51
  youkaichao authored Apr 04, 2024
  
  ca81ff51
- [Benchmark] Refactor sample_requests in benchmark_throughput (#3613) · b7782002
  TianYu GUO authored Apr 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b7782002
- [Bugfix] Fix args in benchmark_serving (#3836) · 819a309c
  Chang Su authored Apr 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  819a309c
- [Core] [Frontend] Make detokenization optional (#3749) · aabe8f40
  Matthias Gerstgrasser authored Apr 03, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  aabe8f40
- [Bugfix] Add kv_scale input parameter to CPU backend (#3840) · 498eb5cf
  Woosuk Kwon authored Apr 03, 2024
  
  498eb5cf
- [Core] Enable hf_transfer by default if available (#3817) · 537ee25f
  Michael Feil authored Apr 03, 2024
  
  537ee25f
- [BugFix] Pass tokenizer_config to local_tokenizer_group (#3754) · 294f8f66
  Tao He authored Apr 04, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  294f8f66