Commits · 3a243095e5e7b655b63ab08fbd5936cb40850415 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "ab0f5e2823d31c150c01fc30c146b6ea801d6bdd"

25 Mar, 2024 11 commits
- Optimize `_get_ranks` in Sampler (#3623) · 3a243095
  Antoni Baum authored Mar 25, 2024
  
  3a243095
- [Feature] Add vision language model support. (#3042) · 64172a97
  xwjiang2010 authored Mar 25, 2024
  
  64172a97
- [Bugfix] API stream returning two stops (#3450) · 0b4997e0
  Dylan Hawk authored Mar 25, 2024
```
Co-authored-by: Dylan Hawk <dylanwawk@gmail.com>
```
  0b4997e0
- feat: implement the min_tokens sampling parameter (#3124) · c13ad1b7
  Travis Johnson authored Mar 25, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  c13ad1b7
- [Core] Adding token ranks along with logprobs (#3516) · 819924e7
  Swapnil Parekh authored Mar 25, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
```
  819924e7
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Bugfix] fix automatic prefix args and add log info (#3608) · e67c295b
  TianYu GUO authored Mar 25, 2024
  
  e67c295b
- [Core] Refactor Attention Take 2 (#3462) · 925f3332
  Woosuk Kwon authored Mar 24, 2024
  
  925f3332
- [Model] Add starcoder2 awq support (#3569) · b0dfa91d
  少年 authored Mar 25, 2024
  
  b0dfa91d
- [Bugfix] store lock file in tmp directory (#3578)" (#3599) · 56a8652f
  Woosuk Kwon authored Mar 24, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  56a8652f
- [BugFix] tensor.get_device() -> tensor.device (#3604) · 6d93d353
  Kunshang Ji authored Mar 25, 2024
  
  6d93d353
24 Mar, 2024 3 commits
- [BugFix] 1D query fix for MoE models (#3597) · 41deac4a
  Nick Hill authored Mar 24, 2024
  
  41deac4a
- [BugFix] Fix Falcon tied embeddings (#3590) · af9e5349
  Woosuk Kwon authored Mar 24, 2024
```
Co-authored-by: 44670 <44670@users.noreply.github.com>
```
  af9e5349
- [Misc] Fix BLOOM copyright notice (#3591) · 3c5ab9b8
  Woosuk Kwon authored Mar 23, 2024
  
  3c5ab9b8
23 Mar, 2024 1 commit
- [Bugfix] use SoftLockFile instead of LockFile (#3578) · 743a0b74
  kota-iizuka authored Mar 24, 2024
  
  743a0b74
22 Mar, 2024 5 commits
- [Core] Improve detokenization performance for prefill (#3469) · bfdb1ba5
  Antoni Baum authored Mar 22, 2024
```
Co-authored-by: MeloYang <meloyang05@gmail.com>
```
  bfdb1ba5
- Dynamic scheduler delay to improve ITL performance (#3279) · cf2f084d
  Thomas Parnell authored Mar 22, 2024
```
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  cf2f084d
- [BugFix] Some fixes for custom allreduce kernels (#2760) · f721096d
  Hanzhi Zhou authored Mar 21, 2024
  
  f721096d
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f
- [Bugfix][Model] Fix Qwen2 (#3554) · ea5f14e6
  Roy authored Mar 22, 2024
  
  ea5f14e6
21 Mar, 2024 6 commits
- [BugFix] gemma loading after quantization or LoRA. (#3553) · b7050ca7
  Taemin Lee authored Mar 22, 2024
  
  b7050ca7
- [Misc] Bump up transformers to v4.39.0 & Remove StarCoder2Config (#3551) · c188ecb0
  Woosuk Kwon authored Mar 21, 2024
```
Co-authored-by: Roy <jasonailu87@gmail.com>
Co-authored-by: Roger Meier <r.meier@siemens.com>
```
  c188ecb0
- [Misc][Log] Add log for tokenizer length not equal to vocabulary size (#3500) · 86573234
  Roy authored Mar 21, 2024
  
  86573234
- [🚀 Ready to be merged] Added support for Jais models (#3183) · 4c07dd28
  Lalit Pradhan authored Mar 21, 2024
  
  4c07dd28
- Fix 1D query issue from `_prune_hidden_states` (#3539) · 3bbff9e5
  SangBin Cho authored Mar 21, 2024
  
  3bbff9e5
- [PREFIX CACHING FOLLOW UP] OrderedDict-based evictor (#3431) · 6ebd02bd
  ElizaWszola authored Mar 21, 2024
```
Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Luka <luka@paperspace>
```
  6ebd02bd
20 Mar, 2024 6 commits
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [1/n] Triton sampling kernel (#3186) · 426ec4ec
  Antoni Baum authored Mar 20, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  426ec4ec
- [Misc] Remove cache stream and cache events (#3461) · 5ee14494
  Woosuk Kwon authored Mar 20, 2024
  
  5ee14494
- [Core] Add generic typing to `LRUCache` (#3511) · 4ad521d8
  Nick Hill authored Mar 20, 2024
  
  4ad521d8
- [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b
  ElizaWszola authored Mar 20, 2024
```
[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9474e89b
19 Mar, 2024 6 commits
- Use lru_cache for some environment detection utils (#3508) · 20478c4d
  Simon Mo authored Mar 19, 2024
  
  20478c4d
- Revert "[Core] Cache some utils" (#3507) · cc63d03f
  Simon Mo authored Mar 19, 2024
  
  cc63d03f
- [BugFix] Avoid initializing CUDA too early (#3487) · 7341c77d
  Nick Hill authored Mar 18, 2024
  
  7341c77d
- [Doc] Add docs about OpenAI compatible server (#3288) · ef65dcfa
  Simon Mo authored Mar 18, 2024
  
  ef65dcfa
- [Core] print error before deadlock (#3459) · 6a9c583e
  youkaichao authored Mar 18, 2024
  
  6a9c583e
- [Core] Cache some utils (#3474) · b37cdce2
  Antoni Baum authored Mar 18, 2024
  
  b37cdce2
18 Mar, 2024 1 commit
- [Core] Zero-copy asdict for InputMetadata (#3475) · 49eedea3
  Antoni Baum authored Mar 18, 2024
  
  49eedea3
17 Mar, 2024 1 commit
- [Misc] Use dataclass for InputMetadata (#3452) · abfc4f33
  Woosuk Kwon authored Mar 17, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  abfc4f33