Commits · 9b945daaf1ce03b8b02d68b37c59baf28566b535 · kecinstone / 2024pra-vllm

".github/workflows/build-rockchip-23.05.yml" did not exist on "d8bf7cbf71d4d8c935292ac271de0efea41c1271"

23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

22 Jan, 2024 1 commit
- [Speculative decoding 2/9] Multi-step worker for draft model (#2424) · 18bfcdd0
  Cade Daniel authored Jan 21, 2024
  
  18bfcdd0
19 Jan, 2024 1 commit
- Simplify broadcast logic for control messages (#2501) · ef9b636e
  Zhuohan Li authored Jan 19, 2024
  
  ef9b636e
18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

14 Jan, 2024 1 commit
- [Minor] Optimize cuda graph memory usage (#2437) · 9f659bf0
  Roy authored Jan 15, 2024
  
  9f659bf0
08 Jan, 2024 1 commit
- Fix eager mode performance (#2377) · c8848191
  Woosuk Kwon authored Jan 08, 2024
  
  c8848191
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
20 Dec, 2023 1 commit
- Make _prepare_sample non-blocking and use pinned memory for input buffers (#2207) · 31bff691
  Hanzhi Zhou authored Dec 19, 2023
  
  31bff691
19 Dec, 2023 1 commit
- Add warning on CUDA graph memory usage (#2182) · 21d5daa4
  Woosuk Kwon authored Dec 18, 2023
  
  21d5daa4
17 Dec, 2023 2 commits
- Remove dependency on CuPy (#2152) · c3372e87
  Woosuk Kwon authored Dec 17, 2023
  
  c3372e87
- Optimize model execution with CUDA graph (#1926) · 37ca5581
  Woosuk Kwon authored Dec 16, 2023
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  37ca5581
13 Dec, 2023 1 commit
- [BugFix] Fix input positions for long context with sliding window (#2088) · f1c85201
  Woosuk Kwon authored Dec 13, 2023
  
  f1c85201
03 Dec, 2023 1 commit
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
30 Nov, 2023 1 commit
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2