Commits · d10f8e1d43bfb0656b6848ad0c681ecbdec812d6 · kecinstone / 2024pra-vllm

18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

14 Jan, 2024 1 commit
- [Minor] Optimize cuda graph memory usage (#2437) · 9f659bf0
  Roy authored Jan 15, 2024
  
  9f659bf0
08 Jan, 2024 1 commit
- Fix eager mode performance (#2377) · c8848191
  Woosuk Kwon authored Jan 08, 2024
  
  c8848191
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
20 Dec, 2023 1 commit
- Make _prepare_sample non-blocking and use pinned memory for input buffers (#2207) · 31bff691
  Hanzhi Zhou authored Dec 19, 2023
  
  31bff691
19 Dec, 2023 1 commit
- Add warning on CUDA graph memory usage (#2182) · 21d5daa4
  Woosuk Kwon authored Dec 18, 2023
  
  21d5daa4
17 Dec, 2023 2 commits
- Remove dependency on CuPy (#2152) · c3372e87
  Woosuk Kwon authored Dec 17, 2023
  
  c3372e87
- Optimize model execution with CUDA graph (#1926) · 37ca5581
  Woosuk Kwon authored Dec 16, 2023
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  37ca5581
13 Dec, 2023 1 commit
- [BugFix] Fix input positions for long context with sliding window (#2088) · f1c85201
  Woosuk Kwon authored Dec 13, 2023
  
  f1c85201
03 Dec, 2023 1 commit
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
30 Nov, 2023 1 commit
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2