Commits · d98fa1e93dc9af557c2dd0aa80f8ba80a2fe65e5 · change / sglang

23 Nov, 2024 1 commit
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
19 Nov, 2024 1 commit
- Use cuda event wait and synchronization instead of busy waiting (#2089) · b7a065ea
  Lianmin Zheng authored Nov 19, 2024
  
  b7a065ea
18 Nov, 2024 1 commit
- Fix cuda illegal memory access in overlap mode (#2070) · 11668533
  Lianmin Zheng authored Nov 17, 2024
  
  11668533
15 Nov, 2024 1 commit
- Offline LLM Engine Benchmark Throughput (#1968) · f6dd6486
  zolinthecow authored Nov 14, 2024
```
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
```
  f6dd6486
11 Nov, 2024 1 commit
- Add engine encode (#1995) · ddeb9d42
  James Xu authored Nov 11, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  ddeb9d42
09 Nov, 2024 1 commit
- Clean up metrics code (#1972) · 9c939a3d
  Lianmin Zheng authored Nov 09, 2024
  
  9c939a3d
07 Nov, 2024 1 commit
- fix black in pre-commit (#1940) · c77c1e05
  Chayenne authored Nov 07, 2024
  
  c77c1e05
27 Oct, 2024 1 commit
- Support token ids in `engine.generate` (#1820) · 6fcd6d7d
  Byron Hsu authored Oct 27, 2024
  
  6fcd6d7d
17 Oct, 2024 1 commit
- Fix engine unit test (#1701) · dd3809fa
  Lianmin Zheng authored Oct 17, 2024
  
  dd3809fa
11 Oct, 2024 1 commit
- [engine] support async and streaming (#1614) · 862cd265
  Byron Hsu authored Oct 11, 2024
  
  862cd265
08 Oct, 2024 1 commit
- [Engine] Fix generate hanging issue after the first call (#1606) · e8613df0
  Byron Hsu authored Oct 07, 2024
  
  e8613df0
07 Oct, 2024 1 commit
- Provide an offline engine API (#1567) · 551a3a9d
  Byron Hsu authored Oct 06, 2024
  
  551a3a9d