Commits · b6cd903604a7439fcd082290ade83a02b164eca0 · change / sglang · GitLab

19 Oct, 2024 7 commits
- Update readme and workflow (#1716) · b6cd9036
  Lianmin Zheng authored Oct 19, 2024
  
  b6cd9036
- Release v0.3.4 (#1714) · 087257ea
  Lianmin Zheng authored Oct 19, 2024
  
  087257ea
- Update README.md (#1713) · 736f0402
  Lianmin Zheng authored Oct 19, 2024
  
  736f0402
- Fix the race condition in overlap mode (#1712) · 769bf11c
  Lianmin Zheng authored Oct 19, 2024
  
  769bf11c
- Fix `is_all_ready` for overlap copy (#1710) · 3db43d1b
  Lianmin Zheng authored Oct 18, 2024
  
  3db43d1b
- Simplify the nan detection and greedy check in sampler (#1709) · f0f8a769
  Lianmin Zheng authored Oct 18, 2024
  
  f0f8a769
- Skip unnecessary penalizer (#1707) · 2bcfba1b
  Lianmin Zheng authored Oct 18, 2024
  
  2bcfba1b
18 Oct, 2024 4 commits
- Add grouped free operations (#1706) · bc12d403
  Lianmin Zheng authored Oct 18, 2024
  
  bc12d403
- Add dtype for more operations (#1705) · 392f2863
  Lianmin Zheng authored Oct 18, 2024
  
  392f2863
- Simplify flashinfer utilities (#1704) · 6d0fa73e
  Lianmin Zheng authored Oct 17, 2024
  
  6d0fa73e
- Fix regex and logprob conflicts when chunked prefilling (#1703) · 9e0dac1a
  Liangsheng Yin authored Oct 17, 2024
  
  9e0dac1a
17 Oct, 2024 12 commits
- Add matched_stop token or str to distinguish between eos or stop str... · a95d5589
  Gleb Drozdov authored Oct 17, 2024
```
Add matched_stop token or str to distinguish between eos or stop str finish_reason generation (#1684)
```
  a95d5589
- Fix mixed batch for multi modal models (#1702) · d17d19e5
  Lianmin Zheng authored Oct 17, 2024
  
  d17d19e5
- Fix engine unit test (#1701) · dd3809fa
  Lianmin Zheng authored Oct 17, 2024
  
  dd3809fa
- Fix failed ci tests on long prompts; Better error messages for embedding models (#1700) · 7feba415
  Lianmin Zheng authored Oct 17, 2024
  
  7feba415
- Fix the failed unit tests (#1699) · 30ee3630
  Lianmin Zheng authored Oct 17, 2024
  
  30ee3630
- ORJson. Faster Json serialization (#1694) · e5db40dc
  Michael Feil authored Oct 17, 2024
  
  e5db40dc
- feat: radix tree code optimize (#1697) · b1709305
  wxsm authored Oct 17, 2024
  
  b1709305
- Use SGLang imports for linear layer (#1696) · 5ab20cce
  Jani Monoses authored Oct 17, 2024
  
  5ab20cce
- Update the transformers version in CI (#1690) · 02f7f3e4
  Lianmin Zheng authored Oct 16, 2024
  
  02f7f3e4
- Add date to logging messages (#1623) (#1679) · 2782132b
  Zeng Zhongchao authored Oct 17, 2024
  
  2782132b
- Update README.md (#1689) · d19cc0b9
  Lianmin Zheng authored Oct 16, 2024
  
  d19cc0b9
- add orjson for jsonresponse (#1688) · b0facb33
  Michael Feil authored Oct 16, 2024
  
  b0facb33
16 Oct, 2024 7 commits
- Returning a per request metric for number of cached_tokens read (#1599) · ecb8bad2
  havetc authored Oct 16, 2024
  
  ecb8bad2
- Launch a thread to overlap CPU and GPU (#1687) · dbec2f18
  Lianmin Zheng authored Oct 16, 2024
  
  dbec2f18
- [Event] Add online meetup meeting link (#1686) · e4b367ba
  Ying Sheng authored Oct 16, 2024
  
  e4b367ba
- Fix srt dependency (#1685) · d10b933a
  Ke Bao authored Oct 16, 2024
  
  d10b933a
- Add a new event loop (#1677) · 9116b289
  Lianmin Zheng authored Oct 16, 2024
  
  9116b289
- Add OLMo model (#1676) · a5114b6f
  Jani Monoses authored Oct 16, 2024
  
  a5114b6f
- Fix filter_batch function call (#1681) · b6b40946
  Liangsheng Yin authored Oct 15, 2024
  
  b6b40946
15 Oct, 2024 4 commits
- Fix memory leak during abort (#1674) · f1088e0f
  Lianmin Zheng authored Oct 15, 2024
  
  f1088e0f
- Improve benchmark scripts (#1672) · 175afed3
  Lianmin Zheng authored Oct 14, 2024
  
  175afed3
- [Minor] Add some utility functions (#1671) · 4a292f67
  Lianmin Zheng authored Oct 14, 2024
  
  4a292f67
- [doc] improve engine doc and add to readme (#1670) · cd0be748
  Byron Hsu authored Oct 14, 2024
  
  cd0be748
14 Oct, 2024 6 commits
- [1/N] Remove `CacheConfig` import in all model files (#1658) · 56503d9b
  Byron Hsu authored Oct 14, 2024
  
  56503d9b
- Simplify chunked prefill (#1667) · 02bc9579
  Lianmin Zheng authored Oct 14, 2024
  
  02bc9579
- [Minor] Improve style (#1666) · 24f3e151
  Lianmin Zheng authored Oct 14, 2024
  
  24f3e151
- Fix unit test order to balance the tasks in CI (#1665) · 6790240c
  Lianmin Zheng authored Oct 14, 2024
  
  6790240c
- Support double sparsity (#1459) · 061e5463
  Shuo Yang authored Oct 14, 2024
  
  061e5463
- Move filter_batch out of stream_output (#1663) · 0c1e8796
  Lianmin Zheng authored Oct 14, 2024
  
  0c1e8796