Commits · 6d0fa73ece7d8e6694ce9a435b5204acaed20876 · change / sglang

"tests/models/unets/__init__.py" did not exist on "d03c9099bc690870f151035393484c3f5dea2d80"

18 Oct, 2024 2 commits
- Simplify flashinfer utilities (#1704) · 6d0fa73e
  Lianmin Zheng authored Oct 17, 2024
  
  6d0fa73e
- Fix regex and logprob conflicts when chunked prefilling (#1703) · 9e0dac1a
  Liangsheng Yin authored Oct 17, 2024
  
  9e0dac1a
17 Oct, 2024 12 commits
- Add matched_stop token or str to distinguish between eos or stop str... · a95d5589
  Gleb Drozdov authored Oct 17, 2024
```
Add matched_stop token or str to distinguish between eos or stop str finish_reason generation (#1684)
```
  a95d5589
- Fix mixed batch for multi modal models (#1702) · d17d19e5
  Lianmin Zheng authored Oct 17, 2024
  
  d17d19e5
- Fix engine unit test (#1701) · dd3809fa
  Lianmin Zheng authored Oct 17, 2024
  
  dd3809fa
- Fix failed ci tests on long prompts; Better error messages for embedding models (#1700) · 7feba415
  Lianmin Zheng authored Oct 17, 2024
  
  7feba415
- Fix the failed unit tests (#1699) · 30ee3630
  Lianmin Zheng authored Oct 17, 2024
  
  30ee3630
- ORJson. Faster Json serialization (#1694) · e5db40dc
  Michael Feil authored Oct 17, 2024
  
  e5db40dc
- feat: radix tree code optimize (#1697) · b1709305
  wxsm authored Oct 17, 2024
  
  b1709305
- Use SGLang imports for linear layer (#1696) · 5ab20cce
  Jani Monoses authored Oct 17, 2024
  
  5ab20cce
- Update the transformers version in CI (#1690) · 02f7f3e4
  Lianmin Zheng authored Oct 16, 2024
  
  02f7f3e4
- Add date to logging messages (#1623) (#1679) · 2782132b
  Zeng Zhongchao authored Oct 17, 2024
  
  2782132b
- Update README.md (#1689) · d19cc0b9
  Lianmin Zheng authored Oct 16, 2024
  
  d19cc0b9
- add orjson for jsonresponse (#1688) · b0facb33
  Michael Feil authored Oct 16, 2024
  
  b0facb33
16 Oct, 2024 7 commits
- Returning a per request metric for number of cached_tokens read (#1599) · ecb8bad2
  havetc authored Oct 16, 2024
  
  ecb8bad2
- Launch a thread to overlap CPU and GPU (#1687) · dbec2f18
  Lianmin Zheng authored Oct 16, 2024
  
  dbec2f18
- [Event] Add online meetup meeting link (#1686) · e4b367ba
  Ying Sheng authored Oct 16, 2024
  
  e4b367ba
- Fix srt dependency (#1685) · d10b933a
  Ke Bao authored Oct 16, 2024
  
  d10b933a
- Add a new event loop (#1677) · 9116b289
  Lianmin Zheng authored Oct 16, 2024
  
  9116b289
- Add OLMo model (#1676) · a5114b6f
  Jani Monoses authored Oct 16, 2024
  
  a5114b6f
- Fix filter_batch function call (#1681) · b6b40946
  Liangsheng Yin authored Oct 15, 2024
  
  b6b40946
15 Oct, 2024 4 commits
- Fix memory leak during abort (#1674) · f1088e0f
  Lianmin Zheng authored Oct 15, 2024
  
  f1088e0f
- Improve benchmark scripts (#1672) · 175afed3
  Lianmin Zheng authored Oct 14, 2024
  
  175afed3
- [Minor] Add some utility functions (#1671) · 4a292f67
  Lianmin Zheng authored Oct 14, 2024
  
  4a292f67
- [doc] improve engine doc and add to readme (#1670) · cd0be748
  Byron Hsu authored Oct 14, 2024
  
  cd0be748
14 Oct, 2024 9 commits
- [1/N] Remove `CacheConfig` import in all model files (#1658) · 56503d9b
  Byron Hsu authored Oct 14, 2024
  
  56503d9b
- Simplify chunked prefill (#1667) · 02bc9579
  Lianmin Zheng authored Oct 14, 2024
  
  02bc9579
- [Minor] Improve style (#1666) · 24f3e151
  Lianmin Zheng authored Oct 14, 2024
  
  24f3e151
- Fix unit test order to balance the tasks in CI (#1665) · 6790240c
  Lianmin Zheng authored Oct 14, 2024
  
  6790240c
- Support double sparsity (#1459) · 061e5463
  Shuo Yang authored Oct 14, 2024
  
  061e5463
- Move filter_batch out of stream_output (#1663) · 0c1e8796
  Lianmin Zheng authored Oct 14, 2024
  
  0c1e8796
- Add a test case to test retract (#1662) · 869f1c02
  Lianmin Zheng authored Oct 13, 2024
  
  869f1c02
- [Minor] Rename no_eos_trim to no_stop_trim (#1661) · 2725f8da
  Ying Sheng authored Oct 13, 2024
  
  2725f8da
- Add output_ids into ScheduleBatch (#1659) · da1ffed6
  Lianmin Zheng authored Oct 13, 2024
  
  da1ffed6
13 Oct, 2024 6 commits
- [Fix] fix eos trim inconsistency (#1650) · 48761171
  Ying Sheng authored Oct 13, 2024
  
  48761171
- [doc] Add engine section in backend.md (#1656) · c3f2fc5a
  Byron Hsu authored Oct 13, 2024
  
  c3f2fc5a
- Simplify the event loop and expose `--num-continuous-decode-steps` as an argument (#1652) · 7ee6c259
  Lianmin Zheng authored Oct 12, 2024
  
  7ee6c259
- Fix the batch_is_full check for jump-forward decoding (#1654) · 9610fcd4
  Lianmin Zheng authored Oct 12, 2024
  
  9610fcd4
- Add get_tokenizer function for Engine class (#1653) · 31fad29a
  Patrick Yi authored Oct 12, 2024
  
  31fad29a
- Add an option to disable penalizer (#1651) · 9da5a60b
  Lianmin Zheng authored Oct 12, 2024
  
  9da5a60b