Commits · 0a24eb850a8ed690c5ae5f3cbffc10f2b0c1c42e · change / sglang · GitLab

28 Oct, 2024 1 commit
- Fix update_weights deadlock for DP (#1825) · 0a24eb85
  Byron Hsu authored Oct 28, 2024
  
  0a24eb85
27 Oct, 2024 3 commits
- Support token ids in `engine.generate` (#1820) · 6fcd6d7d
  Byron Hsu authored Oct 27, 2024
  
  6fcd6d7d
- Fix Triton decode kernel & ut (#1819) · c77762d5
  Ke Bao authored Oct 28, 2024
  
  c77762d5
- Add a watch dog thread (#1816) · 86fc0d79
  Lianmin Zheng authored Oct 27, 2024
  
  86fc0d79
26 Oct, 2024 3 commits
- Provide an argument to set the maximum batch size for cuda graph (#1809) · 2b809788
  Lianmin Zheng authored Oct 26, 2024
  
  2b809788
- Update ci workflows (#1804) · 6aa94b96
  Lianmin Zheng authored Oct 26, 2024
  
  6aa94b96
- [Fix] Fix --skip-tokenizer-init (#1798) · fb99aaa5
  Lianmin Zheng authored Oct 25, 2024
  
  fb99aaa5
25 Oct, 2024 5 commits
- Fix logprob in the overlapped mode (#1795) · e646c590
  Lianmin Zheng authored Oct 25, 2024
  
  e646c590
- Revert "Fix memory leak when doing chunked prefill" (#1797) · c555ce2c
  Lianmin Zheng authored Oct 25, 2024
  
  c555ce2c
- [Fix] Fix the log parsing in chunked prefill uni tests (#1794) · 40900bae
  Lianmin Zheng authored Oct 25, 2024
  
  40900bae
- Fix memory leak when doing chunked prefill (#1787) · a2f5e755
  Liangsheng Yin authored Oct 25, 2024
  
  a2f5e755
- Enhance the test case for chunked prefill (#1785) · 1701b0db
  Lianmin Zheng authored Oct 24, 2024
  
  1701b0db
23 Oct, 2024 3 commits
- Crash the server on warnings in CI (#1772) · 05b3bf5e
  Lianmin Zheng authored Oct 23, 2024
  
  05b3bf5e
- [API] add get memory pool size (#1760) · 2fce449b
  Ying Sheng authored Oct 23, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  2fce449b
- Fuse more ops & Simplify token mapping (#1758) · ad4125d1
  Lianmin Zheng authored Oct 22, 2024
  
  ad4125d1
21 Oct, 2024 4 commits
- Llama3.2 vision model support (#1551) · 94cde109
  Liangsheng Yin authored Oct 21, 2024
  
  94cde109
- Fix sliding window attention and gemma-2 unit tests in CI (#1746) · 00611286
  Lianmin Zheng authored Oct 21, 2024
  
  00611286
- Make token mapping non-blocking in the overlapped mode (#1740) · cf470fea
  Lianmin Zheng authored Oct 20, 2024
  
  cf470fea
- Add GLM-4 TextGeneration Model support for SGLang (#1736) · 45d5af24
  sixgod authored Oct 21, 2024
  
  45d5af24
20 Oct, 2024 5 commits
- [Bugfix] qwen2vl forward_extend (#1727) · 554fbf93
  yizhang2077 authored Oct 20, 2024
  
  554fbf93
- Split the overlapped version of TpModelWorkerClient into a separate file (#1726) · b48edff6
  Lianmin Zheng authored Oct 20, 2024
  
  b48edff6
- Temporarily skip this test_mixed_batch for QWen2VL (#1725) · 593b19f2
  Lianmin Zheng authored Oct 20, 2024
  
  593b19f2
- Support qwen2 vl model (#1721) · cbbc82b7
  Yineng Zhang authored Oct 19, 2024
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ispobock <ISPObaoke@163.com>
```
  cbbc82b7
- Update vllm to 0.6.3 (#1711) (#1720) · 8bee20f8
  Yineng Zhang authored Oct 19, 2024
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  8bee20f8
17 Oct, 2024 5 commits
- Add matched_stop token or str to distinguish between eos or stop str... · a95d5589
  Gleb Drozdov authored Oct 17, 2024
```
Add matched_stop token or str to distinguish between eos or stop str finish_reason generation (#1684)
```
  a95d5589
- Fix mixed batch for multi modal models (#1702) · d17d19e5
  Lianmin Zheng authored Oct 17, 2024
  
  d17d19e5
- Fix engine unit test (#1701) · dd3809fa
  Lianmin Zheng authored Oct 17, 2024
  
  dd3809fa
- Fix failed ci tests on long prompts; Better error messages for embedding models (#1700) · 7feba415
  Lianmin Zheng authored Oct 17, 2024
  
  7feba415
- Fix the failed unit tests (#1699) · 30ee3630
  Lianmin Zheng authored Oct 17, 2024
  
  30ee3630
16 Oct, 2024 3 commits
- Returning a per request metric for number of cached_tokens read (#1599) · ecb8bad2
  havetc authored Oct 16, 2024
  
  ecb8bad2
- Add a new event loop (#1677) · 9116b289
  Lianmin Zheng authored Oct 16, 2024
  
  9116b289
- Add OLMo model (#1676) · a5114b6f
  Jani Monoses authored Oct 16, 2024
  
  a5114b6f
14 Oct, 2024 3 commits
- Support double sparsity (#1459) · 061e5463
  Shuo Yang authored Oct 14, 2024
  
  061e5463
- Move filter_batch out of stream_output (#1663) · 0c1e8796
  Lianmin Zheng authored Oct 14, 2024
  
  0c1e8796
- Add a test case to test retract (#1662) · 869f1c02
  Lianmin Zheng authored Oct 13, 2024
  
  869f1c02
11 Oct, 2024 4 commits
- [Fix] Fix the style of test_large_max_new_tokens.py (#1638) · dafb6a52
  Lianmin Zheng authored Oct 11, 2024
  
  dafb6a52
- [engine] support async and streaming (#1614) · 862cd265
  Byron Hsu authored Oct 11, 2024
  
  862cd265
- Fix constrained decoding (#1634) · 5d09ca57
  Lianmin Zheng authored Oct 11, 2024
  
  5d09ca57
- Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py (#1631) · aba9eae4
  Lianmin Zheng authored Oct 11, 2024
  
  aba9eae4
08 Oct, 2024 1 commit
- [Engine] Fix generate hanging issue after the first call (#1606) · e8613df0
  Byron Hsu authored Oct 07, 2024
  
  e8613df0