Commits · a2486eb58fa32661965bf66034625155e87cfc05 · zhaoyu6 / sglang · GitLab

08 Dec, 2024 2 commits
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
06 Dec, 2024 2 commits
- Fix the overlap for xgrammar (#2377) · 0e7409ad
  Lianmin Zheng authored Dec 06, 2024
  
  0e7409ad
- Add more support for intel Gaudi accelerators (#2357) · 37ee906f
  Qun Yang authored Dec 06, 2024
  
  37ee906f
02 Dec, 2024 3 commits
- Update model_loader deps and qqq quantization deps (#2220) (#2318) · 85e1a6f3
  Yineng Zhang authored Dec 02, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  85e1a6f3
- [Fix] Fix the padded hash value for image tokens (#2309) · 3c79ad35
  Lianmin Zheng authored Dec 01, 2024
  
  3c79ad35
- Online weight updates from torch.distributed (#2279) · 983bfcf3
  Chayenne authored Dec 01, 2024
  
  983bfcf3
01 Dec, 2024 1 commit
- Fix chunked prefill when ignore eos (#2290) · 5f12f0e7
  Liangsheng Yin authored Dec 01, 2024
  
  5f12f0e7
30 Nov, 2024 3 commits
- Add get weights by parameter name for llama (#2266) · 7d1485d3
  Chayenne authored Nov 29, 2024
  
  7d1485d3
- udate weights from disk (#2265) · 7d5d1d3d
  Chayenne authored Nov 29, 2024
  
  7d5d1d3d
- Fix the default chunked prefill size (#2268) · 94e167ea
  Lianmin Zheng authored Nov 29, 2024
  
  94e167ea
29 Nov, 2024 3 commits
- [Minor] fix the style for multimodal models (#2257) · afe1e465
  Lianmin Zheng authored Nov 29, 2024
  
  afe1e465
- Fix hash collision for multi modal models (#2256) · f50a6cf4
  Lianmin Zheng authored Nov 29, 2024
  
  f50a6cf4
- Simplify tokenizer manager (#2254) · fe97a2d4
  Lianmin Zheng authored Nov 29, 2024
  
  fe97a2d4
28 Nov, 2024 3 commits
- Fix memory leak during abort (#2238) · b2ccf36d
  Lianmin Zheng authored Nov 28, 2024
  
  b2ccf36d
- Crash the server correctly during error (#2231) · d4fc1a70
  Lianmin Zheng authored Nov 28, 2024
  
  d4fc1a70
- Disable overlap scheduler for multimodal models (#2235) · fb915bd1
  Lianmin Zheng authored Nov 27, 2024
  
  fb915bd1
27 Nov, 2024 5 commits
- Rename DP_RANK to SGLANG_DP_RANK (#2218) · 2a02185c
  Lianmin Zheng authored Nov 27, 2024
  
  2a02185c
- Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2222) · fb6e04a0
  Lianmin Zheng authored Nov 27, 2024
  
  fb6e04a0
- Revert "Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default" (#2221) · 6997e28f
  Lianmin Zheng authored Nov 27, 2024
  
  6997e28f
- Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2217) · a0e58740
  Lianmin Zheng authored Nov 27, 2024
  
  a0e58740
- [feat] Support session control for vision language models (#2210) · 37c8a576
  Ying Sheng authored Nov 27, 2024
  
  37c8a576
26 Nov, 2024 2 commits
- [CI] Minor fix for CI (#2187) · 1605ae12
  Lianmin Zheng authored Nov 25, 2024
  
  1605ae12
- Input_embeds support (#2052) · 1aea19f6
  Rin Intachuen authored Nov 25, 2024
  
  1aea19f6
25 Nov, 2024 3 commits
- [Performance]: Process affinity to CPU cores with multiple sockets support (#2171) · 10189d08
  HAI authored Nov 25, 2024
  
  10189d08
- [feat] Refactor session control interface and add CI (#2173) · e1e595d7
  Ying Sheng authored Nov 25, 2024
  
  e1e595d7
- Allow overwrite flashinfer use_tensorcore (#2169) · 8e1adb84
  Lianmin Zheng authored Nov 24, 2024
  
  8e1adb84
24 Nov, 2024 3 commits
- Fix mixed chunked prefill in overlap mode (#2158) · 731146f6
  Lianmin Zheng authored Nov 24, 2024
  
  731146f6
- Update CI threshold & Improve code style (#2159) · 5652c565
  Lianmin Zheng authored Nov 24, 2024
  
  5652c565
- Simplify batch update (#2154) · c211e7b6
  Lianmin Zheng authored Nov 24, 2024
  
  c211e7b6
23 Nov, 2024 2 commits
- fix dp_rank env (#2144) · 52f58fc4
  Byron Hsu authored Nov 23, 2024
  
  52f58fc4
- Fix dp print message (#2138) · 751c3a03
  Lianmin Zheng authored Nov 23, 2024
  
  751c3a03
22 Nov, 2024 6 commits
- Revert "Only stream output on tp rank 0" (#2130) · 66d4859a
  Lianmin Zheng authored Nov 22, 2024
  
  66d4859a
- Only stream output on tp rank 0 (#2124) · e1b63624
  Lianmin Zheng authored Nov 22, 2024
  
  e1b63624
- Expose max total num tokens from Runtime & Engine API (#2092) · c35cd1f8
  Henry Hyeonmok Ko authored Nov 22, 2024
  
  c35cd1f8
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- Fix #2037 - Context length check does not take into out pad tokens for visual models (#2106) · 8048c28c
  Jake Poznanski authored Nov 21, 2024
  
  8048c28c
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
20 Nov, 2024 2 commits
- Enable overlap scheduler by default for the triton attention backend (#2105) · 722530fa
  Lianmin Zheng authored Nov 20, 2024
  
  722530fa
- [feat] Add session control (#2073) · 5942dfc0
  Ying Sheng authored Nov 20, 2024
  
  5942dfc0