Commits · 65859754f1463ce280bbaaf68d04797705849240 · change / sglang · GitLab

03 Nov, 2024 9 commits
- Release v0.3.5 (#1908) · 65859754
  Lianmin Zheng authored Nov 03, 2024
  
  65859754
- Let reward model take text inputs instead of message lists (#1907) · 2ce32db6
  Lianmin Zheng authored Nov 03, 2024
```
Co-authored-by: Kyle Corbitt <kyle@corbt.com>
```
  2ce32db6
- feat: support truss endpoint for benchmark serving (#1906) · 793b79db
  Yineng Zhang authored Nov 03, 2024
  
  793b79db
- Escape backwards slash (#1902) · 1363b519
  Iñaki Arango authored Nov 03, 2024
  
  1363b519
- Unify the model type checking (#1905) · 0abbf289
  Lianmin Zheng authored Nov 03, 2024
  
  0abbf289
- Simplify tokenizer manager (#1904) · c17c5781
  Lianmin Zheng authored Nov 03, 2024
  
  c17c5781
- Simplify tokenizer manager (#1899) · 838dcda1
  Lianmin Zheng authored Nov 03, 2024
  
  838dcda1
- Do not use longest prefix matching when #queue-req is large (#1896) · efbc116a
  Lianmin Zheng authored Nov 03, 2024
  
  efbc116a
- turn off log (#1895) · 6aed0445
  Chayenne authored Nov 03, 2024
  
  6aed0445
02 Nov, 2024 1 commit
- Fix incorrect context length for llama3.2-11b (#1873) · 146f6134
  Ran Chen authored Nov 02, 2024
  
  146f6134
01 Nov, 2024 2 commits
- Update vocab embedding deps and add TP switch (#1856) · 16eb33ff
  Ke Bao authored Nov 01, 2024
  
  16eb33ff
- Fix retraction + overlap (#1860) · b9fd178f
  Liangsheng Yin authored Oct 31, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  b9fd178f
31 Oct, 2024 7 commits
- [Build, ROCm] Dockerfile.rocm for Instinct GPUs, with package updates (#1861) · d8e9d61f
  HAI authored Oct 31, 2024
  
  d8e9d61f
- Fix memory leak for chunked prefill 2 (#1858) · a2e0424a
  Lianmin Zheng authored Oct 31, 2024
```
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
```
  a2e0424a
- delete unused character (#1855) · 8ce202a4
  geeker-smallwhite authored Oct 31, 2024
  
  8ce202a4
- Refactor tokenizer manager (#1846) · 438526a8
  Byron Hsu authored Oct 30, 2024
  
  438526a8
- Fix mixed chunked prefill (#1850) · f7102fbd
  Lianmin Zheng authored Oct 30, 2024
  
  f7102fbd
- Make decode log interval configurable (#1847) · a7a0a688
  Byron Hsu authored Oct 30, 2024
  
  a7a0a688
- [Performance, Triton Kernel Args] _decode_grouped_softmax_reducev_fwd… (#1845) · 2d4ce1b7
  HAI authored Oct 30, 2024
  
  2d4ce1b7
30 Oct, 2024 5 commits
- [Performance, Hardware] MoE weights padding to AMD MI300x GPUs (#1836) · 5f65e2b8
  HAI authored Oct 30, 2024
  
  5f65e2b8
- [Production] Drain requests before exit when receive SIGTERM (#1838) · 4e2af03c
  Ying Sheng authored Oct 30, 2024
  
  4e2af03c
- Update docs (#1839) · b548801d
  Lianmin Zheng authored Oct 30, 2024
  
  b548801d
- Imporve openai api documents (#1827) · 539df95d
  Chayenne authored Oct 30, 2024
```
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
```
  539df95d
- Add new model: Gpt2 (#1833) · 5e00ddeb
  DanielC12321 authored Oct 29, 2024
  
  5e00ddeb
29 Oct, 2024 4 commits
- [FP8 KV Cache, Mixtral] Avoid KeyError at loading pre-quantized FP8 m… (#1835) · 54dd3ea1
  HAI authored Oct 29, 2024
  
  54dd3ea1
- stop_str of qwen2-vl template should be a tuple not a str (#1834) · d04899d7
  yizhang2077 authored Oct 30, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  d04899d7
- Support setting `use_thread` in the `run_program` for easier debugging. (#1823) · 5e6c3265
  Yanyi Liu authored Oct 29, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  5e6c3265
- fix get_memory_pool_size deadlock for DP (#1830) · 680cad20
  Byron Hsu authored Oct 28, 2024
  
  680cad20
28 Oct, 2024 1 commit
- Fix update_weights deadlock for DP (#1825) · 0a24eb85
  Byron Hsu authored Oct 28, 2024
  
  0a24eb85
27 Oct, 2024 4 commits
- Support token ids in `engine.generate` (#1820) · 6fcd6d7d
  Byron Hsu authored Oct 27, 2024
  
  6fcd6d7d
- Fix Triton decode kernel & ut (#1819) · c77762d5
  Ke Bao authored Oct 28, 2024
  
  c77762d5
- Fix unit tests (#1817) · eaade87a
  Lianmin Zheng authored Oct 27, 2024
  
  eaade87a
- Add a watch dog thread (#1816) · 86fc0d79
  Lianmin Zheng authored Oct 27, 2024
  
  86fc0d79
26 Oct, 2024 7 commits
- Improve the user control of new_token_ratio (#1811) · 86e0dde5
  Lianmin Zheng authored Oct 26, 2024
  
  86e0dde5
- Provide an argument to set the maximum batch size for cuda graph (#1809) · 2b809788
  Lianmin Zheng authored Oct 26, 2024
  
  2b809788
- Simplify our docs with complicated functions into utils (#1807) · ced362f7
  Chayenne authored Oct 26, 2024
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
```
  ced362f7
- Update links (#1805) · 9084a864
  Lianmin Zheng authored Oct 26, 2024
  
  9084a864
- fix int conversion for `SGLANG_CPU_COUNT` (#1803) · c2650748
  Byron Hsu authored Oct 26, 2024
  
  c2650748
- Allow consecutive ports when launching multiple sglang servers. (#1802) · 07bf2e84
  Liangsheng Yin authored Oct 25, 2024
  
  07bf2e84
- Set `ZMQ` buffer size heuristic (#1801) · a628dd8e
  Liangsheng Yin authored Oct 25, 2024
  
  a628dd8e