Commits · a8ccacc8b8118ded5c993fe42e27fa3c6533e6d8 · change / sglang · GitLab

16 Jan, 2025 16 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Fix zmq binding (#2930) · 0427416b
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
```
  0427416b
- Docs: Update pull_request_template.md (#2928) · bf3edc2c
  Chayenne authored Jan 16, 2025
  
  bf3edc2c
- [kernel] MiniMax-Text-01 decode lightning_attn with triton (#2920) · 78e974b2
  Xiaoyu Zhang authored Jan 17, 2025
  
  78e974b2
- Improve type annotation and styles (#2926) · bc6915e3
  Lianmin Zheng authored Jan 16, 2025
  
  bc6915e3
- Update release-docker-amd.yml to run on amd docker runner. (#2927) · a883f079
  saienduri authored Jan 16, 2025
  
  a883f079
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
- Add CI for sgl-kernel (#2924) · 58f3f2b8
  Ke Bao authored Jan 17, 2025
  
  58f3f2b8
- Simplify the process launch code in server.py (#2923) · 93d69061
  Lianmin Zheng authored Jan 16, 2025
  
  93d69061
- add profiling to bench_one_batch script (#2821) · e00e5385
  Yun Dai authored Jan 16, 2025
  
  e00e5385
- fixed lm_head.weight error for quantized qwen (#2910) · a2f602b5
  Rin Intachuen authored Jan 16, 2025
  
  a2f602b5
- Improve benchmark scripts and error message printing (#2922) · 8f2c522a
  Lianmin Zheng authored Jan 16, 2025
  
  8f2c522a
- minor: use bear for compilation database (#2919) · 75964177
  Yineng Zhang authored Jan 16, 2025
  
  75964177
- fix setup for sgl kernel (#2917) · 2dc957d4
  Yineng Zhang authored Jan 16, 2025
  
  2dc957d4
- feat: patch linear base (#2915) · bf8d07a6
  Yineng Zhang authored Jan 16, 2025
  
  bf8d07a6
- [kernel] MiniMax-Text-01 prefill lightning_attn with triton (#2911) · ab317936
  Xiaoyu Zhang authored Jan 16, 2025
  
  ab317936
15 Jan, 2025 9 commits
- minor: rename bench for sgl kernel (#2909) · b7f3fec1
  Yineng Zhang authored Jan 16, 2025
  
  b7f3fec1
- minor: update pr test (#2908) · 58f42b1d
  Yineng Zhang authored Jan 16, 2025
  
  58f42b1d
- adapt custom allreduce for tensorrt llm (#2511) · 767c9dec
  yizhang2077 authored Jan 16, 2025
  
  767c9dec
- fix: sgl-kernel link cuda (#2906) · a53454c5
  Yineng Zhang authored Jan 16, 2025
  
  a53454c5
- optimize custom allreduce kernel (#2904) · 6cb3974e
  yizhang2077 authored Jan 16, 2025
  
  6cb3974e
- Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902) · f65c13b5
  Lianmin Zheng authored Jan 15, 2025
  
  f65c13b5
- Disable graceful shutdown of tokenizer manager when not in the main thread (#2872) · b803b395
  Cody Yu authored Jan 15, 2025
  
  b803b395
- Add ut for w8a8 int8 quantization (#2897) · bfbda62c
  Ke Bao authored Jan 15, 2025
  
  bfbda62c
- chore: bump v0.4.1.post6 (#2899) · b3e99dfb
  Yineng Zhang authored Jan 15, 2025
  
  b3e99dfb
14 Jan, 2025 6 commits
- introduce CUB in sgl-kernel (#2887) · f005758f
  Xiaoyu Zhang authored Jan 14, 2025
  
  f005758f
- feat: support internlm 3 dense (#2888) · f5c6c667
  Yineng Zhang authored Jan 14, 2025
  
  f5c6c667
- Support w8a8 int8 quantization config (#2881) · cc0485be
  Ke Bao authored Jan 14, 2025
  
  cc0485be
- update ROCm docker for layernorm kernel optimization (#2885) · b8cd09f2
  kk authored Jan 14, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  b8cd09f2
- Adjust flashinfer workspace size for Qwen2 models (#2879) · c19d8482
  Ke Bao authored Jan 14, 2025
  
  c19d8482
- docs: update README (#2878) · 80002562
  Yineng Zhang authored Jan 14, 2025
  
  80002562
13 Jan, 2025 9 commits
- Add a new api configure_logging to allow dumping the requests (#2875) · 46d44318
  Lianmin Zheng authored Jan 13, 2025
  
  46d44318
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Sampling penalties memory interface (#2870) · d08c77c4
  Xiaoyu Zhang authored Jan 13, 2025
  
  d08c77c4
- Revert "Dump requests to a folder" (#2869) · c1e097ca
  Lianmin Zheng authored Jan 13, 2025
  
  c1e097ca
- add qwen2 eagle model (#2863) · 6ec75e62
  Lzhang-hub authored Jan 13, 2025
  
  6ec75e62
- minor: fix release docs (#2868) · d855653b
  Yineng Zhang authored Jan 13, 2025
  
  d855653b
- Fix typos in io_struct.py (#2867) · 336ff5b9
  Lianmin Zheng authored Jan 13, 2025
  
  336ff5b9
- Dump requests (#2862) · 3b141e15
  Lianmin Zheng authored Jan 13, 2025
  
  3b141e15
- Revert "Integration of TurboMind AWQ" (#2866) · 6249e4a1
  Lianmin Zheng authored Jan 13, 2025
  
  6249e4a1