Commits · 51caee740feece0bcfa43af780d7fcab1c213583 · change / sglang · GitLab

07 Jan, 2025 6 commits
- Host memory pool for hierarchical caching (#2771) · 51caee74
  Zhiqiang Xie authored Jan 07, 2025
  
  51caee74
- Update int8 gemm config (#2774) · 58f9060e
  Ke Bao authored Jan 07, 2025
  
  58f9060e
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
- Use Optional with None default (#2770) · 6d08ce2a
  HAI authored Jan 07, 2025
  
  6d08ce2a
- add benchmark_moe_align_blocks (#2767) · 380930a9
  Xiaoyu Zhang authored Jan 07, 2025
  
  380930a9
- Remove --modelopt-config in server_args (#2758) · 9dec582d
  Lianmin Zheng authored Jan 06, 2025
  
  9dec582d
06 Jan, 2025 9 commits
- Update README.md (#2757) · b01febdc
  Lianmin Zheng authored Jan 06, 2025
```
Co-authored-by: Junjie Jin <jjjjohnsonjin@gmail.com>
Co-authored-by: justdoit <24875266+coolhok@users.noreply.github.com>
```
  b01febdc
- Add generator-style run_batch function (#2513) · 1acbaf1b
  Xingyao Wang authored Jan 06, 2025
```
Co-authored-by: openhands <openhands@all-hands.dev>
```
  1acbaf1b
- Enable Nvidia's ModelOpt fp8 quantized models (#2535) · 287427e2
  Zhiyu authored Jan 06, 2025
  
  287427e2
- Clean up eagle code (#2756) · b8574f69
  Lianmin Zheng authored Jan 06, 2025
  
  b8574f69
- feat: add devcontainer.json for VSCode development (#2745) · 2855caa4
  王博伟 authored Jan 07, 2025
  
  2855caa4
- Support llamafy/Qwen-Qwen2.5-7B-Instruct-llamafied (#2748) · 2329e1dd
  Xu-Chen authored Jan 07, 2025
```
Co-authored-by: chenxu02 <chenxu02@zhihu.com>
```
  2329e1dd
- Support cutlass Int8 gemm (#2752) · 0f3eb1d2
  Ke Bao authored Jan 06, 2025
  
  0f3eb1d2
- Remove unused var in moe_align_kernel (#2751) · 06dd2eab
  Ke Bao authored Jan 06, 2025
  
  06dd2eab
- Fix sgl-kernel cu118 compile issue (#2750) · 439f6580
  Ke Bao authored Jan 06, 2025
  
  439f6580
05 Jan, 2025 5 commits
- chore: bump v0.4.1.post4 (#2713) · 2f0d3864
  Yineng Zhang authored Jan 06, 2025
  
  2f0d3864
- Support twoshot kernel (#2688) · 3900a94a
  yizhang2077 authored Jan 06, 2025
  
  3900a94a
- improve moe_align_kernel for deepseek v3 (#2735) · ded9fcd0
  Xiaoyu Zhang authored Jan 06, 2025
  
  ded9fcd0
- fix lint (#2733) · bc6ad367
  Yineng Zhang authored Jan 05, 2025
  
  bc6ad367
- Revert the GLOO_SOCKET_IFNAME change (#2731) · 3a22a303
  Lianmin Zheng authored Jan 04, 2025
  
  3a22a303
03 Jan, 2025 6 commits
- Refactor SchedulePolicy to improve code organization (#2571) · bdb3929d
  libra authored Jan 04, 2025
  
  bdb3929d
- feat: Support VLM in reference_hf (#2726) · f5d0865b
  Ce Gao authored Jan 03, 2025
```
Signed-off-by: Ce Gao <gaocegege@hotmail.com>
```
  f5d0865b
- [Docs] fix 404 - Contributor Guide, again (#2727) · afdee7b1
  Ce Gao authored Jan 03, 2025
```
Signed-off-by: Ce Gao <gaocegege@hotmail.com>
```
  afdee7b1
- Update README.md (#2722) · cb34d848
  Lianmin Zheng authored Jan 03, 2025
```
Co-authored-by: Yangmin Li <2682000734@qq.com>
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  cb34d848
- Fix package loss for small models (#2717) · 0f9cc6d8
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: sdli1995 < mmlmonkey@163.com>
```
  0f9cc6d8
- [Feature, Hardware] Enable DeepseekV3 on AMD GPUs (#2601) · c7ae474a
  yigex authored Jan 03, 2025
```
Co-authored-by: root <root@banff-cyxtera-s83-5.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Bruce Xue <yigex@xilinx.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  c7ae474a
02 Jan, 2025 10 commits
- Support loading pre-sharded moe weights (#2716) · bdf946bf
  Lianmin Zheng authored Jan 02, 2025
  
  bdf946bf
- [Fix] fix retract error in eagle speculative decoding (#2711) · 8c8779cd
  yukavio authored Jan 03, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  8c8779cd
- [Fix] fix incorrectly overwriting the port specified in ServerArgs (#2714) · 1775b963
  Mick authored Jan 03, 2025
  
  1775b963
- Docs: Update documentation workflow and contribution guide (#2704) · dd2e2d27
  Shi Shuai authored Jan 02, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  dd2e2d27
- Included multi-node DeepSeekv3 example (#2707) · a990daff
  Rodrigo Garcia authored Jan 02, 2025
  
  a990daff
- feat: support moe_align_block_size_triton (#2712) · ba5112ff
  Yineng Zhang authored Jan 02, 2025
```
Co-authored-by: WANDY666 <1060304770@qq.com>
```
  ba5112ff
- Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) · 815dce05
  yukavio authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  815dce05
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
- Speed up `update_weights_from_tensor` (#2695) · 9183c23e
  fzyzcjy authored Jan 02, 2025
  
  9183c23e
- Improve moe reduce sum kernel performance (#2705) · 148254d4
  kk authored Jan 02, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  148254d4
01 Jan, 2025 3 commits
- [feat]: Add math eval to CI nightly run (#2663) · a4d6d6f1
  Xiaotong Jiang authored Jan 01, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  a4d6d6f1
- [Docs] Add Support for Pydantic Structured Output Format (#2697) · 062c48d2
  Shi Shuai authored Jan 01, 2025
  
  062c48d2
- ROCm base image update (#2692) · b6e0cfb5
  kk authored Jan 01, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  b6e0cfb5
31 Dec, 2024 1 commit
- Doc: Rename contribution_guide.md (#2691) · 0d8d97b8
  Chayenne authored Dec 31, 2024
  
  0d8d97b8