Commits · 6d08ce2aa9f481f6704b8c04a9d2a5b138db7ebe · change / sglang · GitLab

07 Jan, 2025 2 commits
- Use Optional with None default (#2770) · 6d08ce2a
  HAI authored Jan 07, 2025
  
  6d08ce2a
- Remove --modelopt-config in server_args (#2758) · 9dec582d
  Lianmin Zheng authored Jan 06, 2025
  
  9dec582d
06 Jan, 2025 4 commits
- Add generator-style run_batch function (#2513) · 1acbaf1b
  Xingyao Wang authored Jan 06, 2025
```
Co-authored-by: openhands <openhands@all-hands.dev>
```
  1acbaf1b
- Enable Nvidia's ModelOpt fp8 quantized models (#2535) · 287427e2
  Zhiyu authored Jan 06, 2025
  
  287427e2
- Clean up eagle code (#2756) · b8574f69
  Lianmin Zheng authored Jan 06, 2025
  
  b8574f69
- Support llamafy/Qwen-Qwen2.5-7B-Instruct-llamafied (#2748) · 2329e1dd
  Xu-Chen authored Jan 07, 2025
```
Co-authored-by: chenxu02 <chenxu02@zhihu.com>
```
  2329e1dd
05 Jan, 2025 2 commits
- chore: bump v0.4.1.post4 (#2713) · 2f0d3864
  Yineng Zhang authored Jan 06, 2025
  
  2f0d3864
- Revert the GLOO_SOCKET_IFNAME change (#2731) · 3a22a303
  Lianmin Zheng authored Jan 04, 2025
  
  3a22a303
03 Jan, 2025 3 commits
- Refactor SchedulePolicy to improve code organization (#2571) · bdb3929d
  libra authored Jan 04, 2025
  
  bdb3929d
- Fix package loss for small models (#2717) · 0f9cc6d8
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: sdli1995 < mmlmonkey@163.com>
```
  0f9cc6d8
- [Feature, Hardware] Enable DeepseekV3 on AMD GPUs (#2601) · c7ae474a
  yigex authored Jan 03, 2025
```
Co-authored-by: root <root@banff-cyxtera-s83-5.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Bruce Xue <yigex@xilinx.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  c7ae474a
02 Jan, 2025 8 commits
- Support loading pre-sharded moe weights (#2716) · bdf946bf
  Lianmin Zheng authored Jan 02, 2025
  
  bdf946bf
- [Fix] fix retract error in eagle speculative decoding (#2711) · 8c8779cd
  yukavio authored Jan 03, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  8c8779cd
- [Fix] fix incorrectly overwriting the port specified in ServerArgs (#2714) · 1775b963
  Mick authored Jan 03, 2025
  
  1775b963
- feat: support moe_align_block_size_triton (#2712) · ba5112ff
  Yineng Zhang authored Jan 02, 2025
```
Co-authored-by: WANDY666 <1060304770@qq.com>
```
  ba5112ff
- Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) · 815dce05
  yukavio authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  815dce05
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
- Speed up `update_weights_from_tensor` (#2695) · 9183c23e
  fzyzcjy authored Jan 02, 2025
  
  9183c23e
- Improve moe reduce sum kernel performance (#2705) · 148254d4
  kk authored Jan 02, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  148254d4
01 Jan, 2025 1 commit
- ROCm base image update (#2692) · b6e0cfb5
  kk authored Jan 01, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  b6e0cfb5
31 Dec, 2024 6 commits
- h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B (#2689) · 286cad3e
  Xiaoyu Zhang authored Dec 31, 2024
  
  286cad3e
- [Fix] fix openai adapter (#2685) · dc7eb01f
  Ying Sheng authored Dec 31, 2024
  
  dc7eb01f
- Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) · b0524c37
  Lianmin Zheng authored Dec 31, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  b0524c37
- feat: use CUDA 12.4 by default (for FA3) (#2682) · d49b13c6
  Yineng Zhang authored Dec 31, 2024
  
  d49b13c6
- Support target model verification in the attention backend (#2678) · f44d1439
  Lianmin Zheng authored Dec 30, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  f44d1439
- Improve the computation for time_per_output_token Prometheus metrics (#2674) · 339c69a2
  Lianmin Zheng authored Dec 30, 2024
  
  339c69a2
30 Dec, 2024 6 commits
- Minor follow-up fixes for the logprob refactor (#2670) · 21ec66e5
  Lianmin Zheng authored Dec 30, 2024
  
  21ec66e5
- AMD DeepSeek_V3 FP8 Numerical fix (#2667) · c5210dfa
  HAI authored Dec 30, 2024
  
  c5210dfa
- Add GemLite caching after each capture (#2669) · a29dd950
  mobicham authored Dec 30, 2024
  
  a29dd950
- Refactor logprob computation to return the real logprob used in sampling (#2664) · 9c6ba248
  Lianmin Zheng authored Dec 30, 2024
  
  9c6ba248
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e
- fix typo in python/sglang/srt/layers/quantization/fp8.py (#2655) · e6f523b5
  HAI authored Dec 29, 2024
  
  e6f523b5
29 Dec, 2024 8 commits
- Release 0.4.1.post3 - upload the config.json to PyPI (#2647) · 03d5fbfd
  Lianmin Zheng authored Dec 29, 2024
  
  03d5fbfd
- CI: Fix unittest for engine input token ids and output token ids (#2646) · fad29f7f
  Shi Shuai authored Dec 29, 2024
  
  fad29f7f
- [Feature] Get Token IDs with Engine.generate() (#2636) · 35bdb485
  Shi Shuai authored Dec 29, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  35bdb485
- chore: bump v0.4.1.post2 (#2643) · 3ccf566b
  Yineng Zhang authored Dec 30, 2024
  
  3ccf566b
- Update Triton configs for block fp8 kernels (#2641) · afa0341e
  HandH1998 authored Dec 29, 2024
  
  afa0341e
- AMD: set weights and scaling numbers properly for block FP8 (#2637) · 30828e71
  HAI authored Dec 29, 2024
  
  30828e71
- [Session] Update session control interface (#2635) · e0e09fce
  Ying Sheng authored Dec 29, 2024
  
  e0e09fce
- Add llama_eagle.py (#2640) · 9c05c689
  Lianmin Zheng authored Dec 29, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  9c05c689