Commits · 9472e69963283160b51a617431e936f23910443a · zhaoyu6 / sglang

"IMG/cpio/ventoy/loop/freedombox/ventoy-disk.sh" did not exist on "1cbe4c81b4978311696ab4e9e8dffb7d98b82570"

26 Jan, 2025 4 commits
- Feature/function calling update (#2700) · b045841b
  YAMY authored Jan 26, 2025
```
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  b045841b
- Temporarily skip the openai frontend tests (#3151) · f4a92f4b
  Lianmin Zheng authored Jan 26, 2025
  
  f4a92f4b
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Fix CI tests (#3132) · da6f8081
  Lianmin Zheng authored Jan 25, 2025
  
  da6f8081
21 Jan, 2025 2 commits
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
20 Jan, 2025 5 commits
- [Enhancement] Custom Logit Processor Improvement (#2998) · 583697cd
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  583697cd
- Skip flaky custom_logit_processor tests (#3004) · 51e87f6f
  Lianmin Zheng authored Jan 20, 2025
  
  51e87f6f
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 3 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
18 Jan, 2025 2 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) · d3024f4f
  bjmsong authored Jan 18, 2025
```
Co-authored-by: bjmsong <bjmsong@126.com>
```
  d3024f4f
17 Jan, 2025 1 commit
- Add ut for qwen model (#2947) · d47c5101
  Ke Bao authored Jan 18, 2025
  
  d47c5101
16 Jan, 2025 2 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Improve benchmark scripts and error message printing (#2922) · 8f2c522a
  Lianmin Zheng authored Jan 16, 2025
  
  8f2c522a
15 Jan, 2025 2 commits
- adapt custom allreduce for tensorrt llm (#2511) · 767c9dec
  yizhang2077 authored Jan 16, 2025
  
  767c9dec
- Add ut for w8a8 int8 quantization (#2897) · bfbda62c
  Ke Bao authored Jan 15, 2025
  
  bfbda62c
13 Jan, 2025 7 commits
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Revert "Integration of TurboMind AWQ" (#2866) · 6249e4a1
  Lianmin Zheng authored Jan 13, 2025
  
  6249e4a1
- Integration of TurboMind AWQ (#2828) · 17de02f9
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  17de02f9
- Collect more metrics: num_requests_total (#2859) · 51ab3ccf
  Lianmin Zheng authored Jan 13, 2025
  
  51ab3ccf
- Use only one GPU for MLA CI tests (#2858) · 67008f4b
  Lianmin Zheng authored Jan 13, 2025
  
  67008f4b
- Fix linear.py and improve weight loading (#2851) · 72c77763
  Lianmin Zheng authored Jan 13, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  72c77763
- Support FP8 E4M3 KV Cache (#2786) · 0bb0f763
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  0bb0f763
11 Jan, 2025 2 commits
- Improve: Token-In Token-Out Usage for RLHF (#2843) · c4f9707e
  Shi Shuai authored Jan 11, 2025
  
  c4f9707e
- Update threshold in test_nightly_gsm8k_eval.py (#2836) · f1769586
  Lianmin Zheng authored Jan 10, 2025
  
  f1769586
10 Jan, 2025 2 commits
- [Eagle2] Fix multiple concurrent request crashes (#2730) · a47bf391
  justdoit authored Jan 11, 2025
  
  a47bf391
- [Bugfix] Fix embedding model hangs with `--enable-metrics` (#2822) · f290bd43
  Chang Su authored Jan 10, 2025
  
  f290bd43
08 Jan, 2025 3 commits
- [eagle2] fix end check when target model verify (#2723) · 694e4192
  JJJJOHNSON authored Jan 08, 2025
  
  694e4192
- Fix nightly accuracy tests (#2780) · b22f3f64
  Lianmin Zheng authored Jan 07, 2025
  
  b22f3f64
- Disable math eval on nightly CI temporarily (#2779) · 6fb57683
  Lianmin Zheng authored Jan 07, 2025
  
  6fb57683
03 Jan, 2025 2 commits
- Refactor SchedulePolicy to improve code organization (#2571) · bdb3929d
  libra authored Jan 04, 2025
  
  bdb3929d
- Fix package loss for small models (#2717) · 0f9cc6d8
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: sdli1995 < mmlmonkey@163.com>
```
  0f9cc6d8
02 Jan, 2025 3 commits
- Docs: Update documentation workflow and contribution guide (#2704) · dd2e2d27
  Shi Shuai authored Jan 02, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  dd2e2d27
- Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) · 815dce05
  yukavio authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  815dce05
- Speed up `update_weights_from_tensor` (#2695) · 9183c23e
  fzyzcjy authored Jan 02, 2025
  
  9183c23e