Commits · 8f4d04e5403510fdea9d6ab83fc67f3e07ea4e32 · change / sglang · GitLab

21 Dec, 2024 1 commit
- chore: bump v0.4.0.post2 (#2525) · 8f4d04e5
  Yineng Zhang authored Dec 21, 2024
  
  8f4d04e5
20 Dec, 2024 1 commit
- Add integration with gemlite weight only quant (#2528) · feb2b768
  Jerry Zhang authored Dec 20, 2024
  
  feb2b768
19 Dec, 2024 3 commits
- fix: continue to use flashinfer 0.1.6 temporarily (#2517) · 4b83db24
  Yineng Zhang authored Dec 19, 2024
  
  4b83db24
- docs: update README (#2516) · 64456cf0
  Yineng Zhang authored Dec 19, 2024
  
  64456cf0
- feat: add llama3 eval (#2515) · bb4a9220
  Yineng Zhang authored Dec 19, 2024
  
  bb4a9220
17 Dec, 2024 6 commits
- Print progress bar during cuda graph capture (#2502) · 21e9e63a
  Lianmin Zheng authored Dec 17, 2024
  
  21e9e63a
- Fix openai protocols and pass top_k, min_p (#2499) · 361ea8d9
  Lianmin Zheng authored Dec 17, 2024
  
  361ea8d9
- Add lora_path to chat completion (#2438) · 33c5ff28
  Lei authored Dec 17, 2024
  
  33c5ff28
- ROCm support for sglang.check_env (#2426) · 5ce9daea
  Hui Liu authored Dec 17, 2024
  
  5ce9daea
- Small fix for the order of apply_torchao_config (#2495) · bd619616
  Lianmin Zheng authored Dec 16, 2024
  
  bd619616
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45
16 Dec, 2024 4 commits
- Revert "Small fixes for torchao quant" (#2493) · ba36b552
  Lianmin Zheng authored Dec 16, 2024
  
  ba36b552
- Simplify pytorch sampling kernel and logit processor (#2491) · 7a1aecb9
  Lianmin Zheng authored Dec 16, 2024
  
  7a1aecb9
- Small fixes for torchao quant (#2476) · 82699474
  Jerry Zhang authored Dec 16, 2024
  
  82699474
- fix moe-ep accuracy issue for fp8 (#2489) · b532a5fd
  xiaobochen authored Dec 16, 2024
  
  b532a5fd
14 Dec, 2024 3 commits
- hotfix: checking for HIP (#2485) · 5f2595be
  Yineng Zhang authored Dec 15, 2024
  
  5f2595be
- Remove cuda graph batch size adjustment for dp attention (#2484) · 0ba2c589
  Ke Bao authored Dec 14, 2024
  
  0ba2c589
- Fix correctness issue for triton decoding kernel (#2479) · 2f9bd0fa
  Ke Bao authored Dec 14, 2024
  
  2f9bd0fa
12 Dec, 2024 1 commit
- [Minor] Fix grok model loader (#2473) · 5282a473
  Lianmin Zheng authored Dec 12, 2024
  
  5282a473
11 Dec, 2024 6 commits
- [Core] in batch prefix caching by delay scheduling (#2442) · 9208618b
  SangBin Cho authored Dec 11, 2024
  
  9208618b
- Add support for IBM Granite 3.x models (#2437) · 993956c6
  Fred Reiss authored Dec 11, 2024
  
  993956c6
- Fix warmup in bench_offline_throughput.py (#2449) · f8548295
  Lianmin Zheng authored Dec 11, 2024
  
  f8548295
- Fix model loader for more quantization formats (#2448) · 959735fc
  Lianmin Zheng authored Dec 11, 2024
  
  959735fc
- chore: update ao v0.7.0 (#2447) · 626a99ac
  Yineng Zhang authored Dec 11, 2024
  
  626a99ac
- Make torch TP composable with torchao (#2436) · ece72491
  Ke Wen authored Dec 11, 2024
  
  ece72491
09 Dec, 2024 6 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
- reduce watchdog interval to 5s (#2410) · 27f7bed7
  Byron Hsu authored Dec 08, 2024
  
  27f7bed7
08 Dec, 2024 8 commits
- Simplify stream_output (#2398) · a6ca736c
  Lianmin Zheng authored Dec 08, 2024
  
  a6ca736c
- Fix recv_requests (#2405) · cc858953
  Lianmin Zheng authored Dec 08, 2024
  
  cc858953
- fix: specify dtype with begin_forward aka plan (#2404) · 6128f7cf
  Yineng Zhang authored Dec 08, 2024
  
  6128f7cf
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- Remove unused vars in the triton backend (#2401) · 61dec545
  Ke Bao authored Dec 08, 2024
  
  61dec545
- Optimize Triton decoding kernel for long context (#2394) · 7dc66fcb
  Ke Bao authored Dec 08, 2024
  
  7dc66fcb
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
- Fix shape error that occurred when loading lora weight of gemma2 model. (#2330) · 63dfab1b
  Sangchun Ha (Patrick) authored Dec 08, 2024
  
  63dfab1b
07 Dec, 2024 1 commit
- minor: update killall script (#2391) · 75ae9689
  Yineng Zhang authored Dec 08, 2024
  
  75ae9689