Commits · f65c13b5595a8437b23b716a3d0f76657e15df14 · change / sglang

15 Jan, 2025 1 commit
- Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902) · f65c13b5
  Lianmin Zheng authored Jan 15, 2025
  
  f65c13b5
13 Jan, 2025 1 commit
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
08 Jan, 2025 1 commit
- Improve linear.py to load sharded weights & remove the dependency of Parameters from vllm (#2784) · 8a690612
  Lianmin Zheng authored Jan 07, 2025
```
Co-authored-by: SangBin Cho rkooo567@gmail.com
```
  8a690612
05 Jan, 2025 1 commit
- fix lint (#2733) · bc6ad367
  Yineng Zhang authored Jan 05, 2025
  
  bc6ad367
03 Jan, 2025 1 commit
- feat: Support VLM in reference_hf (#2726) · f5d0865b
  Ce Gao authored Jan 03, 2025
```
Signed-off-by: Ce Gao <gaocegege@hotmail.com>
```
  f5d0865b
31 Dec, 2024 2 commits
- feat: use CUDA 12.4 by default (for FA3) (#2682) · d49b13c6
  Yineng Zhang authored Dec 31, 2024
  
  d49b13c6
- CI: Update scripts to fail fast (#2672) · f7074700
  fzyzcjy authored Dec 31, 2024
  
  f7074700
19 Dec, 2024 1 commit
- fix followup #2517 (#2524) · d95a5f5b
  Yineng Zhang authored Dec 19, 2024
  
  d95a5f5b
17 Dec, 2024 1 commit
- Clean up GPU memory after killing sglang processes (#2457) · ce094a5d
  Ata Fatahi authored Dec 17, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  ce094a5d
16 Dec, 2024 1 commit
- minor: update flashinfer nightly (#2490) · 7154b4b1
  Yineng Zhang authored Dec 16, 2024
  
  7154b4b1
09 Dec, 2024 1 commit
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
08 Dec, 2024 1 commit
- Update killall_sglang.sh (#2397) · 96db0f66
  Lianmin Zheng authored Dec 08, 2024
  
  96db0f66
07 Dec, 2024 1 commit
- minor: update killall script (#2391) · 75ae9689
  Yineng Zhang authored Dec 08, 2024
  
  75ae9689
01 Dec, 2024 4 commits
- minor: rm unused _grouped_size_compiled_for_decode_kernels (#2299) · 3dbd73d3
  Yineng Zhang authored Dec 01, 2024
  
  3dbd73d3
- minor: support flashinfer nightly (#2295) · fc78640e
  Yineng Zhang authored Dec 01, 2024
  
  fc78640e
- feat: add should_use_tensor_core (#2179) · 118b6af3
  Yineng Zhang authored Dec 01, 2024
  
  118b6af3
- [CI] Balance CI tests (#2293) · 9449a954
  Lianmin Zheng authored Dec 01, 2024
  
  9449a954
30 Nov, 2024 1 commit
- [CI] Kill zombie processes (#2280) · 0d6a49bd
  Lianmin Zheng authored Nov 30, 2024
  
  0d6a49bd
29 Nov, 2024 1 commit
- chore: bump v0.3.6.post3 (#2259) · fae4e5e9
  Yineng Zhang authored Nov 30, 2024
  
  fae4e5e9
25 Nov, 2024 2 commits
- [feat] Refactor session control interface and add CI (#2173) · e1e595d7
  Ying Sheng authored Nov 25, 2024
  
  e1e595d7
- [CI] Split test cases in CI for better load balancing (#2180) · 254fd130
  Lianmin Zheng authored Nov 25, 2024
  
  254fd130
22 Nov, 2024 2 commits
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
20 Nov, 2024 2 commits
- Enable overlap scheduler by default for the triton attention backend (#2105) · 722530fa
  Lianmin Zheng authored Nov 20, 2024
  
  722530fa
- Move test_session_id.py to playground (#2104) · 56a347f7
  Lianmin Zheng authored Nov 20, 2024
  
  56a347f7
18 Nov, 2024 1 commit
- Support cuda graph for DP attention (#2061) · 62832bb2
  Ke Bao authored Nov 18, 2024
  
  62832bb2
09 Nov, 2024 1 commit
- Clean up metrics code (#1972) · 9c939a3d
  Lianmin Zheng authored Nov 09, 2024
  
  9c939a3d
08 Nov, 2024 1 commit
- Add sentence_transformers to CI dependency (#1958) · 7ef0084b
  Lianmin Zheng authored Nov 08, 2024
  
  7ef0084b
07 Nov, 2024 1 commit
- fix black in pre-commit (#1940) · c77c1e05
  Chayenne authored Nov 07, 2024
  
  c77c1e05
06 Nov, 2024 1 commit
- minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926) · a5e0defb
  Xuehai Pan authored Nov 06, 2024
  
  a5e0defb
04 Nov, 2024 1 commit
- [router] Impl radix tree and set up CI (#1893) · 530ff541
  Byron Hsu authored Nov 04, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  530ff541
03 Nov, 2024 1 commit
- Allow passing dtype and max_new_tokens to HF reference script (#1903) · 916b3cdd
  Jani Monoses authored Nov 03, 2024
  
  916b3cdd
31 Oct, 2024 1 commit
- Fix memory leak for chunked prefill 2 (#1858) · a2e0424a
  Lianmin Zheng authored Oct 31, 2024
```
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
```
  a2e0424a
30 Oct, 2024 1 commit
- Update docs (#1839) · b548801d
  Lianmin Zheng authored Oct 30, 2024
  
  b548801d
26 Oct, 2024 1 commit
- Update ci workflows (#1804) · 6aa94b96
  Lianmin Zheng authored Oct 26, 2024
  
  6aa94b96
07 Oct, 2024 1 commit
- [Profile] Add pytorch profiler (#1604) · c5325aba
  Ying Sheng authored Oct 07, 2024
  
  c5325aba
30 Sep, 2024 1 commit
- Organize Attention Backends (#1547) · 99ec439d
  Liangsheng Yin authored Sep 30, 2024
  
  99ec439d
25 Sep, 2024 1 commit
- [Fix] Fix clean_up_tokenization_spaces in tokenizer (#1510) · fb2d0680
  Lianmin Zheng authored Sep 24, 2024
  
  fb2d0680
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
22 Sep, 2024 1 commit
- Better unit tests for adding a new model (#1488) · 167591e8
  Lianmin Zheng authored Sep 22, 2024
  
  167591e8