Commits · d8476818efc88188d0aa0a8a176024a0b82e7a1d · change / sglang

19 Aug, 2024 1 commit

[Feat]Add support for optional start len of logprobs (#1035) · b997a18d

yichuan~ authored Aug 18, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>

b997a18d

17 Aug, 2024 1 commit
- fix: use fp16 dtype for sm75 (#1136) · 9208591f
  Yineng Zhang authored Aug 17, 2024
  
  9208591f
16 Aug, 2024 2 commits
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
- Fix the deadlock in multi-node tp (#1122) · 5a261bd0
  Lianmin Zheng authored Aug 16, 2024
  
  5a261bd0
15 Aug, 2024 5 commits
- [Fix] Window attention compatible with RadixAttention and chunked prefill (#1112) · 93d4e354
  Ying Sheng authored Aug 15, 2024
  
  93d4e354
- misc: rm unused model_loader (#1110) · 9195d136
  Yineng Zhang authored Aug 15, 2024
  
  9195d136
- [Fix] fix flashinfer usage for window attention (#1107) · 14cb544d
  Ying Sheng authored Aug 15, 2024
  
  14cb544d
- [Fix] fix the typo bug for window attention (#1106) · 8d2d876f
  Ying Sheng authored Aug 14, 2024
  
  8d2d876f
- Use a single workspace for flashinfer (#1077) · 326df4ba
  Lianmin Zheng authored Aug 14, 2024
  
  326df4ba
14 Aug, 2024 4 commits
- [Fix] Compatibility of window attention and cuda graph (#1090) · 96a2093e
  Ying Sheng authored Aug 14, 2024
  
  96a2093e
- Update grok 1 model (#1095) · a59636bb
  Lianmin Zheng authored Aug 14, 2024
  
  a59636bb
- Fix a bug in cuda graph runner (#1094) · 8f790ac1
  Lianmin Zheng authored Aug 14, 2024
  
  8f790ac1
- [Feat] Add window attention for gemma-2 (#1056) · 0909bb0d
  Ying Sheng authored Aug 13, 2024
  
  0909bb0d
12 Aug, 2024 1 commit
- minor: some potential bugs (#1044) · fb7421db
  Liangsheng Yin authored Aug 11, 2024
  
  fb7421db
11 Aug, 2024 1 commit
- Fix triton args init (#1034) · 7b6a5332
  Liangsheng Yin authored Aug 11, 2024
  
  7b6a5332
10 Aug, 2024 1 commit
- Fix `input_ids` && rename to `fill_ids` (#1021) · 43fbb6d9
  Liangsheng Yin authored Aug 10, 2024
  
  43fbb6d9
08 Aug, 2024 3 commits
- Add e5-mistral embedding model - step 3/3 (#988) · e040a245
  Ying Sheng authored Aug 08, 2024
  
  e040a245
- Move torch.compile configs into cuda_graph_runner.py (#993) · 9f662501
  Ying Sheng authored Aug 08, 2024
  
  9f662501
- Adjust `InputeMetadata` and `ScheduleBatch` (#981) · 1ac304ee
  Liangsheng Yin authored Aug 08, 2024
  
  1ac304ee
07 Aug, 2024 2 commits
- PrefillAdder abstraction (#968) · f724f1f1
  Liangsheng Yin authored Aug 07, 2024
  
  f724f1f1
- Organize code (rename, movement) (#953) · 87e8c090
  Liangsheng Yin authored Aug 06, 2024
  
  87e8c090
04 Aug, 2024 1 commit
- Support MLA for DeepSeek-V2 with Triton - step 1 (#905) · e1eae1fd
  Ke Bao authored Aug 05, 2024
  
  e1eae1fd
02 Aug, 2024 1 commit
- misc: update cuda graph capture exception log (#894) · 6b8f66ef
  Yineng Zhang authored Aug 02, 2024
  
  6b8f66ef
30 Jul, 2024 3 commits
- Add `--max-total-tokens` (#840) · 6b0f2e90
  Liangsheng Yin authored Jul 30, 2024
  
  6b0f2e90
- Adjust default mem fraction to avoid OOM (#823) · e7487b08
  Ying Sheng authored Jul 30, 2024
  
  e7487b08
- Code structure refactor (#807) · cdcbde5f
  Liangsheng Yin authored Jul 29, 2024
  
  cdcbde5f