Commits · 03464890e0e0d048ebc1aa407e5235d7338f6aff · zhaoyu6 / sglang

"git@developer.sourcefind.cn:wuxk1/megatron-lm.git" did not exist on "8ceed7c71a5182eebf5db45381fdcc4b8b97f4f4"

20 Jan, 2025 5 commits
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 14 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- fix custom op version compatibility (#2988) · 2c05f81f
  Yineng Zhang authored Jan 20, 2025
  
  2c05f81f
- [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) · d77caa2b
  Seungduk Kim authored Jan 20, 2025
  
  d77caa2b
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0
- add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) · 24cafe31
  yizhang2077 authored Jan 19, 2025
  
  24cafe31
- fix deepseek v2 with cpu device (#2975) · 5a176c92
  Yineng Zhang authored Jan 19, 2025
  
  5a176c92
- Simplify logits processor (#2974) · 23196d52
  Lianmin Zheng authored Jan 18, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  23196d52
- Fix the request loggings to make it fully able to be easily replayed (#2973) · 93b77c8e
  Lianmin Zheng authored Jan 18, 2025
  
  93b77c8e
- Remove the unused write_with_records (#2972) · 7906d1d2
  Lianmin Zheng authored Jan 18, 2025
  
  7906d1d2
- Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) · 81d27c8e
  fzyzcjy authored Jan 19, 2025
  
  81d27c8e
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
- Memory pool: Minor optimize to avoid to (#2901) · 2bd18e2d
  Yang Zheng authored Jan 19, 2025
  
  2bd18e2d
18 Jan, 2025 5 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- feat: remove vllm get_rope (#2964) · 2add697d
  Yineng Zhang authored Jan 18, 2025
  
  2add697d
- Remove fp8 monkey patch (#2960) · 656dcc1a
  Ke Bao authored Jan 18, 2025
  
  656dcc1a
- Query remaining memory dynamically for PrefillAdder (#2941) · 8af7048d
  Zhiqiang Xie authored Jan 17, 2025
  
  8af7048d
- support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) · d3024f4f
  bjmsong authored Jan 18, 2025
```
Co-authored-by: bjmsong <bjmsong@126.com>
```
  d3024f4f
17 Jan, 2025 6 commits
- feat: use get_rope for gemma2 (#2954) · 78e5b22f
  Yineng Zhang authored Jan 18, 2025
  
  78e5b22f
- cleanup models unused import 2/n (#2952) · 7a15e9ad
  Yineng Zhang authored Jan 18, 2025
  
  7a15e9ad
- cleanup models dependencies 1/n (#2948) · 033c715b
  Yineng Zhang authored Jan 17, 2025
  
  033c715b
- Fix qwen accuracy issue (#2945) · 53e6552f
  Ke Bao authored Jan 17, 2025
  
  53e6552f
- feat: remove vllm distributed (#2907) · 5dc54f1a
  Yineng Zhang authored Jan 17, 2025
```
Co-authored-by: Zhangyi <1109276519@qq.com>
```
  5dc54f1a
- Enable CPU device on SGLang (#2806) · 63051738
  Chunyuan WU authored Jan 17, 2025
  
  63051738
16 Jan, 2025 9 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Fix zmq binding (#2930) · 0427416b
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
```
  0427416b
- Improve type annotation and styles (#2926) · bc6915e3
  Lianmin Zheng authored Jan 16, 2025
  
  bc6915e3
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
- Simplify the process launch code in server.py (#2923) · 93d69061
  Lianmin Zheng authored Jan 16, 2025
  
  93d69061
- add profiling to bench_one_batch script (#2821) · e00e5385
  Yun Dai authored Jan 16, 2025
  
  e00e5385
- fixed lm_head.weight error for quantized qwen (#2910) · a2f602b5
  Rin Intachuen authored Jan 16, 2025
  
  a2f602b5
- Improve benchmark scripts and error message printing (#2922) · 8f2c522a
  Lianmin Zheng authored Jan 16, 2025
  
  8f2c522a
- feat: patch linear base (#2915) · bf8d07a6
  Yineng Zhang authored Jan 16, 2025
  
  bf8d07a6
15 Jan, 2025 1 commit
- adapt custom allreduce for tensorrt llm (#2511) · 767c9dec
  yizhang2077 authored Jan 16, 2025
  
  767c9dec