Commits · 00c7e6368bbf598da9af5443f24353e8ff2a6fd0 · change / sglang · GitLab

11 Oct, 2024 10 commits
- Release v0.3.3.post1 (#1636) · 00c7e636
  Lianmin Zheng authored Oct 11, 2024
  
  00c7e636
- Add back data parallelism (#1635) · 23cc66f7
  Lianmin Zheng authored Oct 11, 2024
  
  23cc66f7
- Fix constrained decoding (#1634) · 5d09ca57
  Lianmin Zheng authored Oct 11, 2024
  
  5d09ca57
- Add image_token in conversation.py (#1632) · f13d86f9
  Lianmin Zheng authored Oct 11, 2024
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  f13d86f9
- Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py (#1631) · aba9eae4
  Lianmin Zheng authored Oct 11, 2024
  
  aba9eae4
- Add the ability to enable and disable the Profiler via HTTP API. (#1626) · bbd72bfc
  科英 authored Oct 11, 2024
  
  bbd72bfc
- [Bug] Fix the Image Input of Batch Generation (#1579) · b503881b
  Yiding-Lu authored Oct 11, 2024
  
  b503881b
- Nit about the decorator of `PortArgs.init_new` (#1611) · 58093b86
  glen-amd authored Oct 11, 2024
  
  58093b86
- Add device support (#1607) · 8275049c
  Zhang, Liangang authored Oct 11, 2024
  
  8275049c
- [Performance, hardware] MoE tuning update to AMD MI300x GPUs (#1619) · e11ab79e
  HAI authored Oct 10, 2024
  
  e11ab79e
10 Oct, 2024 1 commit
- Fix test_vision_openai_server on CI (#1620) · 01fdb2f3
  Byron Hsu authored Oct 10, 2024
  
  01fdb2f3
09 Oct, 2024 1 commit
- [Minor] Fix logging typo (#1615) · c996e8cc
  Amos You authored Oct 08, 2024
  
  c996e8cc
08 Oct, 2024 2 commits
- Release v0.3.3 (#1605) · 7b69d91b
  Lianmin Zheng authored Oct 08, 2024
  
  7b69d91b
- [Engine] Fix generate hanging issue after the first call (#1606) · e8613df0
  Byron Hsu authored Oct 07, 2024
  
  e8613df0
07 Oct, 2024 9 commits
- [Profile] Add pytorch profiler (#1604) · c5325aba
  Ying Sheng authored Oct 07, 2024
  
  c5325aba
- Optimize broadcast & Reorg code (#1598) · ebbc42d9
  Lianmin Zheng authored Oct 07, 2024
  
  ebbc42d9
- Remove references to squeezellm (#1603) · 3ff64113
  Jani Monoses authored Oct 07, 2024
  
  3ff64113
- Fix the port_args in bench_latency (#1597) · 2b302b93
  Lianmin Zheng authored Oct 07, 2024
  
  2b302b93
- Fix chunked prefill condition (#1594) · 68f8b60d
  Ke Bao authored Oct 07, 2024
  
  68f8b60d
- Use is_flashinfer_available to replace is_hip for flashinfer check (#1596) · 6a5b352a
  Lianmin Zheng authored Oct 06, 2024
```
Co-authored-by: Zhang Liangang <liangang.zhang@intel.com>
```
  6a5b352a
- Use `atexit` hook to implicitly shutdown `Runtime` (#1595) · 565b05f0
  Byron Hsu authored Oct 06, 2024
  
  565b05f0
- [Fix] Fix the case where prompt_len = 0 (#1593) · b6aad70a
  Lianmin Zheng authored Oct 06, 2024
  
  b6aad70a
- Provide an offline engine API (#1567) · 551a3a9d
  Byron Hsu authored Oct 06, 2024
  
  551a3a9d
06 Oct, 2024 10 commits
- Fix modality for image inputs (#1592) · 91877a9f
  Lianmin Zheng authored Oct 06, 2024
  
  91877a9f
- [Minor, Performance] Use torch.argmax for greedy sampling (#1589) · c98e84c2
  Ying Sheng authored Oct 06, 2024
  
  c98e84c2
- [LoRA, Performance] Speedup multi-LoRA serving - Step 1 (#1587) · 9c064bf7
  Ying Sheng authored Oct 06, 2024
  
  9c064bf7
- Clean up event loop (#1586) · 58d1082e
  Lianmin Zheng authored Oct 06, 2024
  
  58d1082e
- [Bug] Fix decode stats error on output_len 1 (#1585) · 4d086719
  HAI authored Oct 06, 2024
  
  4d086719
- [Minor] Improve the style and fix flaky tests (#1584) · 9244f27f
  Lianmin Zheng authored Oct 06, 2024
  
  9244f27f
- Support min_tokens in sgl.gen (#1573) · 2422de51
  Byron Hsu authored Oct 05, 2024
  
  2422de51
- Fix runtime.generate when sampling param is not passed (#1582) · 521f862d
  Byron Hsu authored Oct 05, 2024
  
  521f862d
- Fix styling (#1583) · 34c32d28
  Byron Hsu authored Oct 05, 2024
  
  34c32d28
- default sampling param should be deepcopied (#1581) · dde8bb16
  Byron Hsu authored Oct 05, 2024
  
  dde8bb16
05 Oct, 2024 4 commits
- Backend method not found when SRT Runtime is used (#1576) · 8ac3ccc0
  Byron Hsu authored Oct 05, 2024
  
  8ac3ccc0
- Add llama implementation with no tensor parallel linears (#1561) · 9b0926ce
  Jerry Zhang authored Oct 05, 2024
  
  9b0926ce
- [Easy] use .text() instead of .text (#1577) · 6bfdb403
  Byron Hsu authored Oct 05, 2024
  
  6bfdb403
- Refine the add request reasons to avoid corner cases. (#1574) · 5d0ba403
  Liangsheng Yin authored Oct 04, 2024
  
  5d0ba403
04 Oct, 2024 3 commits
- [Fix] Fix major performance bug in certain cases (#1563) · 04b262cd
  Ying Sheng authored Oct 04, 2024
```
Co-authored-by: hnyls2002 <hnyls2002@gmail.com>
```
  04b262cd
- Make input_ids a torch.Tensor (#1568) · 45473d4b
  Lianmin Zheng authored Oct 04, 2024
  
  45473d4b
- Use ipc instead of tcp in zmq (#1566) · 114bbc86
  Lianmin Zheng authored Oct 04, 2024
  
  114bbc86