Commits · 2d4ce1b7928d253144bc4b030a643af2b9267b40 · zhaoyu6 / sglang

"cpp_onnx/src/vscode:/vscode.git/clone" did not exist on "96195981da3b59de7651baea67c9b40715606491"

31 Oct, 2024 1 commit
- [Performance, Triton Kernel Args] _decode_grouped_softmax_reducev_fwd… (#1845) · 2d4ce1b7
  HAI authored Oct 30, 2024
  
  2d4ce1b7
30 Oct, 2024 7 commits
- Fix suggest edit (#1842) · 4ba815b8
  Chayenne authored Oct 30, 2024
```
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
```
  4ba815b8
- [Performance, Hardware] MoE weights padding to AMD MI300x GPUs (#1836) · 5f65e2b8
  HAI authored Oct 30, 2024
  
  5f65e2b8
- [Production] Drain requests before exit when receive SIGTERM (#1838) · 4e2af03c
  Ying Sheng authored Oct 30, 2024
  
  4e2af03c
- Update README.md (#1840) · 3184aa95
  Lianmin Zheng authored Oct 30, 2024
  
  3184aa95
- Update docs (#1839) · b548801d
  Lianmin Zheng authored Oct 30, 2024
  
  b548801d
- Imporve openai api documents (#1827) · 539df95d
  Chayenne authored Oct 30, 2024
```
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
```
  539df95d
- Add new model: Gpt2 (#1833) · 5e00ddeb
  DanielC12321 authored Oct 29, 2024
  
  5e00ddeb
29 Oct, 2024 5 commits
- [FP8 KV Cache, Mixtral] Avoid KeyError at loading pre-quantized FP8 m… (#1835) · 54dd3ea1
  HAI authored Oct 29, 2024
  
  54dd3ea1
- stop_str of qwen2-vl template should be a tuple not a str (#1834) · d04899d7
  yizhang2077 authored Oct 30, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  d04899d7
- [3rdparty, document] Add 3rdparty/amd, with profiling and tuning instructions to be added (#1822) · 5010e0d2
  HAI authored Oct 29, 2024
  
  5010e0d2
- Support setting `use_thread` in the `run_program` for easier debugging. (#1823) · 5e6c3265
  Yanyi Liu authored Oct 29, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  5e6c3265
- fix get_memory_pool_size deadlock for DP (#1830) · 680cad20
  Byron Hsu authored Oct 28, 2024
  
  680cad20
28 Oct, 2024 3 commits
- Fix update_weights deadlock for DP (#1825) · 0a24eb85
  Byron Hsu authored Oct 28, 2024
  
  0a24eb85
- [Router] Add a rust-based router (#1790) · 3839be29
  Byron Hsu authored Oct 28, 2024
  
  3839be29
- Fix docs deploy ci (#1821) · 6e13b650
  Chayenne authored Oct 27, 2024
  
  6e13b650
27 Oct, 2024 6 commits
- Support token ids in `engine.generate` (#1820) · 6fcd6d7d
  Byron Hsu authored Oct 27, 2024
  
  6fcd6d7d
- Fix Triton decode kernel & ut (#1819) · c77762d5
  Ke Bao authored Oct 28, 2024
  
  c77762d5
- Add openAI compatible API (#1810) · 51c81e33
  Chayenne authored Oct 27, 2024
```
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
```
  51c81e33
- Fix unit tests (#1817) · eaade87a
  Lianmin Zheng authored Oct 27, 2024
  
  eaade87a
- Add a watch dog thread (#1816) · 86fc0d79
  Lianmin Zheng authored Oct 27, 2024
  
  86fc0d79
- Update hyperparameter_tuning.md (#1813) · 1be853ee
  Lianmin Zheng authored Oct 26, 2024
  
  1be853ee
26 Oct, 2024 13 commits
- Improve the user control of new_token_ratio (#1811) · 86e0dde5
  Lianmin Zheng authored Oct 26, 2024
  
  86e0dde5
- Provide an argument to set the maximum batch size for cuda graph (#1809) · 2b809788
  Lianmin Zheng authored Oct 26, 2024
  
  2b809788
- Fix docs ci (#1808) · 9d6fb084
  Chayenne authored Oct 26, 2024
  
  9d6fb084
- Simplify our docs with complicated functions into utils (#1807) · ced362f7
  Chayenne authored Oct 26, 2024
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
```
  ced362f7
- Update links (#1805) · 9084a864
  Lianmin Zheng authored Oct 26, 2024
  
  9084a864
- Update ci workflows (#1804) · 6aa94b96
  Lianmin Zheng authored Oct 26, 2024
  
  6aa94b96
- fix int conversion for `SGLANG_CPU_COUNT` (#1803) · c2650748
  Byron Hsu authored Oct 26, 2024
  
  c2650748
- Allow consecutive ports when launching multiple sglang servers. (#1802) · 07bf2e84
  Liangsheng Yin authored Oct 25, 2024
  
  07bf2e84
- Set `ZMQ` buffer size heuristic (#1801) · a628dd8e
  Liangsheng Yin authored Oct 25, 2024
  
  a628dd8e
- Fix possible ZMQ hanging (#1800) · 1e890341
  Liangsheng Yin authored Oct 25, 2024
  
  1e890341
- Add support for ipynb (#1786) · 715b16c1
  Chayenne authored Oct 25, 2024
  
  715b16c1
- move max_position_embeddings to the last (#1799) · 9ce8e1a9
  Hui Liu authored Oct 25, 2024
  
  9ce8e1a9
- [Fix] Fix --skip-tokenizer-init (#1798) · fb99aaa5
  Lianmin Zheng authored Oct 25, 2024
  
  fb99aaa5
25 Oct, 2024 5 commits
- [Performance] Support both xgrammar and outlines for constrained decoding (#1752) · b77a02cd
  DarkSharpness authored Oct 26, 2024
  
  b77a02cd
- Release v0.3.4.post2 (#1796) · 30643fed
  Lianmin Zheng authored Oct 25, 2024
```
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
```
  30643fed
- Fix logprob in the overlapped mode (#1795) · e646c590
  Lianmin Zheng authored Oct 25, 2024
  
  e646c590
- Revert "Fix memory leak when doing chunked prefill" (#1797) · c555ce2c
  Lianmin Zheng authored Oct 25, 2024
  
  c555ce2c
- [Fix] Fix the log parsing in chunked prefill uni tests (#1794) · 40900bae
  Lianmin Zheng authored Oct 25, 2024
  
  40900bae