Skip to content

GitLab

"git@developer.sourcefind.cn:renzhc/diffusers_dcu.git" did not exist on "63dd601758b3f520547513e61e88b6cdb42174f2"
Commit a49dc52b authored by Lianmin Zheng's avatar Lianmin Zheng
Browse files

release v0.1.10

parent 873d0e85
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
README.md
View file @ a49dc52b
...@@ -351,6 +351,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port ...@@ -351,6 +351,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
``` ```
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --mem-fraction-static 0.7 python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --mem-fraction-static 0.7
``` ```
- You can turn on [flashinfer](docs/flashinfer.md) to acclerate the inference by using highly optimized CUDA kernels.
### Supported Models ### Supported Models
- Llama - Llama
......
python/pyproject.toml
View file @ a49dc52b
...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" ...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "sglang" name = "sglang"
version = "0.1.9" version = "0.1.10"
description = "A structured generation langauge for LLMs." description = "A structured generation langauge for LLMs."
readme = "README.md" readme = "README.md"
requires-python = ">=3.8" requires-python = ">=3.8"
......
python/sglang/__init__.py
View file @ a49dc52b
__version__ = "0.1.9" __version__ = "0.1.10"
from sglang.api import * from sglang.api import *
from sglang.global_config import global_config from sglang.global_config import global_config
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment