Commit 8ff870bf authored by Lianmin Zheng's avatar Lianmin Zheng
Browse files

improve docs

parent 26f0bedc
...@@ -9,8 +9,9 @@ The core features of SGLang include: ...@@ -9,8 +9,9 @@ The core features of SGLang include:
- **A High-Performance Runtime with RadixAttention**: This feature significantly accelerates the execution of complex LLM programs by automatic KV cache reuse across multiple calls. It also supports other common techniques like continuous batching and tensor parallelism. - **A High-Performance Runtime with RadixAttention**: This feature significantly accelerates the execution of complex LLM programs by automatic KV cache reuse across multiple calls. It also supports other common techniques like continuous batching and tensor parallelism.
## News ## News
- [2024/01] 🔥 SGLang powers the serving of the offical LLaVA v1.6 release demo ([blog](https://llava-vl.github.io/blog/2024-01-30-llava-1-6/)). - [2024/02] 🔥 SGLang enables **3x faster JSON decoding** with compressed finite state machine ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/)).
- [2024/01] SGLang provides up to 5x faster inference with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)). - [2024/01] 🔥 SGLang powers the serving of the offical **LLaVA v1.6** release demo ([usage](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)).
- [2024/01] SGLang provides up to **5x faster inference** with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)).
## Contents ## Contents
- [Install](#install) - [Install](#install)
...@@ -216,7 +217,7 @@ character_regex = ( ...@@ -216,7 +217,7 @@ character_regex = (
@sgl.function @sgl.function
def character_gen(s, name): def character_gen(s, name):
s += name + " is a character in Harry Potter. Please fill in the following information about him/her.\n" s += name + " is a character in Harry Potter. Please fill in the following information about this character.\n"
s += sgl.gen("json_output", max_tokens=256, regex=character_regex) s += sgl.gen("json_output", max_tokens=256, regex=character_regex)
``` ```
......
...@@ -64,7 +64,7 @@ def character_maker(lm, name): ...@@ -64,7 +64,7 @@ def character_maker(lm, name):
regex_str_no_quote = r"[\w\d\s]+" regex_str_no_quote = r"[\w\d\s]+"
regex_float = r"[0-9]+\.[0-9]+" regex_float = r"[0-9]+\.[0-9]+"
lm += f"""\ lm += f"""\
{name} is a character in Harry Potter. Please fill in the following information about him/her. {name} is a character in Harry Potter. Please fill in the following information about this character.
{{ {{
"name": "{guidance.gen("name", max_tokens=16, regex=regex_str_no_quote)}", "name": "{guidance.gen("name", max_tokens=16, regex=regex_str_no_quote)}",
"house": "{guidance.select(options=['Gryffindor', 'Slytherin', 'Ravenclaw', 'Hufflepuff'], name='house')}", "house": "{guidance.select(options=['Gryffindor', 'Slytherin', 'Ravenclaw', 'Hufflepuff'], name='house')}",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment