@@ -9,8 +9,9 @@ The core features of SGLang include:
...
@@ -9,8 +9,9 @@ The core features of SGLang include:
-**A High-Performance Runtime with RadixAttention**: This feature significantly accelerates the execution of complex LLM programs by automatic KV cache reuse across multiple calls. It also supports other common techniques like continuous batching and tensor parallelism.
-**A High-Performance Runtime with RadixAttention**: This feature significantly accelerates the execution of complex LLM programs by automatic KV cache reuse across multiple calls. It also supports other common techniques like continuous batching and tensor parallelism.
## News
## News
-[2024/01] 🔥 SGLang powers the serving of the offical LLaVA v1.6 release demo ([blog](https://llava-vl.github.io/blog/2024-01-30-llava-1-6/)).
-[2024/02] 🔥 SGLang enables **3x faster JSON decoding** with compressed finite state machine ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/)).
-[2024/01] SGLang provides up to 5x faster inference with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)).
-[2024/01] 🔥 SGLang powers the serving of the offical **LLaVA v1.6** release demo ([usage](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)).
-[2024/01] SGLang provides up to **5x faster inference** with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)).
## Contents
## Contents
-[Install](#install)
-[Install](#install)
...
@@ -216,7 +217,7 @@ character_regex = (
...
@@ -216,7 +217,7 @@ character_regex = (
@sgl.function
@sgl.function
defcharacter_gen(s,name):
defcharacter_gen(s,name):
s+=name+" is a character in Harry Potter. Please fill in the following information about him/her.\n"
s+=name+" is a character in Harry Potter. Please fill in the following information about this character.\n"