Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
93eeb543
Commit
93eeb543
authored
Jan 08, 2024
by
Lianmin Zheng
Browse files
Update readme.md
parent
ead5b39f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
2 deletions
+4
-2
README.md
README.md
+4
-2
No files found.
README.md
View file @
93eeb543
...
...
@@ -98,11 +98,13 @@ You can find more examples at [examples/quick_start](examples/quick_start).
### Multi Modality
```
python
@
sgl
.
function
def
multi_turn_question
(
s
,
image_file
,
question
):
def
image_qa
(
s
,
image_file
,
question
):
s
+=
sgl
.
user
(
sgl
.
image
(
image_file
)
+
question
)
s
+=
sgl
.
assistant
(
sgl
.
gen
(
"answer_1"
,
max_tokens
=
256
))
```
### Constrained decoding
### Batching
### Streaming
...
...
@@ -142,12 +144,12 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
-
Mistral
-
Mixtral
-
LLaVA
-
`python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000`
## Benchmark And Performance
## Roadmap
-
[ ] Function call
-
[ ] Constrained decoding
-
[ ] Quantization
-
[ ] S-LoRA
-
[ ] More models
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment