Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
c5bb0ebd
Unverified
Commit
c5bb0ebd
authored
May 19, 2025
by
Li Wang
Committed by
GitHub
May 19, 2025
Browse files
[Doc] Fix prompt embedding examples (#18350)
Signed-off-by:
wangli
<
wangli858794774@gmail.com
>
parent
d637b960
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
0 deletions
+2
-0
docs/source/serving/prompt_embeds.md
docs/source/serving/prompt_embeds.md
+2
-0
No files found.
docs/source/serving/prompt_embeds.md
View file @
c5bb0ebd
...
...
@@ -36,6 +36,7 @@ llm = LLM(model=model_name, enable_prompt_embeds=True)
chat
=
[{
"role"
:
"user"
,
"content"
:
"Please tell me about the capital of France."
}]
token_ids
=
tokenizer
.
apply_chat_template
(
chat
,
add_generation_prompt
=
True
,
return_tensors
=
'pt'
)
embedding_layer
=
transformers_model
.
get_input_embeddings
()
prompt_embeds
=
embedding_layer
(
token_ids
).
squeeze
(
0
)
# Single prompt inference
...
...
@@ -116,6 +117,7 @@ transformers_model = transformers.AutoModelForCausalLM.from_pretrained(model_nam
chat
=
[{
"role"
:
"user"
,
"content"
:
"Please tell me about the capital of France."
}]
token_ids
=
tokenizer
.
apply_chat_template
(
chat
,
add_generation_prompt
=
True
,
return_tensors
=
'pt'
)
embedding_layer
=
transformers_model
.
get_input_embeddings
()
prompt_embeds
=
embedding_layer
(
token_ids
).
squeeze
(
0
)
# Prompt embeddings
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment