Unverified Commit 50f2aae1 authored by zejunchen-zejun's avatar zejunchen-zejun Committed by GitHub
Browse files

[LMCache][Example] Align the PYTHONHASHSEED for prefillers and decoders for KV...


[LMCache][Example] Align the PYTHONHASHSEED for prefillers and decoders for KV chunks hashing (#21161)
Signed-off-by: default avatarzejunchen-zejun <zejun.chen@amd.com>
parent 46ae7f66
......@@ -15,6 +15,14 @@ else
MODEL=$2
fi
# The prefillers and decoders in LMCache use the same hash seed for all chunk keys.
# This seed must be aligned so that decoders can identify and retrieve KV cache
# entries stored by prefillers.
#
# WARNING: Using a fixed hash seed is insecure and makes the application vulnerable to
# denial-of-service attacks. In a production environment, this should be set to a
# secure random value. This is set to a fixed value for demonstration purposes only.
export PYTHONHASHSEED=${VLLM_PYTHON_HASH_SEED:-123}
if [[ $1 == "prefiller" ]]; then
# Prefiller listens on port 8100
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment