Skip to content

GitLab

Unverified Commit 964472b9 authored by Chen Zhang's avatar Chen Zhang Committed by GitHub
Browse files

[Doc] Update prefix cache metrics to counting tokens (#18138) 


Signed-off-by: default avatarChen Zhang <zhangch99@outlook.com>
parent 59dd311c
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
docs/source/design/v1/metrics.md
View file @ 964472b9
...@@ -415,8 +415,8 @@ The discussion in <gh-issue:10582> about adding prefix cache metrics yielded ...@@ -415,8 +415,8 @@ The discussion in <gh-issue:10582> about adding prefix cache metrics yielded
some interesting points which may be relevant to how we approach some interesting points which may be relevant to how we approach
future metrics. future metrics.
Every time the prefix cache is queried, we record the number of blocks Every time the prefix cache is queried, we record the number of tokens
queried and the number of queried blocks present in the cache queried and the number of queried tokens present in the cache
(i.e. hits). (i.e. hits).
However, the metric of interest is the hit rate - i.e. the number of However, the metric of interest is the hit rate - i.e. the number of
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment