Skip to content

GitLab

Unverified Commit 7c32b686 authored by tomeras91's avatar tomeras91 Committed by GitHub
Browse files

[Frontend] correctly record prefill and decode time metrics (#10853) 


Signed-off-by: default avatarTomer Asida <tomera@ai21.com>
parent 7090c27b
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/engine/metrics.py
View file @ 7c32b686
......@@ -599,9 +599,9 @@ class PrometheusStatLogger(StatLoggerBase):
stats.time_queue_requests)
self._log_histogram(self.metrics.histogram_inference_time_request,
stats.time_inference_requests)
self._log_histogram(self.metrics.histogram_decode_time_request,
stats.time_prefill_requests)
self._log_histogram(self.metrics.histogram_prefill_time_request,
stats.time_prefill_requests)
self._log_histogram(self.metrics.histogram_decode_time_request,
stats.time_decode_requests)
self._log_histogram(self.metrics.histogram_time_in_queue_request,
stats.time_in_queue_requests)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment