Skip to content

GitLab

Unverified Commit 564970d3 authored by Timothy Jaeryang Baek's avatar Timothy Jaeryang Baek Committed by GitHub
Browse files

Merge pull request #784 from dannyl1u/feat/prompt-token/s 

feat: show prompt token/s in perf info
parents 9c3be590 ab57cdd6
Hide whitespace changes
Inline Side-by-side
Showing with 6 additions and 1 deletion
src/lib/components/chat/Messages/ResponseMessage.svelte
View file @ 564970d3
...@@ -72,13 +72,18 @@ ...@@ -72,13 +72,18 @@
if (message.info) { if (message.info) {
tooltipInstance = tippy(`#info-${message.id}`, { tooltipInstance = tippy(`#info-${message.id}`, {
content: `<span class="text-xs" id="tooltip-${message.id}">token/s: ${ content: `<span class="text-xs" id="tooltip-${message.id}">response_token/s: ${
`${ `${
Math.round( Math.round(
((message.info.eval_count ?? 0) / (message.info.eval_duration / 1000000000)) * 100 ((message.info.eval_count ?? 0) / (message.info.eval_duration / 1000000000)) * 100
) / 100 ) / 100
} tokens` ?? 'N/A' } tokens` ?? 'N/A'
}<br/> }<br/>
prompt_token/s: ${
Math.round(
((message.info.prompt_eval_count ?? 0) / (message.info.prompt_eval_duration / 1000000000)) * 100
) / 100 ?? 'N/A'
} tokens<br/>
total_duration: ${ total_duration: ${
Math.round(((message.info.total_duration ?? 0) / 1000000) * 100) / 100 ?? Math.round(((message.info.total_duration ?? 0) / 1000000) * 100) / 100 ??
'N/A' 'N/A'
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment