Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
10682826
"unicore/git@developer.sourcefind.cn:OpenDAS/Uni-Core.git" did not exist on "49c9895b675dad5702200f46f77cf43eca74175e"
Commit
10682826
authored
Apr 01, 2025
by
Kiv Chen
Committed by
GitHub
Apr 01, 2025
Browse files
fix: sglang worker log extraction error (#447)
parent
5beba233
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
12 deletions
+10
-12
lib/llm/src/engines/sglang/worker.rs
lib/llm/src/engines/sglang/worker.rs
+10
-12
No files found.
lib/llm/src/engines/sglang/worker.rs
View file @
10682826
...
@@ -51,7 +51,8 @@ use crate::protocols::TokenIdType;
...
@@ -51,7 +51,8 @@ use crate::protocols::TokenIdType;
const
SGLANG_STOP_TIMEOUT
:
Duration
=
Duration
::
from_millis
(
1500
);
const
SGLANG_STOP_TIMEOUT
:
Duration
=
Duration
::
from_millis
(
1500
);
/// Match sglang python log entries, e.g "[2025-01-30 11:23:16] Some text we want"
/// Match sglang python log entries, e.g "[2025-01-30 11:23:16] Some text we want"
const
SGLANG_LOG_RE
:
&
str
=
r"(\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\] )?(.*)"
;
const
SGLANG_LOG_RE
:
&
str
=
r"(?<timestamp>\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\] )?(?<message>.*)"
;
/// Identify sglang log entries with this prefix
/// Identify sglang log entries with this prefix
const
LOG_PREFIX
:
&
str
=
"SGLANG"
;
const
LOG_PREFIX
:
&
str
=
"SGLANG"
;
...
@@ -506,18 +507,15 @@ async fn start_sglang(
...
@@ -506,18 +507,15 @@ async fn start_sglang(
let
line_re
=
Regex
::
new
(
SGLANG_LOG_RE
)
.unwrap
();
let
line_re
=
Regex
::
new
(
SGLANG_LOG_RE
)
.unwrap
();
let
mut
lines
=
stderr
.lines
();
let
mut
lines
=
stderr
.lines
();
while
let
Ok
(
Some
(
line
))
=
lines
.next_line
()
.await
{
while
let
Ok
(
Some
(
line
))
=
lines
.next_line
()
.await
{
if
let
Some
(
cap
)
=
line_re
.captures
(
&
line
)
{
if
let
Some
(
caps
)
=
line_re
.captures
(
&
line
)
{
match
cap
.len
()
{
match
caps
.name
(
"timestamp"
)
{
2
=>
{
Some
(
_
)
=>
{
// No date/time, these are usually errors
// Skip Python's date/time. Should be normal log
tracing
::
warn!
(
"{LOG_PREFIX}{tp_rank} {line}"
);
tracing
::
debug!
(
"{LOG_PREFIX}{tp_rank} {}"
,
&
caps
[
"message"
]);
}
3
=>
{
// Normal log line. Skip Python's date/time
tracing
::
debug!
(
"{LOG_PREFIX}{tp_rank} {}"
,
&
cap
[
2
]);
}
}
x
=>
{
None
=>
{
unreachable!
(
"sglang log re only has two capture groups, so {x} entries is impossible"
);
// No date/time. Usually errors
tracing
::
warn!
(
"{LOG_PREFIX}{tp_rank} {line}"
);
}
}
}
}
}
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment