Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
f08343d4
Unverified
Commit
f08343d4
authored
May 04, 2023
by
Nicolas Patry
Committed by
GitHub
May 04, 2023
Browse files
fix(server): Removes the parallelism in file convertion (during download) (#275)
parent
b4fe248b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
13 deletions
+6
-13
server/text_generation_server/utils/convert.py
server/text_generation_server/utils/convert.py
+6
-13
No files found.
server/text_generation_server/utils/convert.py
View file @
f08343d4
import
concurrent
import
concurrent
import
time
import
time
import
datetime
import
torch
import
torch
from
concurrent.futures
import
ThreadPoolExecutor
from
concurrent.futures
import
ThreadPoolExecutor
...
@@ -78,17 +79,9 @@ def convert_file(pt_file: Path, st_file: Path):
...
@@ -78,17 +79,9 @@ def convert_file(pt_file: Path, st_file: Path):
def
convert_files
(
pt_files
:
List
[
Path
],
st_files
:
List
[
Path
]):
def
convert_files
(
pt_files
:
List
[
Path
],
st_files
:
List
[
Path
]):
assert
len
(
pt_files
)
==
len
(
st_files
)
assert
len
(
pt_files
)
==
len
(
st_files
)
executor
=
ThreadPoolExecutor
(
max_workers
=
5
)
N
=
len
(
pt_files
)
futures
=
[
executor
.
submit
(
convert_file
,
pt_file
=
pt_file
,
st_file
=
st_file
)
for
pt_file
,
st_file
in
zip
(
pt_files
,
st_files
)
]
# We do this instead of using tqdm because we want to parse the logs with the launcher
# We do this instead of using tqdm because we want to parse the logs with the launcher
start_time
=
time
.
time
()
start
=
datetime
.
datetime
.
now
()
for
i
,
future
in
enumerate
(
concurrent
.
futures
.
as_completed
(
futures
)):
for
i
,
(
pt_file
,
sf_file
)
in
enumerate
(
zip
(
pt_files
,
st_files
)):
elapsed
=
timedelta
(
seconds
=
int
(
time
.
time
()
-
start_time
))
elapsed
=
datetime
.
datetime
.
now
()
-
start
remaining
=
len
(
futures
)
-
(
i
+
1
)
logger
.
info
(
f
"Convert: [
{
i
+
1
}
/
{
N
}
] -- Took:
{
elapsed
}
"
)
eta
=
(
elapsed
/
(
i
+
1
))
*
remaining
if
remaining
>
0
else
0
logger
.
info
(
f
"Convert: [
{
i
+
1
}
/
{
len
(
futures
)
}
] -- ETA:
{
eta
}
"
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment