Skip to content

GitLab

Commit 957b98c0 authored by Lengyue's avatar Lengyue Committed by zjsun
Browse files

Close buffer safely

parent 599529ec
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
preparing_data/wenet_clean/compress_tar.py
View file @ 957b98c0
...@@ -30,6 +30,8 @@ def chunked_tarring(rank, file_list, base_folder, output_folder, chunk_size=1024 ...@@ -30,6 +30,8 @@ def chunked_tarring(rank, file_list, base_folder, output_folder, chunk_size=1024
chunk_count += 1 chunk_count += 1
total_size = 0 total_size = 0
buffer.close()
buffer = io.BytesIO() buffer = io.BytesIO()
tar = tarfile.open(fileobj=buffer, mode="w") tar = tarfile.open(fileobj=buffer, mode="w")
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment