Commit 733a752d authored by guptapriya's avatar guptapriya
Browse files

Add shuffle to dataset records

This shuffling should help in getting shuffling each epoch.
parent 9b7b64be
......@@ -235,7 +235,10 @@ def _read_and_batch_from_files(
# Remove examples where the input or target length exceeds the maximum length,
dataset = dataset.filter(lambda x, y: _filter_max_length((x, y), max_length))
if shuffle:
dataset = dataset.shuffle(20000)
if static_batch:
dataset = dataset.padded_batch(
batch_size // max_length, ([max_length], [max_length]),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment