Skip to content

GitLab

Commit 559eca81 authored by Myle Ott's avatar Myle Ott
Browse files

Remove Google batching stategy (it's not needed)

parent 97b58b46
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 2 deletions
fairseq/data.py
View file @ 559eca81
...@@ -371,8 +371,6 @@ def shuffled_batches_by_size(src, dst, max_tokens=None, max_sentences=None, ...@@ -371,8 +371,6 @@ def shuffled_batches_by_size(src, dst, max_tokens=None, max_sentences=None,
# sort by sizes # sort by sizes
indices = indices[np.argsort(dst.sizes[indices], kind='mergesort')] indices = indices[np.argsort(dst.sizes[indices], kind='mergesort')]
indices = indices[np.argsort(src.sizes[indices], kind='mergesort')] indices = indices[np.argsort(src.sizes[indices], kind='mergesort')]
max_sizes = np.maximum(dst.sizes[indices], src.sizes[indices])
indices = indices[np.argsort(max_sizes[indices], kind='mergesort')]
batches = list(_make_batches( batches = list(_make_batches(
src, dst, indices, max_tokens, max_sentences, max_positions, src, dst, indices, max_tokens, max_sentences, max_positions,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment