Commit 7b4f5517 authored by James Cross's avatar James Cross Committed by Facebook Github Bot
Browse files

get_batch_iterator: allow max_positions=None (#673)

Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/673

This function breaks when leaving the argument `max_positions` with the default value `None`, which is presumably not the intended behavior.

Reviewed By: theweiho, myleott

Differential Revision: D15937221

fbshipit-source-id: 1f5dc1c27ad9b6a89501d2dc015de12181059349
parent b625d53d
......@@ -132,6 +132,7 @@ class FairseqTask(object):
indices = dataset.ordered_indices()
# filter examples that are too large
if max_positions is not None:
indices = data_utils.filter_by_size(
indices, dataset.size, max_positions, raise_exception=(not ignore_invalid_inputs),
)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment