Commit 73ea4187 authored by David Novotny's avatar David Novotny Committed by Facebook GitHub Bot
Browse files

SimpleDataLoaderMapProvider sample batches without replacement if num_samples is not specified

Summary: Samples batches without replacement if the number of samples is not specified. This makes sure that we always iterate over the whole dataset in each epoch.

Reviewed By: bottler

Differential Revision: D39270786

fbshipit-source-id: 0c983d1f5e0af711463abfb23939bc0d2b5172a0
parent f6d43eaa
......@@ -130,7 +130,14 @@ class SimpleDataLoaderMapProvider(DataLoaderMapProviderBase):
num_samples = self.batch_size * num_batches
else:
num_samples = None
sampler = RandomSampler(dataset, replacement=True, num_samples=num_samples)
# sample with replacement only if a custom number of samples is specified
sampler = RandomSampler(
dataset,
replacement=num_samples is not None,
num_samples=num_samples,
)
batch_sampler = BatchSampler(sampler, self.batch_size, drop_last=True)
return DataLoader(
dataset,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment