Commit 6a7eb6ce authored by Jay Mahadeokar's avatar Jay Mahadeokar Committed by Facebook Github Bot
Browse files

bugfix data not in args

Summary:
D15214049 introduced a bug such that if a tasks args does not contain data, then it will give error
```
File "/data/users/jaym/fbsource/fbcode/buck-out/dev/gen/deeplearning/projects/fairspeq/train#link-tree/train.py", line 119, in reload_train
   if len(args.data.split(":")) == 1:
AttributeError: 'Namespace' object has no attribute 'data'
```

This diff checks if data is in args to avoid above error.

Reviewed By: myleott, jmp84

Differential Revision: D15253373

fbshipit-source-id: 14fb9ad878ee50f1b7583349bb17e29c03c40815
parent 20e7836e
...@@ -116,7 +116,7 @@ def main(args, init_distributed=False): ...@@ -116,7 +116,7 @@ def main(args, init_distributed=False):
def reload_train(args, epoch_itr, max_positions, task): def reload_train(args, epoch_itr, max_positions, task):
# nothing needs to be done when the dataset is not sharded. # nothing needs to be done when the dataset is not sharded.
if len(args.data.split(":")) == 1: if "data" not in args or ("data" in args and len(args.data.split(":")) == 1):
return epoch_itr return epoch_itr
print("| Reloading shard of train data at epoch: ", epoch_itr.epoch) print("| Reloading shard of train data at epoch: ", epoch_itr.epoch)
task.load_dataset(args.train_subset, combine=True, epoch=epoch_itr.epoch) task.load_dataset(args.train_subset, combine=True, epoch=epoch_itr.epoch)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment