Skip to content

GitLab

Commit 25c07e14 authored by mohammad's avatar mohammad Committed by Deepak Narayanan
Browse files

Added a comment to justify 80 percent

parent 6e9d5cb0
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
megatron/data/gpt2_dataset.py
View file @ 25c07e14
...@@ -243,6 +243,8 @@ def _build_index_mappings(name, data_prefix, documents, sizes, ...@@ -243,6 +243,8 @@ def _build_index_mappings(name, data_prefix, documents, sizes,
'last epoch number of samples exceeded max value.' 'last epoch number of samples exceeded max value.'
# If we have less than 80% of the samples for the last epoch, # If we have less than 80% of the samples for the last epoch,
# seperate out the epoch and treat it differently. # seperate out the epoch and treat it differently.
# Note: the 80% number is just based on common sense and can
# be adjusted if needed.
separate_last_epoch = (last_epoch_num_samples < separate_last_epoch = (last_epoch_num_samples <
int(0.80 * num_samples_per_epoch)) int(0.80 * num_samples_per_epoch))
if separate_last_epoch: if separate_last_epoch:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment