GitLab

The unconditional generation works now but if the seed is fixed, the sample is the same every time.
n_samples > 1 will give different samples though.
I am giving the start token as '<|endoftext|>' for the unconditional generation.

order.

data on disc as a memmap rather than in memory

data on disc as a memmap rather than in memory

out on the fly without shuffling - the Sampler in the finetuning script
will shuffle for us.

out on the fly without shuffling - the Sampler in the finetuning script
will shuffle for us.

Maybe better.

modify `heruistic` to `heuristic` in line 660, `charcter` to `character` in line 661.

modify `mull` to `null` in line 474 annotation.

delete redundancy line `if args.train`, simplify code.