GitLab

# Conflicts:
#	batch_eval/main.py

Fix eval script to normalize loglikelihoods

Batch model inputs to speed things up

Create CODEOWNERS

Tweak StoryCloze script to be agnostic to tokenization

Add naive eval script for StoryCloze

Refactor to remove generate and fix some bad tokenization.

In particular, the following assumptions are FALSE in general:
tokenize(context + continuation) = tokenize(context) + tokenize(continuation)
len(tokenize(context + continuation)) = len(tokenize(context)) + len(tokenize(continuation))
tokenize(context + continuation)[:len(tokenize(context))] = tokenize(context)

So we need to tip-toe around the problem by being careful with how we do it.

In particular, using Fast is not just for performance; while behavour of GPT2Tokenizer differs across Transformers 2 and 3, GPT2TokenizerFast doesn't.

# Conflicts:
#	write_out.py