GitLab

Create CODEOWNERS

Tweak StoryCloze script to be agnostic to tokenization

Add naive eval script for StoryCloze

Refactor to remove generate and fix some bad tokenization.

In particular, the following assumptions are FALSE in general:
tokenize(context + continuation) = tokenize(context) + tokenize(continuation)
len(tokenize(context + continuation)) = len(tokenize(context)) + len(tokenize(continuation))
tokenize(context + continuation)[:len(tokenize(context))] = tokenize(context)

So we need to tip-toe around the problem by being careful with how we do it.

In particular, using Fast is not just for performance; while behavour of GPT2Tokenizer differs across Transformers 2 and 3, GPT2TokenizerFast doesn't.

# Conflicts:
#	write_out.py

Changing implementation of the —limit flag

This should allow the user to only do the first few eval tasks.

Add SAT Analogy dataset

Add SAT analogies dataset. Manual download needed. Checksums currently not verified, but hash is included as a comment.

Sync up to EAI

add rte text

Add Natural Questions