Commits · ff314d62f38cd1699267df948b3e1f480412f92b · gaoqiong / lm-evaluation-harness

29 Nov, 2021 2 commits
- Make everything request_type · 3b386db4
  Leo Gao authored Nov 28, 2021
  
  3b386db4
- Update base.py · 24fde4a6
  Leo Gao authored Nov 28, 2021
  
  24fde4a6
24 Nov, 2021 1 commit
- Some refactor/clean-up, and GPT-3 update · 76ebb792
  Jason Phang authored Nov 23, 2021
  
  76ebb792
05 Nov, 2021 2 commits
- Add imports and fix comment · 2ff1f5a2
  Leo Gao authored Nov 05, 2021
  
  2ff1f5a2
- Merge TokenizedLM and TorchLM into BaseLM · fea936e6
  Leo Gao authored Nov 05, 2021
  
  fea936e6
30 Oct, 2021 1 commit
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
11 Oct, 2021 1 commit
- Refactor LM organization for more reuse · 7f24a08b
  Leo Gao authored Oct 11, 2021
  
  7f24a08b
10 Jun, 2021 2 commits
- Use hashed version stability test instead · f9cc0267
  Leo Gao authored Jun 10, 2021
  
  f9cc0267
- Implement WikiText · 5e31e40e
  Leo Gao authored Jun 09, 2021
  
  5e31e40e
21 May, 2021 1 commit
- Add missing () in fewshot-no-training-path · 031e6aca
  Leo Gao authored May 21, 2021
  
  031e6aca
11 May, 2021 3 commits
- Make PerplexityTask fit the framework a bit better · 0fa316f8
  Leo Gao authored May 11, 2021
  
  0fa316f8
- Make PerplexityTask fit the framework a bit better · 3830713f
  Leo Gao authored May 11, 2021
  
  3830713f
- Count perplexity correctly · b691c44f
  Leo Gao authored May 11, 2021
  
  b691c44f
06 May, 2021 1 commit
- Rename a ton of stuff · 45127aa7
  Leo Gao authored May 05, 2021
  
  45127aa7
05 May, 2021 3 commits
- Fix some things for tests · 80a739eb
  Leo Gao authored May 05, 2021
  
  80a739eb
- Minor changes · f16c301e
  Leo Gao authored May 05, 2021
  
  f16c301e
- Refactor PerplexityTask · 1e7f884d
  Leo Gao authored May 05, 2021
  
  1e7f884d
03 May, 2021 1 commit
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
11 Apr, 2021 2 commits
- Make acc_norm a separate metric · 3eaa493e
  Leo Gao authored Apr 10, 2021
  
  3eaa493e
- do per character loss aggregation for multiple choice tasks (similar to OAI's... · fae5fe66
  Ben Wang authored Apr 11, 2021
```
do per character loss aggregation for multiple choice tasks (similar to OAI's per token aggregation)
```
  fae5fe66
05 Apr, 2021 1 commit
- Implement partial caching · efbe6e7f
  Leo Gao authored Apr 04, 2021
```
Now, if a run gets interrupted halfway, you can easily resume
```
  efbe6e7f
27 Mar, 2021 3 commits
- Move fewshot stuff back into the right block · 0966e7b6
  Leo Gao authored Mar 26, 2021
  
  0966e7b6
- Pass random state around to fewshot_examples and fewshot_context · 3b2d5f6c
  Leo Gao authored Mar 26, 2021
  
  3b2d5f6c
- Fix fewshot for tasks without train docs · 0c402690
  Leo Gao authored Mar 26, 2021
  
  0c402690
25 Mar, 2021 1 commit
- Deterministic fewshot_experiments · 6a78fdaf
  Leo Gao authored Mar 25, 2021
  
  6a78fdaf
19 Feb, 2021 1 commit
- Add gpt2/3 tokenizer sanity check · 77b44470
  Leo Gao authored Feb 18, 2021
  
  77b44470
12 Feb, 2021 1 commit
- metrics file · 1fb90b91
  & authored Feb 12, 2021
  
  1fb90b91
11 Feb, 2021 3 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Fixes to make greedy_until work · 7b649ded
  Leo Gao authored Feb 10, 2021
  
  7b649ded
- Implement GPT2 greedy_until · e8f9dc71
  Leo Gao authored Feb 10, 2021
  
  e8f9dc71
08 Feb, 2021 2 commits
- LM: handle empty context · 359114fd
  Leo Gao authored Feb 07, 2021
  
  359114fd
- Fix caching · 77d4b087
  Leo Gao authored Feb 07, 2021
  
  77d4b087
05 Feb, 2021 4 commits
- Add test to make sure right space conventions are used · 049dfa34
  Leo Gao authored Feb 04, 2021
  
  049dfa34
- Add MultipleChoiceTest target · 2f5f42c6
  Leo Gao authored Feb 04, 2021
  
  2f5f42c6
- Add MultipleChoiceTask · 706cb53a
  Leo Gao authored Feb 04, 2021
  
  706cb53a
- Implement caching · d5cd9655
  Leo Gao authored Feb 04, 2021
  
  d5cd9655
04 Feb, 2021 2 commits

Leo Gao authored Feb 03, 2021

- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup

778e0f91

Fix lambada · b57d059a
Leo Gao authored Feb 03, 2021

b57d059a

03 Feb, 2021 2 commits
- Fix naming convention to avoid `pytest` name mangling invocation · 5cfb7308
  Jonathan Tow authored Feb 02, 2021
  
  5cfb7308
- Refactor `Dataset` naming and `HFTask` properties · a60ef6fa
  Jonathan Tow authored Feb 02, 2021
  
  a60ef6fa