Commits · 6f4f9e1caffb82d351f7441f7ef9edd2f575f7bb · gaoqiong / lm-evaluation-harness

12 Dec, 2023 1 commit
- allow N/A as a possible value for Stderr · d6f8faad
  lintangsutawika authored Dec 12, 2023
  
  d6f8faad
29 Nov, 2023 1 commit
- fixed chunking · da2d160b
  baberabb authored Nov 29, 2023
  
  da2d160b
28 Nov, 2023 1 commit
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
21 Nov, 2023 1 commit
- update multi-token stopsequence handling · f7873a49
  haileyschoelkopf authored Nov 21, 2023
  
  f7873a49
20 Nov, 2023 1 commit
- add docs · b22f3440
  baberabb authored Nov 20, 2023
  
  b22f3440
17 Nov, 2023 1 commit
- edits and format · 10cc0a56
  lintangsutawika authored Nov 17, 2023
  
  10cc0a56
14 Nov, 2023 1 commit
- enables tasks of different group but same task_alis (for example, if... · 7760573f
  lintangsutawika authored Nov 14, 2023
```
enables tasks of different group but same task_alis (for example, if evaluating on different versions of MMLU
```
  7760573f
13 Nov, 2023 1 commit
- num fewshot is printed in table · 81495b0a
  lintangsutawika authored Nov 13, 2023
  
  81495b0a
10 Nov, 2023 1 commit
- added initialize_task and updated where eval_logger is imported from · 056c9d85
  lintangsutawika authored Nov 10, 2023
  
  056c9d85
06 Nov, 2023 1 commit
- fxied brier score · 5cc65a79
  lintangsutawika authored Nov 06, 2023
  
  5cc65a79
02 Nov, 2023 2 commits
- precommit format · 73f3029c
  lintangsutawika authored Nov 02, 2023
  
  73f3029c
- eval_logger is not imported from logger.py anymore · f701ba7d
  lintangsutawika authored Nov 02, 2023
  
  f701ba7d
11 Oct, 2023 1 commit
- add _batch_scheduler in greedy_until · 1aa3bc1e
  Zhiwei Zhuang authored Oct 11, 2023
  
  1aa3bc1e
03 Oct, 2023 1 commit
- Allow Generation arguments on greedy_until reqs · ec5846d7
  USVSN Sai Prashanth authored Oct 03, 2023
  
  ec5846d7
26 Sep, 2023 2 commits
- update loading prompts · b2d16321
  lintangsutawika authored Sep 26, 2023
  
  b2d16321
- moved files · 3f090027
  lintangsutawika authored Sep 26, 2023
  
  3f090027
22 Sep, 2023 2 commits
- trigger precommit · e4aa240e
  haileyschoelkopf authored Sep 22, 2023
  
  e4aa240e
- hotfix bugs · 43c91b5c
  haileyschoelkopf authored Sep 22, 2023
  
  43c91b5c
15 Sep, 2023 1 commit
- allow for trailing comma in model_args · 4ff8260d
  haileyschoelkopf authored Sep 15, 2023
  
  4ff8260d
14 Sep, 2023 1 commit
- remove omegaconf usage · 98747638
  haileyschoelkopf authored Sep 14, 2023
  
  98747638
12 Sep, 2023 2 commits
- changed term to groups · 6bba33db
  lintangsutawika authored Sep 12, 2023
  
  6bba33db
- changed term to groups · 4e4f0de2
  lintangsutawika authored Sep 12, 2023
  
  4e4f0de2
11 Sep, 2023 1 commit
- update · c87703f3
  lintangsutawika authored Sep 11, 2023
  
  c87703f3
07 Sep, 2023 1 commit
- normalize path · 4b711aa6
  lintangsutawika authored Sep 07, 2023
  
  4b711aa6
29 Aug, 2023 1 commit
- expanded benchmark to allow new source of prompt templates · d1c3cb3d
  lintangsutawika authored Aug 29, 2023
  
  d1c3cb3d
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

08 Aug, 2023 1 commit
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
04 Aug, 2023 1 commit
- remove num_fewshot in table that was supposed to be removed in benchmark-scripts · 07f94446
  lintangsutawika authored Aug 04, 2023
  
  07f94446
25 Jul, 2023 2 commits
- reorder · f148c2e2
  Lintang Sutawika authored Jul 25, 2023
  
  f148c2e2
- Reorder · 0b8446a4
  Lintang Sutawika authored Jul 25, 2023
  
  0b8446a4
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
19 Jul, 2023 1 commit
- can now process a benchmark that uses promptsource · 748a9898
  lintangsutawika authored Jul 19, 2023
  
  748a9898
18 Jul, 2023 1 commit
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
17 Jul, 2023 1 commit
- add column as option · ef7588b6
  lintangsutawika authored Jul 17, 2023
  
  ef7588b6
05 Jul, 2023 1 commit
- raise ValueError when invalid task passed · 7f557daa
  haileyschoelkopf authored Jul 05, 2023
  
  7f557daa
03 Jul, 2023 2 commits
- fix squeeze() in pad_and_concat · 46ca66de
  haileyschoelkopf authored Jul 03, 2023
  
  46ca66de
- add typehint · 62f4d663
  haileyschoelkopf authored Jul 03, 2023
  
  62f4d663
29 Jun, 2023 2 commits
- Fix typo · d15ee17a
  Hailey Schoelkopf authored Jun 29, 2023
  
  d15ee17a
- Remove "select_continuation_from_batch_left_padding" · 909f7346
  Hailey Schoelkopf authored Jun 29, 2023
  
  909f7346
28 Jun, 2023 1 commit
- add Grouper · 83f957bc
  haileyschoelkopf authored Jun 28, 2023
  
  83f957bc