Commits · c21240c0a9b68f5845e8705aeb5326b17da2d3bb · gaoqiong / lm-evaluation-harness

17 Nov, 2023 1 commit
- edits and format · 10cc0a56
  lintangsutawika authored Nov 17, 2023
  
  10cc0a56
14 Nov, 2023 1 commit
- enables tasks of different group but same task_alis (for example, if... · 7760573f
  lintangsutawika authored Nov 14, 2023
```
enables tasks of different group but same task_alis (for example, if evaluating on different versions of MMLU
```
  7760573f
10 Nov, 2023 1 commit
- added initialize_task and updated where eval_logger is imported from · 056c9d85
  lintangsutawika authored Nov 10, 2023
  
  056c9d85
06 Nov, 2023 1 commit
- fxied brier score · 5cc65a79
  lintangsutawika authored Nov 06, 2023
  
  5cc65a79
02 Nov, 2023 2 commits
- precommit format · 73f3029c
  lintangsutawika authored Nov 02, 2023
  
  73f3029c
- eval_logger is not imported from logger.py anymore · f701ba7d
  lintangsutawika authored Nov 02, 2023
  
  f701ba7d
11 Oct, 2023 1 commit
- add _batch_scheduler in greedy_until · 1aa3bc1e
  Zhiwei Zhuang authored Oct 11, 2023
  
  1aa3bc1e
26 Sep, 2023 2 commits
- update loading prompts · b2d16321
  lintangsutawika authored Sep 26, 2023
  
  b2d16321
- moved files · 3f090027
  lintangsutawika authored Sep 26, 2023
  
  3f090027
22 Sep, 2023 2 commits
- trigger precommit · e4aa240e
  haileyschoelkopf authored Sep 22, 2023
  
  e4aa240e
- hotfix bugs · 43c91b5c
  haileyschoelkopf authored Sep 22, 2023
  
  43c91b5c
15 Sep, 2023 1 commit
- allow for trailing comma in model_args · 4ff8260d
  haileyschoelkopf authored Sep 15, 2023
  
  4ff8260d
14 Sep, 2023 1 commit
- remove omegaconf usage · 98747638
  haileyschoelkopf authored Sep 14, 2023
  
  98747638
12 Sep, 2023 2 commits
- changed term to groups · 6bba33db
  lintangsutawika authored Sep 12, 2023
  
  6bba33db
- changed term to groups · 4e4f0de2
  lintangsutawika authored Sep 12, 2023
  
  4e4f0de2
11 Sep, 2023 1 commit
- update · c87703f3
  lintangsutawika authored Sep 11, 2023
  
  c87703f3
07 Sep, 2023 1 commit
- normalize path · 4b711aa6
  lintangsutawika authored Sep 07, 2023
  
  4b711aa6
29 Aug, 2023 1 commit
- expanded benchmark to allow new source of prompt templates · d1c3cb3d
  lintangsutawika authored Aug 29, 2023
  
  d1c3cb3d
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

08 Aug, 2023 1 commit
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
04 Aug, 2023 1 commit
- remove num_fewshot in table that was supposed to be removed in benchmark-scripts · 07f94446
  lintangsutawika authored Aug 04, 2023
  
  07f94446
25 Jul, 2023 2 commits
- reorder · f148c2e2
  Lintang Sutawika authored Jul 25, 2023
  
  f148c2e2
- Reorder · 0b8446a4
  Lintang Sutawika authored Jul 25, 2023
  
  0b8446a4
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
19 Jul, 2023 1 commit
- can now process a benchmark that uses promptsource · 748a9898
  lintangsutawika authored Jul 19, 2023
  
  748a9898
18 Jul, 2023 1 commit
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
17 Jul, 2023 1 commit
- add column as option · ef7588b6
  lintangsutawika authored Jul 17, 2023
  
  ef7588b6
05 Jul, 2023 1 commit
- raise ValueError when invalid task passed · 7f557daa
  haileyschoelkopf authored Jul 05, 2023
  
  7f557daa
03 Jul, 2023 2 commits
- fix squeeze() in pad_and_concat · 46ca66de
  haileyschoelkopf authored Jul 03, 2023
  
  46ca66de
- add typehint · 62f4d663
  haileyschoelkopf authored Jul 03, 2023
  
  62f4d663
29 Jun, 2023 2 commits
- Fix typo · d15ee17a
  Hailey Schoelkopf authored Jun 29, 2023
  
  d15ee17a
- Remove "select_continuation_from_batch_left_padding" · 909f7346
  Hailey Schoelkopf authored Jun 29, 2023
  
  909f7346
28 Jun, 2023 1 commit
- add Grouper · 83f957bc
  haileyschoelkopf authored Jun 28, 2023
  
  83f957bc
24 Jun, 2023 1 commit
- squeeze tensors before pad_and_concat · b3aab393
  haileyschoelkopf authored Jun 24, 2023
  
  b3aab393
23 Jun, 2023 1 commit
- add additional err handling for get_git_commit_hash · f832c776
  haileyschoelkopf authored Jun 23, 2023
  
  f832c776
22 Jun, 2023 4 commits
- more pre-commit · b48f5205
  haileyschoelkopf authored Jun 20, 2023
  
  b48f5205
- init tensors on-device where possible · 0e4f5361
  haileyschoelkopf authored Jun 20, 2023
  
  0e4f5361
- improve pad_and_concat · fb8abcec
  haileyschoelkopf authored Jun 20, 2023
  
  fb8abcec
- resolved merge conflict · 0a3b8069
  lintangsutawika authored Jun 22, 2023
  
  0a3b8069
21 Jun, 2023 1 commit
- reformat · 3c895d56
  lintangsutawika authored Jun 21, 2023
  
  3c895d56