Commits · 0d5748b7ca3e0c5ecf8ea9e0d9bb709e64a79446 · gaoqiong / lm-evaluation-harness

13 Dec, 2023 6 commits
- added mmlu alt world prompts · 0d5748b7
  lintangsutawika authored Dec 13, 2023
  
  0d5748b7
- add alt prompts for truthfulqa · b9e55312
  lintangsutawika authored Dec 13, 2023
  
  b9e55312
- removed · edb21d21
  lintangsutawika authored Dec 13, 2023
  
  edb21d21
- removed · 6e220b17
  lintangsutawika authored Dec 13, 2023
  
  6e220b17
- remove style · 09cac5dc
  lintangsutawika authored Dec 13, 2023
  
  09cac5dc
- fixed file path · ec4cdfed
  lintangsutawika authored Dec 13, 2023
  
  ec4cdfed
08 Dec, 2023 5 commits
- remove README.md · 1653632c
  lintangsutawika authored Dec 08, 2023
  
  1653632c
- added prompt and output variations · 1ff18c9a
  lintangsutawika authored Dec 08, 2023
  
  1ff18c9a
- added prompt and output variations · af6e46fd
  lintangsutawika authored Dec 08, 2023
  
  af6e46fd
- add prompt variation · 66421b57
  lintangsutawika authored Dec 08, 2023
  
  66421b57
- made prompt and output variations · 55eff889
  lintangsutawika authored Dec 08, 2023
  
  55eff889
06 Dec, 2023 5 commits
- reformat · c74e2761
  lintangsutawika authored Dec 06, 2023
  
  c74e2761
- add style 01 · cc572624
  lintangsutawika authored Dec 06, 2023
  
  cc572624
- removed mmlu altworlds · 50df117b
  lintangsutawika authored Dec 06, 2023
  
  50df117b
- add aggregation · c84190cb
  lintangsutawika authored Dec 06, 2023
  
  c84190cb
- fixed brier score to accomodate samples with different number of choices · 4acb339e
  lintangsutawika authored Dec 06, 2023
  
  4acb339e
04 Dec, 2023 2 commits
- add \n\n to end of description · c30b71c3
  lintangsutawika authored Dec 04, 2023
  
  c30b71c3
- add \n\n to end of description · b32b3793
  lintangsutawika authored Dec 04, 2023
  
  b32b3793
29 Nov, 2023 2 commits
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
- change torch req for mps · 7bb147b5
  baberabb authored Nov 29, 2023
  
  7bb147b5
28 Nov, 2023 16 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- Delete lm_eval/tasks/siqa/bigbench.yaml · 331d7c51
  Stella Biderman authored Nov 28, 2023
  
  331d7c51
- Update default.yml · 20463a3f
  Stella Biderman authored Nov 28, 2023
  
  20463a3f
- Rename bigbench.yml to default.yml · 5dcbcd63
  Stella Biderman authored Nov 28, 2023
```
Changing the name since this is not actually the big bench format.
```
  5dcbcd63
- reformat · 5ded4d5b
  lintangsutawika authored Nov 28, 2023
  
  5ded4d5b
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- update version · 43f615c9
  lintangsutawika authored Nov 28, 2023
  
  43f615c9
- update version · 94cdface
  lintangsutawika authored Nov 28, 2023
  
  94cdface
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
- renamed template file to make it easier to search · 19738ed2
  lintangsutawika authored Nov 28, 2023
  
  19738ed2
- renamed template file to make it easier to search · db8928b0
  lintangsutawika authored Nov 28, 2023
  
  db8928b0
- updated oa_completion and fix import error · 8ea23e2d
  lintangsutawika authored Nov 28, 2023
  
  8ea23e2d
- removed flan from the naming · 3b9640b8
  lintangsutawika authored Nov 28, 2023
  
  3b9640b8
27 Nov, 2023 4 commits
- Update README.md · 5535fbd4
  Stella Biderman authored Nov 27, 2023
  
  5535fbd4
- Update README.md · 4706fbb2
  Stella Biderman authored Nov 27, 2023
  
  4706fbb2
- Create bigbench.yml · 11129906
  Stella Biderman authored Nov 27, 2023
  
  11129906
- Create README.md · babdc8d6
  Stella Biderman authored Nov 27, 2023
  
  babdc8d6