Commits · 0fa9f03f4a1bb9914a9cc486680f0056c5300958 · gaoqiong / lm-evaluation-harness

"llm/ggml-metal.metal" did not exist on "688661ab9b5bb821555efda75f016689dfd1b2da"

10 Jul, 2023 2 commits
- pass override bs to _loglikelihood_tokens · 0fa9f03f
  Benjamin Fattori authored Jul 10, 2023
  
  0fa9f03f
- initial autobatching commit · d42e1706
  Benjamin Fattori authored Jul 10, 2023
  
  d42e1706
07 Jul, 2023 1 commit
- Update README.md · 884ba785
  Hailey Schoelkopf authored Jul 07, 2023
  
  884ba785
06 Jul, 2023 3 commits
- move race on HF to EleutherAI group · 31250b07
  ben authored Jul 06, 2023
  
  31250b07
- Update task statuses · fdcdc494
  Hailey Schoelkopf authored Jul 06, 2023
  
  fdcdc494
- Change `target_delimiter` to `fewshot_delimiter` · 7ffa1d1b
  Lintang Sutawika authored Jul 06, 2023
  
  7ffa1d1b
05 Jul, 2023 15 commits
- Update README.md · 29940d16
  Hailey Schoelkopf authored Jul 05, 2023
  
  29940d16
- Update README.md · 33add999
  Hailey Schoelkopf authored Jul 05, 2023
  
  33add999
- add headqa · 3ab3c388
  haileyschoelkopf authored Jul 05, 2023
  
  3ab3c388
- remove acc_mutual_info · 80a0302a
  haileyschoelkopf authored Jul 05, 2023
  
  80a0302a
- check off Lambada Multilingual from task list · b5697205
  haileyschoelkopf authored Jul 05, 2023
  
  b5697205
- remove batch_size param from config · 38bd2cc8
  haileyschoelkopf authored Jul 05, 2023
  
  38bd2cc8
- Update __init__.py · 797f47c7
  Lintang Sutawika authored Jul 05, 2023
  
  797f47c7
- remove unused line in evaluator.py · 23318d0e
  haileyschoelkopf authored Jul 05, 2023
  
  23318d0e
- fix typo in copa group name · 0e5da3b0
  haileyschoelkopf authored Jul 05, 2023
  
  0e5da3b0
- add lambada_mt tasks · 9735ac34
  haileyschoelkopf authored Jul 05, 2023
  
  9735ac34
- Update task.py · 59b047fb
  Lintang Sutawika authored Jul 05, 2023
  
  59b047fb
- Update task.py · 633fb458
  Lintang Sutawika authored Jul 05, 2023
  
  633fb458
- Update qa4mre status · 42c10bd6
  Hailey Schoelkopf authored Jul 05, 2023
  
  42c10bd6
- raise ValueError when invalid task passed · 7f557daa
  haileyschoelkopf authored Jul 05, 2023
  
  7f557daa
- list out answer choices · c7508401
  haileyschoelkopf authored Jul 05, 2023
  
  c7508401
04 Jul, 2023 10 commits
- revert utils black · 5a75f82a
  FarzanehNakhaee authored Jul 04, 2023
  
  5a75f82a
- black update · d75128dc
  FarzanehNakhaee authored Jul 04, 2023
  
  d75128dc
- resolve newline issue · dd584086
  FarzanehNakhaee authored Jul 04, 2023
  
  dd584086
- resolve new line error · ddc26f34
  FarzanehNakhaee authored Jul 04, 2023
  
  ddc26f34
- address PR comments · b1b5239d
  Benjamin Fattori authored Jul 04, 2023
  
  b1b5239d
- Upload qa4mre_2013 · e6a8846c
  FarzanehNakhaee authored Jul 04, 2023
  
  e6a8846c
- Upload qa4mre_2012 · 09459176
  FarzanehNakhaee authored Jul 04, 2023
  
  09459176
- Update __init__.py · d289277d
  Lintang Sutawika authored Jul 04, 2023
```
Should fix a potential error if a dataset has no `dataset_name` argument.

cc: @haileyschoelkopf
```
  d289277d
- Update doc to target and gold_alias in the config · e8bb77db
  FarzanehNakhaee authored Jul 04, 2023
  
  e8bb77db
- edit doc to text in yaml file · ccae768a
  FarzanehNakhaee authored Jul 04, 2023
  
  ccae768a
03 Jul, 2023 9 commits
- update task readme with completed race · 026d2c21
  Benjamin Fattori authored Jul 04, 2023
  
  026d2c21
- acc_norm not needed for pubmed_qa · 81bdd5bd
  Benjamin Fattori authored Jul 04, 2023
  
  81bdd5bd
- cleanup metric loading code · 328f0e85
  haileyschoelkopf authored Jul 03, 2023
  
  328f0e85
- update race preprocess + rename dataset · 2411c9d3
  Benjamin Fattori authored Jul 03, 2023
  
  2411c9d3
- add race task, with custom HF dataset · 7feab5e3
  Benjamin Fattori authored Jul 03, 2023
  
  7feab5e3
- remove output_base_path · 6a2620ad
  haileyschoelkopf authored Jul 03, 2023
  
  6a2620ad
- Update README.md · 070b6b9c
  Hailey Schoelkopf authored Jul 03, 2023
  
  070b6b9c
- add up-to-date config fields to docs · 97220c9b
  haileyschoelkopf authored Jul 03, 2023
  
  97220c9b
- Update task.py · 0b62e042
  Hailey Schoelkopf authored Jul 03, 2023
  
  0b62e042