Commits · 7bb147b53f406f94b2f6e1e5b0601ded263fe8a3 · gaoqiong / lm-evaluation-harness

29 Nov, 2023 1 commit
- change torch req for mps · 7bb147b5
  baberabb authored Nov 29, 2023
  
  7bb147b5
28 Nov, 2023 15 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- Delete lm_eval/tasks/siqa/bigbench.yaml · 331d7c51
  Stella Biderman authored Nov 28, 2023
  
  331d7c51
- Update default.yml · 20463a3f
  Stella Biderman authored Nov 28, 2023
  
  20463a3f
- Rename bigbench.yml to default.yml · 5dcbcd63
  Stella Biderman authored Nov 28, 2023
```
Changing the name since this is not actually the big bench format.
```
  5dcbcd63
- reformat · 5ded4d5b
  lintangsutawika authored Nov 28, 2023
  
  5ded4d5b
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- update version · 43f615c9
  lintangsutawika authored Nov 28, 2023
  
  43f615c9
- update version · 94cdface
  lintangsutawika authored Nov 28, 2023
  
  94cdface
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
- renamed template file to make it easier to search · 19738ed2
  lintangsutawika authored Nov 28, 2023
  
  19738ed2
- renamed template file to make it easier to search · db8928b0
  lintangsutawika authored Nov 28, 2023
  
  db8928b0
- removed flan from the naming · 3b9640b8
  lintangsutawika authored Nov 28, 2023
  
  3b9640b8
27 Nov, 2023 12 commits
- Update README.md · 5535fbd4
  Stella Biderman authored Nov 27, 2023
  
  5535fbd4
- Update README.md · 4706fbb2
  Stella Biderman authored Nov 27, 2023
  
  4706fbb2
- Create bigbench.yml · 11129906
  Stella Biderman authored Nov 27, 2023
  
  11129906
- Create README.md · babdc8d6
  Stella Biderman authored Nov 27, 2023
  
  babdc8d6
- 'flan' in groupnames optional for all variants · 1e5ca7a4
  haileyschoelkopf authored Nov 27, 2023
  
  1e5ca7a4
- make 'bbh' a group name for flan cot 3-shot, add stopseqs · e408c2ce
  haileyschoelkopf authored Nov 27, 2023
  
  e408c2ce
- fix precommint · 7fa49671
  baberabb authored Nov 27, 2023
  
  7fa49671
- Update vllm_causallms.py · 08074cf9
  Hailey Schoelkopf authored Nov 27, 2023
  
  08074cf9
- Fix gpu_memory_utilization kwarg typing, change to conditional import · b01c4893
  Hailey Schoelkopf authored Nov 27, 2023
  
  b01c4893
- always import vllm model · d86400b5
  Hailey Schoelkopf authored Nov 27, 2023
  
  d86400b5
- add auto batching · bbd6d051
  baberabb authored Nov 27, 2023
  
  bbd6d051
- add auto batching · 37f10cad
  baberabb authored Nov 27, 2023
  
  37f10cad
26 Nov, 2023 1 commit
- use Seq2Seq backend where either can be loaded from HF · 7ab782ec
  haileyschoelkopf authored Nov 26, 2023
  
  7ab782ec
24 Nov, 2023 4 commits
- udate · a5b65c9b
  lintangsutawika authored Nov 24, 2023
  
  a5b65c9b
- fix missing lines · b64e592d
  lintangsutawika authored Nov 24, 2023
  
  b64e592d
- gen_kwargs is parsed once · 47933b98
  lintangsutawika authored Nov 24, 2023
  
  47933b98
- update on args · fde2a1d5
  lintangsutawika authored Nov 24, 2023
  
  fde2a1d5
22 Nov, 2023 7 commits
- fix args · 4f79fa7b
  baberabb authored Nov 23, 2023
  
  4f79fa7b
- fix tensor parallel · af884028
  baberabb authored Nov 23, 2023
  
  af884028
- fix imports · 5ba54539
  baberabb authored Nov 23, 2023
  
  5ba54539
- fix imports · ce164bb1
  baberabb authored Nov 23, 2023
  
  ce164bb1
- disable tqdm · 667fc837
  baberabb authored Nov 23, 2023
  
  667fc837
- fix logliklehood_rolling · ac664bce
  baberabb authored Nov 23, 2023
  
  ac664bce
- bugfix · 2362ab41
  baberabb authored Nov 22, 2023
  
  2362ab41