Commits · a3d082778e84f8f708cf849ece855d4a4dab5fc0 · gaoqiong / lm-evaluation-harness

27 Nov, 2023 6 commits
- fix precommint · 7fa49671
  baberabb authored Nov 27, 2023
  
  7fa49671
- Update vllm_causallms.py · 08074cf9
  Hailey Schoelkopf authored Nov 27, 2023
  
  08074cf9
- Fix gpu_memory_utilization kwarg typing, change to conditional import · b01c4893
  Hailey Schoelkopf authored Nov 27, 2023
  
  b01c4893
- always import vllm model · d86400b5
  Hailey Schoelkopf authored Nov 27, 2023
  
  d86400b5
- add auto batching · bbd6d051
  baberabb authored Nov 27, 2023
  
  bbd6d051
- add auto batching · 37f10cad
  baberabb authored Nov 27, 2023
  
  37f10cad
22 Nov, 2023 8 commits
- fix args · 4f79fa7b
  baberabb authored Nov 23, 2023
  
  4f79fa7b
- fix tensor parallel · af884028
  baberabb authored Nov 23, 2023
  
  af884028
- fix imports · 5ba54539
  baberabb authored Nov 23, 2023
  
  5ba54539
- fix imports · ce164bb1
  baberabb authored Nov 23, 2023
  
  ce164bb1
- disable tqdm · 667fc837
  baberabb authored Nov 23, 2023
  
  667fc837
- fix logliklehood_rolling · ac664bce
  baberabb authored Nov 23, 2023
  
  ac664bce
- bugfix · 2362ab41
  baberabb authored Nov 22, 2023
  
  2362ab41
- fix greedy_until · e5491709
  baberabb authored Nov 22, 2023
  
  e5491709
21 Nov, 2023 7 commits
- (temp) fix import · 24f4e8d7
  baberabb authored Nov 22, 2023
  
  24f4e8d7
- bugfix · f2890cc1
  baberabb authored Nov 22, 2023
  
  f2890cc1
- bugfix · 0635af13
  baberabb authored Nov 22, 2023
  
  0635af13
- bugfix · 103a10e3
  baberabb authored Nov 21, 2023
  
  103a10e3
- add vllm model · a2920e3d
  baberabb authored Nov 21, 2023
  
  a2920e3d
- add vllm model · 3d8bc82f
  baberabb authored Nov 21, 2023
  
  3d8bc82f
- update multi-token stopsequence handling · f7873a49
  haileyschoelkopf authored Nov 21, 2023
  
  f7873a49
20 Nov, 2023 2 commits
- add typehints · 1e9c8b59
  baberabb authored Nov 20, 2023
  
  1e9c8b59
- add docs · b22f3440
  baberabb authored Nov 20, 2023
  
  b22f3440
17 Nov, 2023 8 commits
- update bigbench datasets to mirror, rename groupnames · 487767ab
  haileyschoelkopf authored Nov 17, 2023
  
  487767ab
- fix error when using wildcard task names · 2444b364
  haileyschoelkopf authored Nov 17, 2023
  
  2444b364
- fix error when using wildcard task names · aa9e1062
  haileyschoelkopf authored Nov 17, 2023
  
  aa9e1062
- format · bf6cf1de
  lintangsutawika authored Nov 17, 2023
  
  bf6cf1de
- will check if group_name is None · 2314ff4f
  lintangsutawika authored Nov 17, 2023
  
  2314ff4f
- precommit · f40b7d0e
  lintangsutawika authored Nov 17, 2023
  
  f40b7d0e
- removed provide_description arg · 0d209e2e
  lintangsutawika authored Nov 17, 2023
  
  0d209e2e
- edits and format · 10cc0a56
  lintangsutawika authored Nov 17, 2023
  
  10cc0a56
16 Nov, 2023 1 commit
- remove provide_description flag · a745d589
  haileyschoelkopf authored Nov 16, 2023
  
  a745d589
15 Nov, 2023 1 commit
- update squad file · c9a3fd3f
  lintangsutawika authored Nov 15, 2023
  
  c9a3fd3f
14 Nov, 2023 2 commits
- removed unused variables · d9230aa8
  lintangsutawika authored Nov 14, 2023
  
  d9230aa8
- enables tasks of different group but same task_alis (for example, if... · 7760573f
  lintangsutawika authored Nov 14, 2023
```
enables tasks of different group but same task_alis (for example, if evaluating on different versions of MMLU
```
  7760573f
10 Nov, 2023 2 commits
- fixes · 69186ef2
  lintangsutawika authored Nov 10, 2023
  
  69186ef2
- added initialize_task and updated where eval_logger is imported from · 056c9d85
  lintangsutawika authored Nov 10, 2023
  
  056c9d85
09 Nov, 2023 3 commits
- updates to scrolls · 51753750
  lintangsutawika authored Nov 09, 2023
  
  51753750
- reformat · 097c9253
  lintangsutawika authored Nov 09, 2023
  
  097c9253
- reformat · 6c812bb5
  lintangsutawika authored Nov 09, 2023
  
  6c812bb5