Commits · ca0b8d45c7943d60b58207912dfe0ac761422079 · gaoqiong / lm-evaluation-harness

13 May, 2023 1 commit
- modified how yaml and python functions are added to groups and task registry · ca0b8d45
  lintangsutawika authored May 13, 2023
  
  ca0b8d45
12 May, 2023 1 commit
- TaskConfig with yaml · 2196490c
  lintangsutawika authored May 12, 2023
  
  2196490c
11 May, 2023 1 commit
- minor fix · fc09c8f7
  lintangsutawika authored May 11, 2023
  
  fc09c8f7
10 May, 2023 2 commits
- configurable can be overidden with newer configs · cf79b1c6
  lintangsutawika authored May 10, 2023
  
  cf79b1c6
- alternate step so that we can change num_fewshots · cac85281
  lintangsutawika authored May 10, 2023
  
  cac85281
08 May, 2023 2 commits
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
02 May, 2023 2 commits
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 1 commit
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 2 commits
- added opt8ion to use prompts, removed typos on metric kwargs · 3ec68d38
  lintangsutawika authored Apr 28, 2023
  
  3ec68d38
- can process looglikelihood requests · cea713dc
  lintangsutawika authored Apr 28, 2023
  
  cea713dc
27 Apr, 2023 1 commit
- ported changes here · f107ae29
  lintangsutawika authored Apr 27, 2023
  
  f107ae29
24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759