Commits · d924ca336fd78df3619b39f633f3f464d9314727 · gaoqiong / lm-evaluation-harness

19 May, 2023 3 commits
- pre-commit stuff · c4c20ff5
  lintangsutawika authored May 19, 2023
  
  c4c20ff5
- able to use prompts from promptsource · e56b950a
  lintangsutawika authored May 19, 2023
  
  e56b950a
- handles prompt from promptsource · 6399ef9e
  lintangsutawika authored May 19, 2023
  
  6399ef9e
18 May, 2023 1 commit
- update fewshot_docs · c37268bb
  lintangsutawika authored May 18, 2023
  
  c37268bb
16 May, 2023 4 commits
- updated filter process · d75da0dd
  lintangsutawika authored May 16, 2023
  
  d75da0dd
- updated filter process · c3dabb32
  lintangsutawika authored May 16, 2023
  
  c3dabb32
- set template_aliases as conditional · 4a3c1f19
  lintangsutawika authored May 16, 2023
  
  4a3c1f19
- fewshot_doc · 7d6b39ed
  lintangsutawika authored May 16, 2023
  
  7d6b39ed
15 May, 2023 2 commits
- detect if doc_to_x is a function · e786cbac
  lintangsutawika authored May 15, 2023
  
  e786cbac
- can process doc_to_text and doc_to_target as function · 66c58194
  lintangsutawika authored May 15, 2023
  
  66c58194
13 May, 2023 1 commit
- modified how yaml and python functions are added to groups and task registry · ca0b8d45
  lintangsutawika authored May 13, 2023
  
  ca0b8d45
12 May, 2023 1 commit
- TaskConfig with yaml · 2196490c
  lintangsutawika authored May 12, 2023
  
  2196490c
11 May, 2023 1 commit
- minor fix · fc09c8f7
  lintangsutawika authored May 11, 2023
  
  fc09c8f7
10 May, 2023 2 commits
- configurable can be overidden with newer configs · cf79b1c6
  lintangsutawika authored May 10, 2023
  
  cf79b1c6
- alternate step so that we can change num_fewshots · cac85281
  lintangsutawika authored May 10, 2023
  
  cac85281
08 May, 2023 2 commits
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
05 May, 2023 2 commits
- add utility func for slicing iterators, only tqdm on main process · eddd627a
  Benjamin Fattori authored May 05, 2023
  
  eddd627a
- sync working changes with upstream · d4c5315a
  Benjamin Fattori authored May 05, 2023
  
  d4c5315a
02 May, 2023 2 commits
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 1 commit
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 2 commits
- added opt8ion to use prompts, removed typos on metric kwargs · 3ec68d38
  lintangsutawika authored Apr 28, 2023
  
  3ec68d38
- can process looglikelihood requests · cea713dc
  lintangsutawika authored Apr 28, 2023
  
  cea713dc
27 Apr, 2023 1 commit
- ported changes here · f107ae29
  lintangsutawika authored Apr 27, 2023
  
  f107ae29
24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759