Commits · 8171906d0ca0d1741a2f7858cd3f45cc3995b816 · gaoqiong / lm-evaluation-harness

"llm/llama.cpp/examples/llama.android/build.gradle.kts" did not exist on "b0135f4b9b176eab9155b660d04c9ca2a1ec2341"

08 Jun, 2023 2 commits

[Refactor] Non-greedy generation ; WIP GSM8k yaml (#559) · 232632c6

Hailey Schoelkopf authored Jun 07, 2023

* add wip gsm8k yaml

* cleanup tasks dir

* push gsm8k yaml changes

* rename gpt2.py

* add updated gsm8k , triviaqa baseline

* add new cot yaml

* allow for multiple filter pipelines, new filter types

* updated gsm8k + sampling gen configs

* cleanup self-consistency yaml

232632c6

fixed acc_mutula_info calc position · a2e41158
lintangsutawika authored Jun 08, 2023

a2e41158

07 Jun, 2023 7 commits
- fixed register_model origin and other imports · 424a4280
  lintangsutawika authored Jun 07, 2023
  
  424a4280
- fixed error in aggregation · c75ce009
  lintangsutawika authored Jun 07, 2023
  
  c75ce009
- modified import orgin · a22d8ffa
  lintangsutawika authored Jun 07, 2023
  
  a22d8ffa
- MODEL_REGISTRY to registry.py · f2166089
  lintangsutawika authored Jun 07, 2023
  
  f2166089
- moved METRIC_REGISTRY to registry.py · 9e75a84d
  lintangsutawika authored Jun 07, 2023
  
  9e75a84d
- removed import metric · 9b39df9b
  lintangsutawika authored Jun 07, 2023
  
  9b39df9b
- renamed register.py to registry.py · 79db4088
  lintangsutawika authored Jun 07, 2023
  
  79db4088
06 Jun, 2023 4 commits
- added more registry · 7d0e3ebf
  lintangsutawika authored Jun 06, 2023
  
  7d0e3ebf
- mejor revision into how metrics are properly loaded for all output_types · ba41ab7c
  lintangsutawika authored Jun 06, 2023
  
  ba41ab7c
- metrics are now in a special folder so that registry can work better · b3591562
  lintangsutawika authored Jun 06, 2023
  
  b3591562
- much better way to process all metrics chosen · 0a9ad6ee
  lintangsutawika authored Jun 06, 2023
  
  0a9ad6ee
05 Jun, 2023 4 commits
- fixed typo · 6d7a24b2
  lintangsutawika authored Jun 05, 2023
  
  6d7a24b2
- revert to take_first filter · 47b888bb
  lintangsutawika authored Jun 05, 2023
  
  47b888bb
- fixing metric for each output_type · c746d1fb
  lintangsutawika authored Jun 05, 2023
  
  c746d1fb
- allow to use alternative methods to use hf datasets, allow configuration with dataset_kwargs · a339ffd8
  lintangsutawika authored Jun 05, 2023
  
  a339ffd8
02 Jun, 2023 2 commits
- fix default .py task filters · f61974ed
  haileyschoelkopf authored Jun 02, 2023
  
  f61974ed
- result_dict process · 6acc5c47
  lintangsutawika authored Jun 02, 2023
  
  6acc5c47
19 May, 2023 3 commits
- pre-commit stuff · c4c20ff5
  lintangsutawika authored May 19, 2023
  
  c4c20ff5
- able to use prompts from promptsource · e56b950a
  lintangsutawika authored May 19, 2023
  
  e56b950a
- handles prompt from promptsource · 6399ef9e
  lintangsutawika authored May 19, 2023
  
  6399ef9e
18 May, 2023 1 commit
- update fewshot_docs · c37268bb
  lintangsutawika authored May 18, 2023
  
  c37268bb
16 May, 2023 4 commits
- updated filter process · d75da0dd
  lintangsutawika authored May 16, 2023
  
  d75da0dd
- updated filter process · c3dabb32
  lintangsutawika authored May 16, 2023
  
  c3dabb32
- set template_aliases as conditional · 4a3c1f19
  lintangsutawika authored May 16, 2023
  
  4a3c1f19
- fewshot_doc · 7d6b39ed
  lintangsutawika authored May 16, 2023
  
  7d6b39ed
15 May, 2023 3 commits
- detect if doc_to_x is a function · e786cbac
  lintangsutawika authored May 15, 2023
  
  e786cbac
- can process doc_to_text and doc_to_target as function · 66c58194
  lintangsutawika authored May 15, 2023
  
  66c58194
- removed commented lines · 3cdc47c5
  lintangsutawika authored May 15, 2023
  
  3cdc47c5
13 May, 2023 1 commit
- modified how yaml and python functions are added to groups and task registry · ca0b8d45
  lintangsutawika authored May 13, 2023
  
  ca0b8d45
12 May, 2023 1 commit
- TaskConfig with yaml · 2196490c
  lintangsutawika authored May 12, 2023
  
  2196490c
11 May, 2023 1 commit
- minor fix · fc09c8f7
  lintangsutawika authored May 11, 2023
  
  fc09c8f7
10 May, 2023 3 commits
- moved METRICS related dicts · 35730ace
  lintangsutawika authored May 10, 2023
  
  35730ace
- configurable can be overidden with newer configs · cf79b1c6
  lintangsutawika authored May 10, 2023
  
  cf79b1c6
- alternate step so that we can change num_fewshots · cac85281
  lintangsutawika authored May 10, 2023
  
  cac85281
08 May, 2023 2 commits
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
05 May, 2023 2 commits
- add utility func for slicing iterators, only tqdm on main process · eddd627a
  Benjamin Fattori authored May 05, 2023
  
  eddd627a
- sync working changes with upstream · d4c5315a
  Benjamin Fattori authored May 05, 2023
  
  d4c5315a