Commits · a57ffba1be6aab24b1f4ff5ad264ca7fc37cc00f · gaoqiong / lm-evaluation-harness

25 Sep, 2025 40 commits
- fixup! merge · bcd6faaa
  Baber authored Sep 26, 2025
  
  bcd6faaa
- fix · 58fc07b1
  Baber authored Sep 26, 2025
  
  58fc07b1
- init model registry on import · d547b663
  Baber authored Aug 05, 2025
  
  d547b663
- add docs · 9ec26934
  Baber authored Jul 29, 2025
  
  9ec26934
- add tests · 2b32f7be
  Baber authored Jul 28, 2025
  
  2b32f7be
- better placeholder materialization · 124d3049
  Baber authored Jul 28, 2025
  
  124d3049
- refactor registry for simplicity and improved maintainability · 9af24b7e
  Baber authored Jul 28, 2025
  
  9af24b7e
- refactor registry to simplify API and improve clarity · 907f5f28
  Baber authored Jul 28, 2025
  
  907f5f28
- cleanup and and add types · e9451269
  Baber authored Jul 28, 2025
  
  e9451269
- add better type safety · 48eabc04
  Baber authored Jul 28, 2025
  
  48eabc04
- refactor registry · 93b2ab37
  Baber authored Jul 27, 2025
  
  93b2ab37
- fix process_results · 73202a2e
  Baber authored Sep 24, 2025
  
  73202a2e
- add create_cloze_choices function · 7551cc34
  Baber authored Sep 08, 2025
  
  7551cc34
- update template · 4097aad3
  Baber authored Sep 08, 2025
  
  4097aad3
- add todo · d183d244
  Baber authored Aug 25, 2025
  
  d183d244
- update default values; fixes · b89af51e
  Baber authored Jul 10, 2025
  
  b89af51e
- add tests · fadd26e4
  Baber authored Jul 04, 2025
  
  fadd26e4
- fix logging · 649ca8fc
  Baber authored Jul 04, 2025
  
  649ca8fc
- add subcommands · 61520ad6
  Baber authored Jul 04, 2025
  
  61520ad6
- modularize cli · f9d5d3e7
  Baber authored Jul 03, 2025
  
  f9d5d3e7
- types · 223b9488
  Baber authored Jul 23, 2025
  
  223b9488
- move test one doc to method · 7cef4d38
  Baber authored Jul 23, 2025
  
  7cef4d38
- overload Task methods if callable in yaml dict · ec767666
  Baber authored Jul 23, 2025
  
  ec767666
- update `scrolls` · 2009ec4b
  Baber authored Jul 23, 2025
  
  2009ec4b
- remove deps; types · 4ad6cd9f
  Baber authored Jul 22, 2025
  
  4ad6cd9f
- make multiple_input explicit · 689e0c91
  Baber authored Jul 22, 2025
  
  689e0c91
- `check_gold_index_error` util; fix `process_results`; rm generate_until multiple-choice · d9876b22
  Baber authored Jul 22, 2025
  
  d9876b22
- improve metric aggregation default and higher-better checks; add `TaskConfig.from_template` · d19bd889
  Baber authored Jul 21, 2025
  
  d19bd889
- cleanup · 69d14fb3
  Baber authored Jul 21, 2025
  
  69d14fb3
- move multi_target to `exact_match`; `nq_open` · 57b8c0b1
  Baber authored Jul 21, 2025
  
  57b8c0b1
- refactor masakhapos · 7c585f96
  Baber authored Jul 21, 2025
  
  7c585f96
- ruff rules; types · 1768fd3b
  Baber authored Jul 19, 2025
  
  1768fd3b
- refactor build_filter_ensemble to simplify filter creation · f650197a
  Baber authored Jul 18, 2025
  
  f650197a
- remove prompt-source for now · 70f5e2f0
  Baber authored Jul 18, 2025
  
  70f5e2f0
- refactor: improve dataset and metric handling in TaskConfig · 227f1a74
  Baber authored Jul 08, 2025
  
  227f1a74
- refactor: update type hints and improve filter ensemble construction · 3b4d0af1
  Baber authored Jul 08, 2025
  
  3b4d0af1
- cleanup · c81c03ee
  Baber authored Jul 08, 2025
  
  c81c03ee
- serialize better · 674611e9
  Baber authored Jul 05, 2025
  
  674611e9
- refactor configs to files · 57adbd35
  Baber authored Jul 04, 2025
  
  57adbd35
- cleanup · 04e74420
  Baber authored Jul 03, 2025
  
  04e74420