Commits · abd172769e7e563587f53ed01297520d87eed887 · gaoqiong / lm-evaluation-harness

25 Sep, 2025 31 commits
- add docs · 9ec26934
  Baber authored Jul 29, 2025
  
  9ec26934
- better placeholder materialization · 124d3049
  Baber authored Jul 28, 2025
  
  124d3049
- refactor registry for simplicity and improved maintainability · 9af24b7e
  Baber authored Jul 28, 2025
  
  9af24b7e
- refactor registry to simplify API and improve clarity · 907f5f28
  Baber authored Jul 28, 2025
  
  907f5f28
- cleanup and and add types · e9451269
  Baber authored Jul 28, 2025
  
  e9451269
- add better type safety · 48eabc04
  Baber authored Jul 28, 2025
  
  48eabc04
- refactor registry · 93b2ab37
  Baber authored Jul 27, 2025
  
  93b2ab37
- fix process_results · 73202a2e
  Baber authored Sep 24, 2025
  
  73202a2e
- update default values; fixes · b89af51e
  Baber authored Jul 10, 2025
  
  b89af51e
- add subcommands · 61520ad6
  Baber authored Jul 04, 2025
  
  61520ad6
- modularize cli · f9d5d3e7
  Baber authored Jul 03, 2025
  
  f9d5d3e7
- move test one doc to method · 7cef4d38
  Baber authored Jul 23, 2025
  
  7cef4d38
- overload Task methods if callable in yaml dict · ec767666
  Baber authored Jul 23, 2025
  
  ec767666
- remove deps; types · 4ad6cd9f
  Baber authored Jul 22, 2025
  
  4ad6cd9f
- make multiple_input explicit · 689e0c91
  Baber authored Jul 22, 2025
  
  689e0c91
- `check_gold_index_error` util; fix `process_results`; rm generate_until multiple-choice · d9876b22
  Baber authored Jul 22, 2025
  
  d9876b22
- improve metric aggregation default and higher-better checks; add `TaskConfig.from_template` · d19bd889
  Baber authored Jul 21, 2025
  
  d19bd889
- cleanup · 69d14fb3
  Baber authored Jul 21, 2025
  
  69d14fb3
- move multi_target to `exact_match`; `nq_open` · 57b8c0b1
  Baber authored Jul 21, 2025
  
  57b8c0b1
- ruff rules; types · 1768fd3b
  Baber authored Jul 19, 2025
  
  1768fd3b
- remove prompt-source for now · 70f5e2f0
  Baber authored Jul 18, 2025
  
  70f5e2f0
- refactor: improve dataset and metric handling in TaskConfig · 227f1a74
  Baber authored Jul 08, 2025
  
  227f1a74
- refactor: update type hints and improve filter ensemble construction · 3b4d0af1
  Baber authored Jul 08, 2025
  
  3b4d0af1
- cleanup · c81c03ee
  Baber authored Jul 08, 2025
  
  c81c03ee
- refactor configs to files · 57adbd35
  Baber authored Jul 04, 2025
  
  57adbd35
- cleanup · 04e74420
  Baber authored Jul 03, 2025
  
  04e74420
- add temlplateconfigs · b0173d57
  Baber authored Jul 01, 2025
  
  b0173d57
- update type hints · bbf79d44
  Baber authored Jun 30, 2025
  
  bbf79d44
- add `sample_metric` and `is_elementwise` to MetricConfig · 7f7872c1
  Baber authored Jun 30, 2025
  
  7f7872c1
- add FewshotConfig · 9c647fc1
  Baber authored Jun 30, 2025
  
  9c647fc1
- add MetricConfig · 28c78d30
  Baber authored Jun 30, 2025
  
  28c78d30
26 Jul, 2025 1 commit
- add task factory · 4254c7bd
  Baber authored Jul 26, 2025
  
  4254c7bd
23 Jul, 2025 1 commit

Baber Abbasi authored Jul 23, 2025

* Fix: pin datasets < 4.0

* fix

* update type hints in HF

* fix hellaswag path

904bba12

14 Jul, 2025 1 commit
- Fix for hang due to mp.Pool in bootstrap_stderr (#3135) · cf631de0
  Ankit Gola authored Jul 14, 2025
  
  cf631de0
12 Jul, 2025 2 commits
- refactor: replace ConfigurableGroup with GroupConfig · 0aca6958
  Baber authored Jul 13, 2025
  
  0aca6958
- refactor: simplify docstrings and improve task name matching logic · 7fcfb4ac
  Baber authored Jul 12, 2025
  
  7fcfb4ac
11 Jul, 2025 2 commits
- fix circular · 6fc2ac49
  Baber authored Jul 12, 2025
  
  6fc2ac49
- refactor: migrate utils functions to lm_eval.tasks and update references · a9c16905
  Baber authored Jul 12, 2025
  
  a9c16905
05 Jul, 2025 1 commit
- remove all; reformat table (#3107) · 28001d29
  Baber Abbasi authored Jul 05, 2025
  
  28001d29
04 Jul, 2025 1 commit

[FIX] Initial code to disable multi-proc for stderr (#3106) · 71d0289d

Neel Gupta authored Jul 04, 2025



* [FIX] Initial code to disable multi-proc for stderr

* add docs; align no-mp bootstrap with mp

---------
Co-authored-by: Baber <baber@hey.com>

71d0289d