- 08 Oct, 2025 1 commit
-
-
Baber authored
-
- 04 Oct, 2025 3 commits
-
-
Baber authored
-
Baber Abbasi authored
* overhaul `ContextSampler` * refactor masakhapos * move multi_target to `exact_match` * remove doc_to_choice from `boolq-seq2seq` * remove doc_to_choice in generation process_results * Remove unused `doc_to_choice` and fix superglue whitespaces * require multiple_inputs and multiple_targets to be explicitly set in taskconfig * fix copa; better logging in task init * fix doc_to_target to return int rather than str (deprecated) * fix processing regression; recursively parse lists fron template * remove redundant jinja parsing logic * remove promptsource * for multiple_inputs use `doc_to_text: list[str]`` * Refactor `ContextSampler` `fewshot_context` * fix multiple_input context * fix `target_delimiter` with `gen_prefix` * `doc_to_text` is list for multiple_inputs * Refactor `count_bytes` and `count_words` methods to `@staticmethod` * make has_*(train/test/validation) to properties * remove `multi_target` `generate_until` * `fix doc_to_target/multiple_targets handling add tests * rename `multi_target` to `multiple_targets` * evalaute list when multiple targets * allow doc_to_target to return list * Remove gen_prefix space and add warning (#3239) * Remove gen_prefix space and add warning * fix null gen_prefix bug again * use git tests --------- Co-authored-by:Boaz Ben-Dov <bendboaz@gmail.com>
-
Baber authored
-
- 29 Sep, 2025 1 commit
-
-
Baber authored
-
- 26 Sep, 2025 2 commits
-
-
Baber authored
-
Baber Abbasi authored
Add `tasklist`
-
- 25 Sep, 2025 33 commits
-
-
Baber authored
-
Baber authored
-
Baber authored
# Conflicts: # lm_eval/__main__.py # lm_eval/api/group.py # lm_eval/api/task.py # lm_eval/evaluator_utils.py # lm_eval/tasks/__init__.py # lm_eval/utils.py # pyproject.toml
-
Baber Abbasi authored
refactor registry
-
Baber authored
# Conflicts: # lm_eval/__init__.py # lm_eval/api/metrics.py # lm_eval/api/registry.py # lm_eval/api/task.py # lm_eval/filters/__init__.py # pyproject.toml
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-
Baber authored
-