Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
dc5b3d5d
Unverified
Commit
dc5b3d5d
authored
Nov 28, 2023
by
Stella Biderman
Committed by
GitHub
Nov 28, 2023
Browse files
Merge pull request #1031 from EleutherAI/versioning
[Refactor] Versioning
parents
39c2bb4e
52f75f0e
Changes
128
Hide whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
16 additions
and
0 deletions
+16
-0
lm_eval/tasks/wikitext/wikitext.yaml
lm_eval/tasks/wikitext/wikitext.yaml
+2
-0
lm_eval/tasks/winogrande/default.yaml
lm_eval/tasks/winogrande/default.yaml
+2
-0
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
+2
-0
lm_eval/tasks/wsc273/default.yaml
lm_eval/tasks/wsc273/default.yaml
+2
-0
lm_eval/tasks/xcopa/default_et.yaml
lm_eval/tasks/xcopa/default_et.yaml
+2
-0
lm_eval/tasks/xnli/xnli_common_yaml
lm_eval/tasks/xnli/xnli_common_yaml
+2
-0
lm_eval/tasks/xstorycloze/default_ar.yaml
lm_eval/tasks/xstorycloze/default_ar.yaml
+2
-0
lm_eval/tasks/xwinograd/xwinograd_common_yaml
lm_eval/tasks/xwinograd/xwinograd_common_yaml
+2
-0
No files found.
lm_eval/tasks/wikitext/wikitext.yaml
View file @
dc5b3d5d
...
...
@@ -14,3 +14,5 @@ metric_list:
-
metric
:
word_perplexity
-
metric
:
byte_perplexity
-
metric
:
bits_per_byte
metadata
:
-
version
:
2.0
lm_eval/tasks/winogrande/default.yaml
View file @
dc5b3d5d
...
...
@@ -13,3 +13,5 @@ metric_list:
-
metric
:
acc
aggregation
:
mean
higher_is_better
:
true
metadata
:
-
version
:
1.0
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
View file @
dc5b3d5d
...
...
@@ -15,3 +15,5 @@ metric_list:
-
metric
:
!function
metrics.bleu
aggregation
:
!function
metrics.agg_bleu
higher_is_better
:
true
metadata
:
-
version
:
0.0
lm_eval/tasks/wsc273/default.yaml
View file @
dc5b3d5d
...
...
@@ -13,3 +13,5 @@ metric_list:
-
metric
:
acc
aggregation
:
mean
higher_is_better
:
true
metadata
:
-
version
:
1.0
lm_eval/tasks/xcopa/default_et.yaml
View file @
dc5b3d5d
...
...
@@ -10,3 +10,5 @@ doc_to_target: label
doc_to_choice
:
!function
utils.doc_to_choice
metric_list
:
-
metric
:
acc
metadata
:
-
version
:
1.0
lm_eval/tasks/xnli/xnli_common_yaml
View file @
dc5b3d5d
...
...
@@ -15,3 +15,5 @@ metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
metadata:
- version: 1.0
lm_eval/tasks/xstorycloze/default_ar.yaml
View file @
dc5b3d5d
...
...
@@ -14,3 +14,5 @@ metric_list:
-
metric
:
acc
aggregation
:
mean
higher_is_better
:
true
metadata
:
-
version
:
1.0
lm_eval/tasks/xwinograd/xwinograd_common_yaml
View file @
dc5b3d5d
...
...
@@ -16,3 +16,5 @@ metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
metadata:
- version: 1.0
Prev
1
…
3
4
5
6
7
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment