Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
ae4d9ed2
Commit
ae4d9ed2
authored
Jun 08, 2023
by
lintangsutawika
Browse files
changes for pre-commit
parent
6b72d7b7
Changes
36
Show whitespace changes
Inline
Side-by-side
Showing
16 changed files
with
28 additions
and
19 deletions
+28
-19
lm_eval/tasks/pile/pile_opensubtitles.yaml
lm_eval/tasks/pile/pile_opensubtitles.yaml
+1
-1
lm_eval/tasks/pile/pile_openwebtext2.yaml
lm_eval/tasks/pile/pile_openwebtext2.yaml
+1
-1
lm_eval/tasks/pile/pile_philpapers.yaml
lm_eval/tasks/pile/pile_philpapers.yaml
+1
-1
lm_eval/tasks/pile/pile_pile-cc.yaml
lm_eval/tasks/pile/pile_pile-cc.yaml
+1
-1
lm_eval/tasks/pile/pile_pubmed-abstracts.yaml
lm_eval/tasks/pile/pile_pubmed-abstracts.yaml
+1
-1
lm_eval/tasks/pile/pile_pubmed-central.yaml
lm_eval/tasks/pile/pile_pubmed-central.yaml
+1
-1
lm_eval/tasks/pile/pile_stackexchange.yaml
lm_eval/tasks/pile/pile_stackexchange.yaml
+1
-1
lm_eval/tasks/pile/pile_ubuntu-irc.yaml
lm_eval/tasks/pile/pile_ubuntu-irc.yaml
+1
-1
lm_eval/tasks/pile/pile_uspto.yaml
lm_eval/tasks/pile/pile_uspto.yaml
+1
-1
lm_eval/tasks/pile/pile_wikipedia.yaml
lm_eval/tasks/pile/pile_wikipedia.yaml
+1
-1
lm_eval/tasks/pile/pile_youtubesubtitles.yaml
lm_eval/tasks/pile/pile_youtubesubtitles.yaml
+1
-1
lm_eval/tasks/piqa/piqa.yaml
lm_eval/tasks/piqa/piqa.yaml
+2
-2
lm_eval/tasks/sciq/sciq.yaml
lm_eval/tasks/sciq/sciq.yaml
+3
-3
lm_eval/tasks/wikitext/README.md
lm_eval/tasks/wikitext/README.md
+1
-1
lm_eval/tasks/wikitext/preprocess_wikitext.py
lm_eval/tasks/wikitext/preprocess_wikitext.py
+1
-0
lm_eval/utils.py
lm_eval/utils.py
+10
-2
No files found.
lm_eval/tasks/pile/pile_opensubtitles.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_openwebtext2.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_philpapers.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_pile-cc.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_pubmed-abstracts.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_pubmed-central.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_stackexchange.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_ubuntu-irc.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_uspto.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_wikipedia.yaml
View file @
ae4d9ed2
lm_eval/tasks/pile/pile_youtubesubtitles.yaml
View file @
ae4d9ed2
lm_eval/tasks/piqa/piqa.yaml
View file @
ae4d9ed2
lm_eval/tasks/sciq/sciq.yaml
View file @
ae4d9ed2
lm_eval/tasks/wikitext/README.md
View file @
ae4d9ed2
lm_eval/tasks/wikitext/preprocess_wikitext.py
View file @
ae4d9ed2
import
re
import
re
def
wikitext_detokenizer
(
doc
):
def
wikitext_detokenizer
(
doc
):
string
=
doc
[
"page"
]
string
=
doc
[
"page"
]
# contractions
# contractions
...
...
lm_eval/utils.py
View file @
ae4d9ed2
...
@@ -158,7 +158,15 @@ def make_table(result_dict):
...
@@ -158,7 +158,15 @@ def make_table(result_dict):
md_writer
=
MarkdownTableWriter
()
md_writer
=
MarkdownTableWriter
()
latex_writer
=
LatexTableWriter
()
latex_writer
=
LatexTableWriter
()
md_writer
.
headers
=
[
"Task"
,
"Version"
,
"Filter"
,
"Metric"
,
"Value"
,
""
,
"Stderr"
]
md_writer
.
headers
=
[
"Task"
,
"Version"
,
"Filter"
,
"Metric"
,
"Value"
,
""
,
"Stderr"
]
latex_writer
.
headers
=
[
"Task"
,
"Version"
,
"Filter"
,
"Metric"
,
"Value"
,
""
,
"Stderr"
]
latex_writer
.
headers
=
[
"Task"
,
"Version"
,
"Filter"
,
"Metric"
,
"Value"
,
""
,
"Stderr"
,
]
values
=
[]
values
=
[]
...
@@ -166,7 +174,7 @@ def make_table(result_dict):
...
@@ -166,7 +174,7 @@ def make_table(result_dict):
version
=
result_dict
[
"versions"
][
k
]
version
=
result_dict
[
"versions"
][
k
]
for
(
mf
),
v
in
dic
.
items
():
for
(
mf
),
v
in
dic
.
items
():
m
,
_
,
f
=
mf
.
partition
(
","
)
m
,
_
,
f
=
mf
.
partition
(
","
)
print
(
m
,
f
)
print
(
m
,
f
)
if
m
.
endswith
(
"_stderr"
):
if
m
.
endswith
(
"_stderr"
):
continue
continue
...
...
Prev
1
2
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment