Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
51a43a76
Commit
51a43a76
authored
Oct 04, 2023
by
haileyschoelkopf
Browse files
change back to original dataset path
parent
54498fcc
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
4 deletions
+4
-4
lm_eval/tasks/bigbench/push_bigbench_dataset.py
lm_eval/tasks/bigbench/push_bigbench_dataset.py
+4
-4
No files found.
lm_eval/tasks/bigbench/push_bigbench_dataset.py
View file @
51a43a76
...
@@ -2,9 +2,9 @@
...
@@ -2,9 +2,9 @@
A utility script that pushes all Bigbench subtasks from their form in the `bigbench` HF dataset
A utility script that pushes all Bigbench subtasks from their form in the `bigbench` HF dataset
into `{org name}/bigbench`.
into `{org name}/bigbench`.
Prior to running,
must
log into HF Hub for the target HF hub org via `huggingface-cli login`.
Prior to running, log into HF Hub for the target HF hub org via `huggingface-cli login`.
Requires the installation of
Requires the installation of
`pip install "bigbench @ https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz"`
`pip install "bigbench @ https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz"`
and is included so that the bigbench dependency can be avoided.
and is included so that the bigbench dependency can be avoided.
"""
"""
...
@@ -20,7 +20,7 @@ num_shots = [0]
...
@@ -20,7 +20,7 @@ num_shots = [0]
for
shots
in
num_shots
:
for
shots
in
num_shots
:
for
task_name
in
tqdm
(
all_task_names
):
for
task_name
in
tqdm
(
all_task_names
):
try
:
try
:
print
(
f
"Loading '
{
task_name
}
' with num_shots=
{
shots
}
..."
)
print
(
f
"Loading '
{
task_name
}
' with num_shots=
{
shots
}
..."
)
task_ds
=
datasets
.
load_dataset
(
"bigbench"
,
name
=
task_name
,
num_shots
=
shots
)
task_ds
=
datasets
.
load_dataset
(
"bigbench"
,
name
=
task_name
,
num_shots
=
shots
)
...
@@ -29,4 +29,4 @@ for shots in num_shots:
...
@@ -29,4 +29,4 @@ for shots in num_shots:
del
task_ds
del
task_ds
except
Exception
as
e
:
except
Exception
as
e
:
raise
e
raise
e
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment