Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
4bf44695
Commit
4bf44695
authored
Jun 19, 2024
by
lintangsutawika
Browse files
update on names
parent
81ec804e
Changes
16
Show whitespace changes
Inline
Side-by-side
Showing
16 changed files
with
23 additions
and
45 deletions
+23
-45
lm_eval/tasks/paloma/paloma.yaml
lm_eval/tasks/paloma/paloma.yaml
+0
-22
lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml
lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml
+2
-2
lm_eval/tasks/paloma/paloma_c4_100_domains.yaml
lm_eval/tasks/paloma/paloma_c4_100_domains.yaml
+2
-2
lm_eval/tasks/paloma/paloma_c4_en.yaml
lm_eval/tasks/paloma/paloma_c4_en.yaml
+1
-1
lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml
lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml
+1
-1
lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml
lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml
+2
-2
lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml
lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml
+2
-2
lm_eval/tasks/paloma/paloma_gab.yaml
lm_eval/tasks/paloma/paloma_gab.yaml
+2
-2
lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml
lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml
+1
-1
lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml
lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml
+1
-1
lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml
lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml
+2
-2
lm_eval/tasks/paloma/paloma_mc4.yaml
lm_eval/tasks/paloma/paloma_mc4.yaml
+2
-2
lm_eval/tasks/paloma/paloma_ptb.yaml
lm_eval/tasks/paloma/paloma_ptb.yaml
+2
-2
lm_eval/tasks/paloma/paloma_redpajama.yaml
lm_eval/tasks/paloma/paloma_redpajama.yaml
+1
-1
lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml
lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml
+1
-1
lm_eval/tasks/paloma/paloma_wikitext_103.yaml
lm_eval/tasks/paloma/paloma_wikitext_103.yaml
+1
-1
No files found.
lm_eval/tasks/paloma/paloma.yaml
deleted
100644 → 0
View file @
81ec804e
group
:
-
paloma
dataset_path
:
allenai/paloma
output_type
:
loglikelihood_rolling
validation_split
:
val
test_split
:
test
doc_to_text
:
"
"
doc_to_target
:
!function
paloma_utils.doc_to_target
should_decontaminate
:
true
doc_to_decontamination_query
:
!function
paloma_utils.doc_to_target
metric_list
:
-
metric
:
word_perplexity
aggregation
:
weighted_perplexity
higher_is_better
:
false
-
metric
:
byte_perplexity
aggregation
:
weighted_perplexity
higher_is_better
:
false
-
metric
:
bits_per_byte
aggregation
:
bits_per_byte
higher_is_better
:
false
metadata
:
version
:
1
lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_4chan_meta_sep
task
:
paloma_4chan_meta_sep
task_alias
:
4chan
Corpus
task_alias
:
4chan
dataset_name
:
4chan_meta_sep
dataset_name
:
4chan_meta_sep
lm_eval/tasks/paloma/paloma_c4_100_domains.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_c4_100_domains
task
:
paloma_c4_100_domains
task_alias
:
C4
-
100
-d
omains
task_alias
:
C4
100
D
omains
dataset_name
:
c4_100_domains
dataset_name
:
c4_100_domains
lm_eval/tasks/paloma/paloma_c4_en.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_c4_en
task
:
paloma_c4_en
task_alias
:
C4
task_alias
:
C4
dataset_name
:
c4_en
dataset_name
:
c4_en
lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_dolma-v1_5
task
:
paloma_dolma-v1_5
task_alias
:
Dolma V1.5
task_alias
:
Dolma V1.5
dataset_name
:
dolma-v1_5
dataset_name
:
dolma-v1_5
lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_dolma_100_subreddits
task
:
paloma_dolma_100_subreddits
task_alias
:
Dolma-
100
-s
ubreddits
task_alias
:
100
S
ubreddits
dataset_name
:
dolma_100_subreddits
dataset_name
:
dolma_100_subreddits
lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_falcon-refinedweb
task
:
paloma_falcon-refinedweb
task_alias
:
Falcon
Refinedweb
task_alias
:
Falcon
dataset_name
:
falcon-refinedweb
dataset_name
:
falcon-refinedweb
lm_eval/tasks/paloma/paloma_gab.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_gab
task
:
paloma_gab
task_alias
:
Gab
Corpus
task_alias
:
Gab
dataset_name
:
gab
dataset_name
:
gab
lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_m2d2_s2orc_unsplit
task
:
paloma_m2d2_s2orc_unsplit
task_alias
:
M2D2 S2ORC
task_alias
:
M2D2 S2ORC
dataset_name
:
m2d2_s2orc_unsplit
dataset_name
:
m2d2_s2orc_unsplit
lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_m2d2_wikipedia_unsplit
task
:
paloma_m2d2_wikipedia_unsplit
task_alias
:
M2D2 Wikipedia
task_alias
:
M2D2 Wikipedia
dataset_name
:
m2d2_wikipedia_unsplit
dataset_name
:
m2d2_wikipedia_unsplit
lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_manosphere_meta_sep
task
:
paloma_manosphere_meta_sep
task_alias
:
Manosphere
Corpus
task_alias
:
Manosphere
dataset_name
:
manosphere_meta_sep
dataset_name
:
manosphere_meta_sep
lm_eval/tasks/paloma/paloma_mc4.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_mc4
task
:
paloma_mc4
task_alias
:
mC4
-en
task_alias
:
mC4
dataset_name
:
mc4
dataset_name
:
mc4
lm_eval/tasks/paloma/paloma_ptb.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_ptb
task
:
paloma_ptb
task_alias
:
P
enn Treebank
task_alias
:
P
TB
dataset_name
:
ptb
dataset_name
:
ptb
lm_eval/tasks/paloma/paloma_redpajama.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_redpajama
task
:
paloma_redpajama
task_alias
:
RedPajama
task_alias
:
RedPajama
dataset_name
:
redpajama
dataset_name
:
redpajama
lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_twitterAAE_HELM_fixed
task
:
paloma_twitterAAE_HELM_fixed
task_alias
:
Twitter AAE
task_alias
:
Twitter AAE
dataset_name
:
twitterAAE_HELM_fixed
dataset_name
:
twitterAAE_HELM_fixed
lm_eval/tasks/paloma/paloma_wikitext_103.yaml
View file @
4bf44695
include
:
paloma
.yaml
include
:
_
paloma
_template
task
:
paloma_wikitext_103
task
:
paloma_wikitext_103
task_alias
:
Wikitext-103
task_alias
:
Wikitext-103
dataset_name
:
wikitext_103
dataset_name
:
wikitext_103
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment