Unverified Commit 3d9bb4ae authored by bittersweet1999's avatar bittersweet1999 Committed by GitHub
Browse files

[Fix] fix strings (#833)



* add compass arena

* add compass_arena

* add compass arena

* Update opencompass/summarizers/subjective/compass_arena.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/compass_arena.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>

* fix check position bias

* fix string

---------
Co-authored-by: default avatarSongyang Zhang <tonysy@users.noreply.github.com>
parent 2d4da8dd
...@@ -117,7 +117,7 @@ class CompassArenaSummarizer: ...@@ -117,7 +117,7 @@ class CompassArenaSummarizer:
'answer2'] 'answer2']
for prediction, reference in zip(judged_answers, for prediction, reference in zip(judged_answers,
references): references):
if dataset_abbr == 'zhihu_hot_0113': if dataset_abbr == 'qa':
reference['capability'] = 'QA' reference['capability'] = 'QA'
categories['total'] += 1 categories['total'] += 1
categories[reference['capability']] += 1 categories[reference['capability']] += 1
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment