Unverified Commit cbc31eb8 authored by Wonseok Hwang's avatar Wonseok Hwang Committed by GitHub
Browse files

kbl-v0.1.1 (#2493)

* release kbl-v0.1

* fix linting

* remove rag tasks as  doc_to_text functions cause trouble

* remove remaining rag tasks

* remove unnecessary repeat in yaml files and rag dataset in hf-hub

* remove unncessary newline; introduce cfg files in lbox/kbl in hf

* Make task yaml files consistent to hf-datasets-config

* Make task yaml files consistent to hf-datasets-config

* Remove trailing empty space in doc-to-text

* Remove unncessary yaml file

* Fix task nameing error

* trailing space removed
parent badf273a
task: kbl_bar_exam_em_public_2021
dataset_name: bar_exam_public_2021
include: _base_em_yaml
task: kbl_bar_exam_em_public_2022
dataset_name: bar_exam_public_2022
include: _base_em_yaml
task: kbl_bar_exam_em_public_2023
dataset_name: bar_exam_public_2023
include: _base_em_yaml
task: kbl_bar_exam_em_public_2024
dataset_name: bar_exam_public_2024
include: _base_em_yaml
tag:
- kbl
- kbl_bar_exam_em
- kbl_bar_exam_em_responsibility
description: '당신은 사용자의 질문에 친절하고 논리적으로 답변해 주는 법률 전문가 챗봇 입니다.\n'
dataset_path: lbox/kbl
test_split: test
output_type: generate_until
doc_to_text: '### 질문: {{question}}
다음 각 선택지를 읽고 A, B, C, D 중 하나를 선택하여 ''답변: A'' 와 같이 단답식으로 답해 주세요.
A. {{A}}
B. {{B}}
C. {{C}}
D. {{D}}
### 답변:'
doc_to_target: gt
metric_list:
- metric: exact_match
aggregation: mean
higher_is_better: true
ignore_case: true
ignore_punctuation: true
filter_list:
- name: get-answer
filter:
- function: regex
regex_pattern: ([A-D]).*
- function: take_first
task: kbl_bar_exam_em_responsibility_2010
dataset_name: bar_exam_responsibility_2010
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2011
dataset_name: bar_exam_responsibility_2011
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2012
dataset_name: bar_exam_responsibility_2012
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2013
dataset_name: bar_exam_responsibility_2013
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2014
dataset_name: bar_exam_responsibility_2014
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2015
dataset_name: bar_exam_responsibility_2015
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2016
dataset_name: bar_exam_responsibility_2016
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2017
dataset_name: bar_exam_responsibility_2017
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2018
dataset_name: bar_exam_responsibility_2018
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2019
dataset_name: bar_exam_responsibility_2019
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2020
dataset_name: bar_exam_responsibility_2020
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2021
dataset_name: bar_exam_responsibility_2021
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2022
dataset_name: bar_exam_responsibility_2022
include: _base_em_yaml
task: kbl_bar_exam_em_responsibility_2023
dataset_name: bar_exam_responsibility_2023
include: _base_em_yaml
tag:
- kbl
- kbl_knowledge_em
description: '당신은 사용자의 질문에 친절하고 논리적으로 답변해 주는 법률 전문가 챗봇 입니다.\n'
dataset_path: lbox/kbl
test_split: test
output_type: generate_until
doc_to_target: "{{label}}"
metric_list:
- metric: exact_match
aggregation: mean
higher_is_better: true
ignore_case: true
ignore_punctuation: true
filter_list:
- name: "get-answer"
filter:
- function: "regex"
regex_pattern: "([A-E]).*"
- function: "take_first"
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment