"python/sglang/git@developer.sourcefind.cn:change/sglang.git" did not exist on "2187f36237eb532f7a9eab92c198ebd3571e1494"
Unverified Commit f78fcf6e authored by Fengzhe Zhou's avatar Fengzhe Zhou Committed by GitHub
Browse files

[Docs] Update contamination docs (#775)

parent 0b286303
...@@ -11,5 +11,5 @@ with read_base(): ...@@ -11,5 +11,5 @@ with read_base():
from .summarizers.contamination import summarizer from .summarizers.contamination import summarizer
datasets = [*ceval_datasets] datasets = [*ceval_datasets, *mmlu_datasets, *hellaswag_datasets, *ARC_c_datasets]
models = [*hf_yi_6b_model, *hf_qwen_7b_model, *hf_llama2_7b_model] models = [*hf_yi_6b_model, *hf_qwen_7b_model, *hf_llama2_7b_model]
...@@ -102,7 +102,7 @@ ceval-hard - ppl 44.35 ...@@ -102,7 +102,7 @@ ceval-hard - ppl 44.35
ceval - ppl 67.32 71.01 81.17 58.97 49.28 67.82 ... ceval - ppl 67.32 71.01 81.17 58.97 49.28 67.82 ...
``` ```
Currently, this solution only supports the C-Eval dataset. [Contamination_Detector](https://github.com/liyucheng09/Contamination_Detector) also includes ARC, CSQA, HellaSwag, MMLU, and WinoGrande, but these have not yet been implemented in OpenCompass. We welcome the community to contribute more datasets. Currently, this solution only supports the C-Eval, MMLU, HellaSwag and ARC. [Contamination_Detector](https://github.com/liyucheng09/Contamination_Detector) also includes CSQA and WinoGrande, but these have not yet been implemented in OpenCompass. We welcome the community to contribute more datasets.
Consider cite the following paper if you find it helpful: Consider cite the following paper if you find it helpful:
......
...@@ -100,7 +100,7 @@ ceval-hard - ppl 44.35 ...@@ -100,7 +100,7 @@ ceval-hard - ppl 44.35
ceval - ppl 67.32 71.01 81.17 58.97 49.28 67.82 ... ceval - ppl 67.32 71.01 81.17 58.97 49.28 67.82 ...
``` ```
目前该方案仅支持 C-Eval 数据集,[Contamination_Detector](https://github.com/liyucheng09/Contamination_Detector) 中还包含了 ARC, CSQA, HellaSwag, MMLU 和 WinoGrande,但目前还没有在 OpenCompass 中实现。我们欢迎社区贡献更多的数据集。 目前该方案仅支持 C-Eval, MMLU, HellaSwag 和 ARC 数据集,[Contamination_Detector](https://github.com/liyucheng09/Contamination_Detector) 中还包含了 CSQA 和 WinoGrande,但目前还没有在 OpenCompass 中实现。我们欢迎社区贡献更多的数据集。
如果使用了该方法,请添加引用: 如果使用了该方法,请添加引用:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment