Skip to content

GitLab

Unverified Commit f8ca66fb authored by Lianmin Zheng's avatar Lianmin Zheng Committed by GitHub
Browse files

Update thresholds in test_nightly_gsm8k_eval.py (#3176)

parent 53cef815
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
test/srt/test_nightly_gsm8k_eval.py
View file @ f8ca66fb
...@@ -27,7 +27,7 @@ MODEL_SCORE_THRESHOLDS = { ...@@ -27,7 +27,7 @@ MODEL_SCORE_THRESHOLDS = {
"google/gemma-2-27b-it": 0.92, "google/gemma-2-27b-it": 0.92,
"meta-llama/Llama-3.1-70B-Instruct": 0.95, "meta-llama/Llama-3.1-70B-Instruct": 0.95,
"mistralai/Mixtral-8x7B-Instruct-v0.1": 0.63, "mistralai/Mixtral-8x7B-Instruct-v0.1": 0.63,
"Qwen/Qwen2-57B-A14B-Instruct": 0.87, "Qwen/Qwen2-57B-A14B-Instruct": 0.86,
"neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8": 0.83, "neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8": 0.83,
"neuralmagic/Mistral-7B-Instruct-v0.3-FP8": 0.54, "neuralmagic/Mistral-7B-Instruct-v0.3-FP8": 0.54,
"neuralmagic/DeepSeek-Coder-V2-Lite-Instruct-FP8": 0.84, "neuralmagic/DeepSeek-Coder-V2-Lite-Instruct-FP8": 0.84,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment