Skip to content

GitLab

Commit 6031d7d7 authored by JessicaOjo's avatar JessicaOjo
Browse files

remove test limit

parent 2a285714
Hide whitespace changes
Inline Side-by-side
Showing with 12 additions and 12 deletions
lm_eval/tasks/afrimgsm/gpt_direct.sh
View file @ 6031d7d7
...@@ -9,17 +9,18 @@ task=afrimgsm_direct_eng,afrimgsm_direct_fra,afrimgsm_direct_swa #afrimgsm_direc ...@@ -9,17 +9,18 @@ task=afrimgsm_direct_eng,afrimgsm_direct_fra,afrimgsm_direct_swa #afrimgsm_direc
for model in "${models[@]}" for model in "${models[@]}"
do do
echo "Evaluating model: $model" echo "Evaluating model: $model"
for fewshot in 0 2 4 6 8 # for fewshot in 0 2 4 6 8
do # do
export OUTPUT_DIR=results/$fewshot export OUTPUT_DIR=results/$fewshot
mkdir -p "$OUTPUT_DIR" mkdir -p "$OUTPUT_DIR"
lm_eval --model openai-chat-completions \ lm_eval --model openai-chat-completions \
--model_args model="${model}" \ --model_args model="${model}" \
--tasks $task \ --tasks $task \
--output_path "$OUTPUT_DIR" \ --output_path "$OUTPUT_DIR" \
--num_fewshot $fewshot \ --num_fewshot 0 \
--verbosity DEBUG --limit 5 \
done --verbosity DEBUG
# done
done done
\ No newline at end of file
lm_eval/tasks/afrixnli/en_direct.sh
View file @ 6031d7d7
...@@ -33,7 +33,6 @@ do ...@@ -33,7 +33,6 @@ do
--batch_size 16 \ --batch_size 16 \
--output_path "$OUTPUT_DIR" \ --output_path "$OUTPUT_DIR" \
--num_fewshot $fewshot \ --num_fewshot $fewshot \
--limit 1 \
--verbosity DEBUG --verbosity DEBUG
done done
done done
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment