Skip to content

GitLab

Commit dfb6a9b3 authored by Mostofa Patwary's avatar Mostofa Patwary
Browse files

updated the evaluation script for retriver

parent 7577931b
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 4 deletions
examples/evaluate_ict_zeroshot_nq.sh → examples/evaluate_retriever_nq.sh
View file @ dfb6a9b3
#!/bin/bash #!/bin/bash
# Evaluate natural question test data given Wikipedia embeddings and pretrained # Evaluate natural question test data given Wikipedia embeddings and pretrained
# ICT model # ICT model or a finetuned model for Natural Question task
# Datasets can be downloaded from the following link: # Datasets can be downloaded from the following link:
# https://github.com/facebookresearch/DPR/blob/master/data/download_data.py # https://github.com/facebookresearch/DPR/blob/master/data/download_data.py
EVIDENCE_DATA_DIR=<Specify path of Wikipedia dataset> EVIDENCE_DATA_DIR=<Specify path of Wikipedia dataset>
EMBEDDING_PATH=<Specify path of the embeddings> EMBEDDING_PATH=<Specify path of the embeddings>
CHECKPOINT_PATH=<Specify path of pretrained ICT model> CHECKPOINT_PATH=<Specify path of pretrained ICT model or finetuned model>
QA_FILE=<Path of the natural question test dataset> QA_FILE=<Path of the natural question dev or test dataset>
python tasks/main.py \ python tasks/main.py \
--task ICT-ZEROSHOT-NQ \ --task RETRIEVER-EVAL \
--tokenizer-type BertWordPieceLowerCase \ --tokenizer-type BertWordPieceLowerCase \
--num-layers 12 \ --num-layers 12 \
--hidden-size 768 \ --hidden-size 768 \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment