Updating run_squad information in readme

1a5bbd83 · Thomas Wolf · GitHub · 79e1b95e · 1a5bbd83
Unverified Commit 1a5bbd83 authored Nov 06, 2018 by Thomas Wolf Committed by GitHub Nov 06, 2018
Hide whitespace changes
Inline Side-by-side

Showing with 2 additions and 0 deletions

README.md README.md +2 -0

No files found.
--- a/README.md
+++ b/README.md
@@ -194,3 +194,5 @@ python run_squad.py \
  --doc_stride 128 \
  --output_dir ../debug_squad/
 ```
+
+There is currently a bug in the `run_squad.py` script that we are investigating. The reported numbers are very low (F1 of 41.8 and exact match of 21.7) even though the correct answer is usually in the n-best predictions. We are investigating that right now on the develop branch, follow [this issue](https://github.com/huggingface/pytorch-pretrained-BERT/issues/3) for more updates.