Commit 52371ffe authored by A. Unique TensorFlower's avatar A. Unique TensorFlower
Browse files

Fix QQP preprocessing for test set. Test set does not contain qid1 or qid2...

Fix QQP preprocessing for test set. Test set does not contain qid1 or qid2 columns, so the current try/except block causes the entire test set to be ignored.

PiperOrigin-RevId: 326259054
parent fa66c645
...@@ -438,6 +438,12 @@ class QqpProcessor(DataProcessor): ...@@ -438,6 +438,12 @@ class QqpProcessor(DataProcessor):
if i == 0: if i == 0:
continue continue
guid = "%s-%s" % (set_type, line[0]) guid = "%s-%s" % (set_type, line[0])
if set_type == "test":
text_a = line[1]
text_b = line[2]
label = "0"
else:
# There appear to be some garbage lines in the train dataset.
try: try:
text_a = line[3] text_a = line[3]
text_b = line[4] text_b = line[4]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment