Commit 02cb5a43 authored by freewym's avatar freewym Committed by Facebook Github Bot
Browse files

remove shlex.quote in scripts/spm_train.py (#972)

Summary:
to resolve the issue https://github.com/pytorch/fairseq/issues/971
Pull Request resolved: https://github.com/pytorch/fairseq/pull/972

Differential Revision: D16892827

Pulled By: myleott

fbshipit-source-id: baf277961f1e292f4593eefe31e3541aa9d0d8c4
parent 0c75c760
......@@ -7,11 +7,10 @@
from __future__ import absolute_import, division, print_function, unicode_literals
import shlex
import sys
import sentencepiece as spm
if __name__ == "__main__":
spm.SentencePieceTrainer.Train(" ".join(map(shlex.quote, sys.argv[1:])))
spm.SentencePieceTrainer.Train(" ".join(sys.argv[1:]))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment