Add quotes to paths in MeCab arguments (#7142)

Without quotes directories with spaces in them will fail to be processed correctly.

Add quotes to paths in MeCab arguments (#7142)
Without quotes directories with spaces in them will fail to be processed correctly.
e29c3f1b · Paul O'Leary McCann · GitHub · cb061e78 · e29c3f1b
Unverified Commit e29c3f1b authored Sep 15, 2020 by Paul O'Leary McCann Committed by GitHub Sep 15, 2020
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

src/transformers/tokenization_bert_japanese.py src/transformers/tokenization_bert_japanese.py +1 -1

No files found.
--- a/src/transformers/tokenization_bert_japanese.py
+++ b/src/transformers/tokenization_bert_japanese.py
@@ -249,7 +249,7 @@ class MecabTokenizer:
                raise ValueError("Invalid mecab_dic is specified.")
            mecabrc = os.path.join(dic_dir, "mecabrc")
-            mecab_option = "-d {} -r {} ".format(dic_dir, mecabrc) + mecab_option
+            mecab_option = '-d "{}" -r "{}" '.format(dic_dir, mecabrc) + mecab_option
        self.mecab = fugashi.GenericTagger(mecab_option)