Skip to content

GitLab

"...git@developer.sourcefind.cn:renzhc/diffusers_dcu.git" did not exist on "77373c5eb15ac3d1474bf49bf2ed30ac6bfcb670"
Commit 4340b34e authored by Qian Wang's avatar Qian Wang Committed by Facebook Github Bot
Browse files

Stringlize 2-d tensors with Dictionary. 

Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/828

Differential Revision: D15960629

Pulled By: myleott

fbshipit-source-id: ca631651e9a90ce8ed90ca23987519001fea3656
parent 7b4f5517
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
fairseq/data/dictionary.py
View file @ 4340b34e
...@@ -54,7 +54,7 @@ class Dictionary(object): ...@@ -54,7 +54,7 @@ class Dictionary(object):
Can optionally remove BPE symbols or escape <unk> words. Can optionally remove BPE symbols or escape <unk> words.
""" """
if torch.is_tensor(tensor) and tensor.dim() == 2: if torch.is_tensor(tensor) and tensor.dim() == 2:
return '\n'.join(self.string(t) for t in tensor) return '\n'.join(self.string(t, bpe_symbol, escape_unk) for t in tensor)
def token_string(i): def token_string(i):
if i == self.unk(): if i == self.unk():
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment