vggblock support without pooling and pooling_kernel_size missing self (#839)

Summary:
1) VggBlock was not supported if pooling kernel size was None.
2) Since we modify pooling kernel size by using _pair. We should use self.pooling_kernel_size. But I agree it doesn't matter as pytorch is robust to this.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/839

Differential Revision: D16934112

Pulled By: okhonko

fbshipit-source-id: b6b95163b0e7f7203d76d535f01a41912382bdc3

fairseq/modules/vggblock.py

@@ -103,7 +103,8 @@ class VGGBlock(torch.nn.Module):
                 input_dim = per_channel_dim
             self.layers.append(nn.ReLU())
 
-        pool_op = nn.MaxPool2d(kernel_size=pooling_kernel_size, ceil_mode=True)
+        if self.pooling_kernel_size is not None:
+            pool_op = nn.MaxPool2d(kernel_size=self.pooling_kernel_size, ceil_mode=True)
             self.layers.append(pool_op)
             self.total_output_dim, self.output_dim = infer_conv_output_dim(
                 pool_op, input_dim, out_channels