GitLab

dot_prods() that can accumulate in addition to assign.

stride values.  This lets you run the tensor resizing routine on subwindows in
a tensor.

concat layer's backward() method.  It was assigning the gradient to previous
layers instead of adding the gradient, as required by the layer interface
specification.  This change also noticeably speeds up concat layers since only
one CUDA kernel launch now happens per concat operation, rather than one
kernel launch for each sample in a tensor.

rather than the entire tensor.

versions were calling into cuDNN, however, the cuDNN functions for doing this
are horrifically slow, well over 100x slower than they should be, which is
surprising since these functions are so trivial.

little.

layers.  Updated the solvers to support this.

skip layers and add_prev style layers.  In particular, now in-place layers only
overwrite the gradient information in their child layer if they are operating
in in-place mode.  Otherwise, they add their gradients to their child layers.

It should also be noted that it's safe for in-place layers to overwrite
gradients when in in-place mode since their child layers are inaccessible when
in-place layers operate in in-place mode.  This prevents any other layers from
trying to add to the child layer, thereby avoiding the potability of layer
interference.  So the bug this change fixes is that, when not in in-place mode
the child layers are still accessible but in-place layers were *still*
overwriting child gradients.

implementation of assign_conv_bias_gradient().

the number of threads and blocks rather than using the hard coded numbers I had
in there.  This makes some functions noticeably faster.

Also added a dot() function that is fully asynchronous.

tensors with different sizes and it will zero pad them as needed.

since that's a little different in cuDNN.  I also removed my CUDA code for
doing batch normalization and replaced it with cuDNN's new batch normalization
methods.

Finally, I forgot to add a convolutional option to the bn_ object.  Now it has
one so you can set the mode however you like, either BATCH_NORM_FC or
BATCH_NORM_CONV.

CUDA version of add_bias_gradient().