GitLab

network (which would needlessly double VRAM usage).  I also added a
set_synchronization_file() method so you can tell it to automatically
synchronize itself to disk every so often during training.  This makes resuming
an interrupted training session trivially easy.

the loss isn't being reduced.  Also, there is a stopping condition now based on
how large the current learning rate is.  That is, training stops when the learning
rate gets small enough and it is clear that no progress is being made.

explicitly gave it a solver.

the number of threads and blocks rather than using the hard coded numbers I had
in there.  This makes some functions noticeably faster.

Also added a dot() function that is fully asynchronous.

networks.  This revolved mostly around removing really deep template recursions
since that upsets the compiler when you make really deep networks.

method to more efficiently give the input gradient in some instances.

network is allocated on the heap rather than resulting in really large
stack usage for large networks.

tensors with different sizes and it will zero pad them as needed.

define combination layers made out of other combination layers without being
hassled by the compiler.

in-place/out-of-place logic present.

to the range [0,1].

symmetry between the non-inplace version. This also enables additional
optimizations in the resulting network.

since that's a little different in cuDNN.  I also removed my CUDA code for
doing batch normalization and replaced it with cuDNN's new batch normalization
methods.

Finally, I forgot to add a convolutional option to the bn_ object.  Now it has
one so you can set the mode however you like, either BATCH_NORM_FC or
BATCH_NORM_CONV.