GitLab

* Add new loss layer for binary loss per pixel

cuda_data_ptr<T>.  Also moved some memcpy() functions to namespace scope
so that calling them like dlib::cuda::memcpy() can referene them.  It
was slightly annoting before.

* Adding Mish activation function

* Bug fixed

* Added test for Mish

* Removed unwanted comments

* Simplified calculation and removed comments

* Kernel added and gradient computation simplified

* Gradient simplified

* Corrected gradient calculations

* Compute output when input greater than 8

* Minor correction

* Remove unnecessary pgrad for Mish

* Removed CUDNN calls

* Add standalone CUDA implementation of the Mish activation function

* Fix in-place gradient in the CUDA version; refactor a little

* Swap delta and omega

* Need to have src (=x) (and not dest) available for Mish

* Add test case that makes sure that cuda::mish and cpu::mish return the same results

* Minor tweaking to keep the previous behaviour
Co-authored-by: Juha Reunanen <juha.reunanen@tomaattinen.com>

It's a cheap check, and easy for someone to forget about otherwise.

the user provided objective function.  If it's faster than the LIPO
upper bounding Monte Carlo model then we skip or limit the Monte Carlo
stuff and just fun the objective function more often.  Previously,
find_max_global() simply assumed the objective functions were really
expensive to invoke.

TLDR: this change makes find_max_global() run a lot faster on objective
functions that are themselves very fast to execute, since it will skip
the expensive Monte Carlo modeling and just call the objective function
a bunch instead.