GitLab

* wip: add apis for clipped_relu and elu, and layer implementation for clipped_relu

* add tensor_tools documentation

* add cpu implementations for new activations

* add elu layer

* use upperbound and lowerbound for clipped_relu

* fix clipped_relu gradient due to wrong variable naming

* fix elu_gradient due to wrong variable naming

* fix elu_gradient documentation

* add documentation

* WIP: add test_layer cases for clipped_relu and elu

For some reason that I can't see, ELU is failing...

* add clipped_relu and elu tests... cuda elu layer still does not work

* fix spacing

* add custom cuda implementation for elu_gradient (this one works)

* Revert "add custom cuda implementation for elu_gradient (this one works)"

This reverts commit 446dd803964cc6ecca598ddf6688e5d89ca0b112.

* Revert "Revert "add custom cuda implementation for elu_gradient (this one works)""

This reverts commit 0b615f50081d0d90e71d502b6767fcb6ba62f28a.

* add comment about custom elu gradient implementation

* add gradient tests, restore cudnn elu gradient

* re add custom elu gradient implementation

* update docs

* use own cuda implementation for clipped_relu and elu
Co-authored-by: Davis E. King <davis@dlib.net>

* Fix input/output mappings with repeat layers

* add test for input/output tensor mappers

* fix output to input order

* Add letterbox image

* use && instead of and

* make function adhere to the generic image interface

* avoid extra copy

* add some overloads and a simple test

* add documentation

* use zero_border_pixels and remove superfluous temporary image

* allow different input and out images and update docs

* remove empty line

* be more explicit about output image size

* Add softmax function for matrix type

* make softmax inherit from basic_op_m

* fix comment

* add test for matrix softmax

* remove include

* take inspiration from op_normalize

* use multiplication instead of division

* fix typo in documentation

This reverts commit 8b9d0439.

Reverting this because the cmake pip pakage is still busted.  I've been
getting messages from many people about how it's breaking their
systems/installs.

This dependency was explicitly removed two years ago because pip was
installing a broken cmake on some systems.  I'm adding the dependency
back in the hope that the pip copy of cmake has been fixed on all
systems by this point.

Co-authored-by: pf <pf@pf-ubuntu-dev>

* [SERIALIZATION]	- vectorstream can now be used with vector<int8_t> and vector<uint8_t>

* [SERIALIZATION]	- update proxy_serialize and proxy_deserialize to work with vector<int8_t> and vector<uint8_t>

* [SERIALIZATION]	- updated vectorstream tests

* [SERIALIZATION]	- updated serialize tests. check you can go to and from any of vector<char>, vector<int8_t> and vector<uint8_t>

* [SERIALIZATION]	- updated matrix tests. check you can go to and from any of vector<char>, vector<int8_t> and vector<uint8_t>

* [SERIALIZATION]	- updated dnn tests. check you can go to and from any of vector<char>, vector<int8_t> and vector<uint8_t>

* [SERIALIZATION] improved and possibly safer

* [SERIALIZATION] use placement new. best of all worlds i think. we have least object overhead. but code looks a tad uglier. oh well, user doesn't have to care

* [SERIALIZATION] i hope this is easier on the eyes.
Co-authored-by: pf <pf@pf-ubuntu-dev>

* fix set_learning_rate_multipliers not working

* add tests for set_learning_rate_multipliers

Since we are dividing by 1024, the unit should be MiB instead of MB.
I also added a space between the number and the unit

[MATRIX] bug fix. If you #include <dlib/matrix.h> in a header file you might get a compiler error saying ambiguous call to max(). This commit fixes that. (#2287)
Co-authored-by: pf <pf@pf-ubuntu-dev>

* [FFT] added fft, ifft, fft_inplace and ifft_inplace overloads for std::vector

* [FFT] 	- static_assert T is a floating point type. There are static asserts in mkl_fft and kiss_fft, but it doesn't hurt adding them in the matrix API too so users get helpful warnings higher up in the API.

* [FFT] 	- added documentation for std::vector overloads in matrix_fft_abstract.h file
Co-authored-by: pf <pf@pf-ubuntu-dev>

I'm finding there are other issues with gcc 4.8 now and it's old enough that we can just drop support for gcc 4.8.

* Early termination for find_{min,max}_global

This patch adds a callback to allow the user to request cancellation of a
search using find_{min,max}_global.  This enables users to cancel
searches when they are no-longer relevent, or when the user has some
special knowledge of the solution that they can use the stop the search
early.

closes  #2250

* Moved default stopping condition into find_max_global.h since that's the code it relates to and did some minor cleanup.
Co-authored-by: Davis King <davis@dlib.net>

This also makes it so the num and max_runtime arguments can now appear
in any order.

This does include a minor backwards compatibility break.  Which is
someone passing in initial function evaluations by directly supplying an
initializer list like {function_evaluation({1.1, 0.9}, rosen({1.1, 0.9}))} may have
to do std::vector<function_evaluation>{function_evaluation({1.1, 0.9},
rosen({1.1, 0.9}))} instead or make it a variable.  This is due to C++
not supporting direct use of initializer lists with variadic templates in this
context.  But in any case, I doubt many users do this and it is not hard
for those that do to update as described above.

* fix find_min_global finding wrong answers

Previously, find_min_global would produce wrong output when passed
a collection of initial evaluations because the solver expected the
y-values to be multiplied by -1.  This fix does that when minimizing.

closes #2283

* fixed tabbing
Co-authored-by: Davis King <davis@dlib.net>

* Fix build with clang on ppc64le

Clang doesn't implement vec_rsqrt. Code taken from https://github.com/opencv/opencv/blob/master/modules/core/include/opencv2/core/vsx_utils.hpp#L360

* Define vec_rsqrt only if we use VSX

* [FFT] added kissfft wrappers, moved kiss and mkl wrappers into separate files, call the right functions in matrix_fft.h
Co-authored-by: pf <pf@pf-ubuntu-dev>
Co-authored-by: Davis King <davis@dlib.net>

* Remove GUI dependency from fonts

* use #ifndef

* Updated docs more
Co-authored-by: Davis King <davis@dlib.net>

* add function to draw string on images

* add documentation

* formatting

* add semicolon

* add test for draw_string on images

* Allow the face detector to be run concurrently.

* Use img_view instead of numpy_image
Co-authored-by: Akshay Modi <amodi@netflix.com>

Simple typo in the inception training

that cuda is available even though it knows it didn't find cublas, which
is part of the standard cuda install.  So we need to add a check to see
if cmake *really for realz* found cuda.

* Added exponential distribution

* tab problem removed?

* forgot std::

* Also added Weibull distribution. Very useful indeed.

* Simple Weibull distribution unit test

* don't forget std::

* sorry, typo

* [RAND] 	- seed the random number generators for consistency and no nasty surprises
		- added parameter for tolerance
		- added unit test for exponential distribution

* [RAND] print the spinner more often

* [RAND] up the tolerance for kurtosis a bit

* [RAND] refactored parameters to reflect documentation on wikipedia.

* [RAND] added documentation to _abstract

* [RAND] i switched the order of the arguments to get_random_weibull and didn't update the unit tests. oops
Co-authored-by: pf <pf@pf-ubuntu-dev>