GitLab

* add helper methods to implement fused convolutions

* fix grammar

* add method to disable affine layer and updated serialization

* add documentation for .disable()

* add fuse_convolutions visitor and documentation

* update docs: net is not constant

* fix xml formatting and use std::boolalpha

* fix warning and updated net requirement for visitor

* fix segfault in fuse_convolutions visitor

* copy unconditionally

* make the visitor class a friend of the con_ class

* setup the biases alias tensor after enabling bias

* simplify visitor a bit

* fix comment

* setup the biases size, somehow this got lost

* copy the parameters before resizing

* remove enable_bias() method, since the visitor is now a friend

* Revert "remove enable_bias() method, since the visitor is now a friend"

This reverts commit 35b92b16316f19a7f1f1b1313c9ab874f4d6199b.

* update the visitor to remove the friend requirement

* improve behavior of enable_bias

* better describe the behavior of enable_bias

* wip: use cudnncudnnConvolutionBiasActivationForward when activation has bias

* wip: fix cpu compilation

* WIP: not working fused ReLU

* WIP: forgot do disable ReLU in visitor (does not change the fact that it does not work)

* WIP: more general set of 4d tensor (still not working)

* fused convolutions seem to be working now, more testing needed

* move visitor to the bottom of the file

* fix CPU-side and code clean up

* Do not try to fuse the activation layers

Fusing the activation layers in one cuDNN call is only supported when using
the cuDNN ones (ReLU, Sigmoid, TanH...) which might lead to suprising
behavior. So, let's just fuse the batch norm and the convolution into one
cuDNN call using the IDENTITY activation function.

* Set the correct forward algorithm for the identity activation

Ref: https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnConvolutionBiasActivationForward



* move the affine alias template to its original position

* wip

* remove unused param in relu and simplify example (I will delete it before merge)

* simplify conv bias logic and fix deserialization issue

* fix enabling bias on convolutions

* remove test example

* fix typo

* update documentation

* update documentation

* remove ccache leftovers from CMakeLists.txt

* Re-add new line

* fix enable/disable bias on unallocated networks

* update comment to mention cudnnConvolutionBiasActivationForward

* fix typo
Co-authored-by: Davis E. King <davis@dlib.net>

* Apply documentation suggestions from code review
Co-authored-by: Davis E. King <davis@dlib.net>

* update affine docs to talk in terms of gamma and beta

* simplify tensor_conv interface

* fix tensor_conv operator() with biases

* add fuse_layers test

* add an example on how to use the fuse_layers function

* fix typo
Co-authored-by: Davis E. King <davis@dlib.net>

* Don't try to use labels in unsupervised losses

I hope that is the right way of fixing this...

* fix it by duplicating most code in send_job (works on my machine)

I will probably need to find a way to reuse the code

* try to fix it reusing the code... not sure though

* Revert "try to fix it reusing the code... not sure though"

This reverts commit f308cac6df712da3619fb05b14f3345f0ec07b9a.

* check the type of the training label to fix the issue instead

* Missing include for `dlib::loss_multiclass_log_per_pixel_::label_to_ignore`

I was trying to compile the examples and encountered this issue after moving `rgb_label_image_to_index_label_image` to cpp file. Headers should include all symbols they mention.

* Update pascal_voc_2012.h

Should use the official entrypoint for including dnn stuff.
Co-authored-by: Davis E. King <davis685@gmail.com>

* add more cuda capabilities

* Allow setting custom cuda capabilities

* improve default behavior

* rename to compute capabilities

* add progress information (current/total and percent)

* print a new line instead of overwritting with spaces

* check if target_val is an integer with std::trunc

* add function to compute string dimensions in pixels

* use custom struct as a return value, remove first and last params

* Update dlib/image_transforms/draw_abstract.h
Co-authored-by: Davis E. King <davis@dlib.net>

* Fix cannot compile iso only code (#579)

also fixing (#1742)

* Remove GUI dependency from fonts (#2273)

* Fix disable_duplicative_biases when the input is a skip layer

* fix template parameters

* try to make sure travis uses C++17

* fix unbound variable

* Update dlib/travis/build-and-test.sh
Co-authored-by: Davis E. King <davis@dlib.net>

* Fix serialize variant with C++17

* fix order of parameters

* added support for std::optional if using C++

* oops, bug fix + check if item already holds a type

* oops, another bug fix

* remove warnings about unused parameters
Co-authored-by: pf <pf@me>

* [SERIALIZATION] addes support for std::variant

* [SERIALIZATION] bug fix + added tests

* support immutable types

* put an immutable type in std::variant
Co-authored-by: pf <pf@me>

Do this by just making all layers have a .input_layer() method, which in
that context can be implemented in a simple manner.

Also fix this error from cmake 3.5.1:

```
CMake Error at CMakeLists.txt:62 (if):
  if given arguments:

    "CMAKE_COMPILER_IS_GNUCXX" "AND" "CMAKE_CXX_COMPILER_VERSION" "VERSION_LESS_EQUAL" "4.8.5"

  Unknown arguments specified
```