• Adrià Arrufat's avatar
    Add dnn self supervised learning example (#2434) · 2e8bac19
    Adrià Arrufat authored
    
    
    * wip: loss goes down when training without a dnn_trainer
    
    if I use a dnn_trainer, it segfaults (also with bigger batch sizes...)
    
    * remove commented code
    
    * fix gradient computation (hopefully)
    
    * fix loss computation
    
    * fix crash in input_rgb_image_pair::to_tensor
    
    * fix alias tensor offset
    
    * refactor loss and input layers and complete the example
    
    * add more data augmentation
    
    * add documentation
    
    * add documentation
    
    * small fix in the gradient computation and reuse terms
    
    * fix warning in comment
    
    * use tensor_tools instead of matrix to compute the gradients
    
    * complete the example program
    
    * add support for mult-gpu
    
    * Update dlib/dnn/input_abstract.h
    
    * Update dlib/dnn/input_abstract.h
    
    * Update dlib/dnn/loss_abstract.h
    
    * Update examples/dnn_self_supervised_learning_ex.cpp
    
    * Update examples/dnn_self_supervised_learning_ex.cpp
    
    * Update examples/dnn_self_supervised_learning_ex.cpp
    
    * Update examples/dnn_self_supervised_learning_ex.cpp
    
    * [TYPE_SAFE_UNION] upgrade (#2443)
    
    * [TYPE_SAFE_UNION] upgrade
    
    * MSVC doesn't like keyword not
    
    * MSVC doesn't like keyword and
    
    * added tests for emplate(), copy semantics, move semantics, swap, overloaded and apply_to_contents with non void return types
    
    * - didn't need is_void anymore
    - added result_of_t
    - didn't really need ostream_helper or istream_helper
    - split apply_to_contents into apply_to_contents (return void) and visit (return anything so long as visitor is publicly accessible)
    
    * - updated abstract file
    
    * - added get_type_t
    - removed deserialize_helper dupplicate
    - don't use std::decay_t, that's c++14
    
    * - removed white spaces
    - don't need a return-statement when calling apply_to_contents_impl()
    - use unchecked_get() whenever possible to minimise explicit use of pointer casting. lets keep that to a minimum
    
    * - added type_safe_union_size
    - added type_safe_union_size_v if C++14 is available
    - added tests for above
    
    * - test type_safe_union_size_v
    
    * testing nested unions with visitors.
    
    * re-added comment
    
    * added index() in abstract file
    
    * - refactored reset() to clear()
    - added comment about clear() in abstract file
    - in deserialize(), only reset the object if necessary
    
    * - removed unecessary comment about exceptions
    - removed unecessary // -------------
    - struct is_valid is not mentioned in abstract. Instead rather requiring T to be a valid type, it is ensured!
    - get_type and get_type_t are private. Client code shouldn't need this.
    - shuffled some functions around
    - type_safe_union_size and type_safe_union_size_v are removed. not needed
    - reset() -> clear()
    - bug fix in deserialize() index counts from 1, not 0
    - improved the abstract file
    
    * refactored index() to get_current_type_id() as per suggestion
    
    * maybe slightly improved docs
    
    * - HURRAY, don't need std::result_of or std::invoke_result for visit() to work. Just privately define your own type trait, in this case called return_type and return_type_t. it works!
    - apply_to_contents() now always calls visit()
    
    * example with private visitor using friendship with non-void return types.
    
    * Fix up contracts
    
    It can't be a post condition that T is a valid type, since the choice of T is up to the caller, it's not something these functions decide.  Making it a precondition.
    
    * Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
    
    * Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
    
    * Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
    
    * - added more tests for copy constructors/assignments, move constructors/assignments, and converting constructors/assignments
    - helper_copy -> helper_forward
    - added validate_type<T> in a couple of places
    
    * - helper_move only takes non-const lvalue references. So we are not using std::move with universal references !
    - use enable_if<is_valid<T>> in favor of validate_type<T>()
    
    * - use enable_if<is_valid<T>> in favor of validate_type<T>()
    
    * - added is_valid_check<>. This wraps enable_if<is_valid<T>,bool> and makes use of SFINAE more robust
    Co-authored-by: default avatarpfeatherstone <peter@me>
    Co-authored-by: default avatarpf <pf@me>
    Co-authored-by: default avatarDavis E. King <davis685@gmail.com>
    
    * Just minor cleanup of docs and renamed some stuff, tweaked formatting.
    
    * fix spelling error
    
    * fix most vexing parse error
    Co-authored-by: default avatarDavis E. King <davis@dlib.net>
    Co-authored-by: default avatarpfeatherstone <45853521+pfeatherstone@users.noreply.github.com>
    Co-authored-by: default avatarpfeatherstone <peter@me>
    Co-authored-by: default avatarpf <pf@me>
    Co-authored-by: default avatarDavis E. King <davis685@gmail.com>
    2e8bac19
dnn_self_supervised_learning_ex.cpp 15.8 KB