Added a bunch of THREAD SAFETY blocks to the specs for various machine learning

function objects.  In general, I noted that in most cases they are safe to use
from multiple threads so long as they are not modified.

dlib/graph_cuts/graph_labeler_abstract.h

@@ -40,6 +40,15 @@ namespace dlib
                 to compute these score functions and then call find_max_factor_graph_potts().  
                 Additionally, this object uses linear functions to represent these score 
                 functions.    
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads.  This is because the const members are purely read-only
+                    operations.  However, any operation that modifies a graph_labeler is
+                    not threadsafe.
         !*/
 
     public:

dlib/svm/assignment_function_abstract.h

@@ -143,6 +143,15 @@ namespace dlib
                     match_score(l,r) == dot(w, PSI(l,r))
                 where l is an element of LHS, r is an element of RHS, w is a parameter 
                 vector, and PSI() is defined by the feature_extractor template argument.  
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads so long as the feature_extractor is also threadsafe.  This is
+                    because the const members are purely read-only operations.  However,
+                    any operation that modifies an assignment_function is not threadsafe.
         !*/
 
     public:

dlib/svm/function_abstract.h

@@ -30,6 +30,15 @@ namespace dlib
                 This object represents a classification or regression function that was 
                 learned by a kernel based learning algorithm.   Therefore, it is a function 
                 object that takes a sample object and returns a scalar value.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call operator() on this object from multiple threads so
+                    long as the kernel, K, is also threadsafe.  This is because operator()
+                    is a read-only operation.  However, any operation that modifies a
+                    decision_function is not threadsafe.
         !*/
 
         typedef K kernel_type;
@@ -131,6 +140,15 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS 
                 This object represents a binary decision function that returns an 
                 estimate of the probability that a given sample is in the +1 class.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call operator() on this object from multiple threads so
+                    long as decision_funct is also threadsafe.  This is because operator()
+                    is a read-only operation.  However, any operation that modifies a
+                    probabilistic_function is not threadsafe.
         !*/
 
         typedef typename function_type::scalar_type scalar_type;
@@ -233,6 +251,15 @@ namespace dlib
                 changed from being a function type to a kernel type.  Therefore, this
                 type is just a convenient version of probabilistic_function
                 for the case where the decision function is a dlib::decision_function<K>.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call operator() on this object from multiple threads so
+                    long as the kernel, K, is also threadsafe.  This is because operator()
+                    is a read-only operation.  However, any operation that modifies a
+                    probabilistic_decision_function is not threadsafe.
         !*/
 
         typedef K kernel_type;
@@ -348,6 +375,15 @@ namespace dlib
 
                 I.e.  It represents a linear combination of the basis vectors where 
                 the weights of the linear combination are stored in the alpha vector.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads so long as the kernel, K, is also threadsafe.  This is because
+                    the const members are purely read-only operations.  However, any
+                    operation that modifies a distance_function is not threadsafe.
         !*/
 
     public:
@@ -723,6 +759,14 @@ namespace dlib
                 This object represents a function that takes a data sample and projects
                 it into kernel feature space.  The result is a real valued column vector that 
                 represents a point in a kernel feature space.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    Instances of this object have a mutable cache which is used by const
+                    member functions.  Therefore, it is not safe to use one instance of
+                    this object from multiple threads (unless protected by a mutex).
         !*/
 
         typedef K kernel_type;
@@ -841,6 +885,15 @@ namespace dlib
                 if you have N classes then there will be N binary classifiers inside 
                 this object.  Additionally, this object is linear in the sense that
                 each of these binary classifiers is a simple linear plane.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const member functions of this object from
+                    multiple threads.  This is because the const members are purely
+                    read-only operations.  However, any operation that modifies a
+                    multiclass_linear_decision_function is not threadsafe.
         !*/
 
         typedef result_type_ result_type;

dlib/svm/kernel_abstract.h

@@ -37,6 +37,14 @@ namespace dlib
                 - an overloaded == operator that tells you if two kernels are
                   identical or not.
 
+        THREAD SAFETY
+            For a kernel function to be threadsafe it means that it must be safe to
+            evaluate an expression like val = kernel_function(sample1,sample2)
+            simultaneously from multiple threads, even when the threads operate on the same
+            object instances (i.e. kernel_function, sample1, and sample2).  The most common
+            way to make this safe is to ensure that the kernel function does not mutate any
+            data, either in itself or in its arguments.  
+
         For examples of kernel functions see the following objects
         (e.g. the radial_basis_kernel).
     !*/
@@ -52,6 +60,9 @@ namespace dlib
 
             WHAT THIS OBJECT REPRESENTS
                 This object represents a radial basis function kernel
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::type scalar_type;
@@ -153,6 +164,9 @@ namespace dlib
 
             WHAT THIS OBJECT REPRESENTS
                 This object represents a sigmoid kernel
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::type scalar_type;
@@ -260,6 +274,9 @@ namespace dlib
 
             WHAT THIS OBJECT REPRESENTS
                 This object represents a polynomial kernel
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::type scalar_type;
@@ -372,6 +389,9 @@ namespace dlib
 
             WHAT THIS OBJECT REPRESENTS
                 This object represents a linear function kernel
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::type scalar_type;
@@ -435,6 +455,9 @@ namespace dlib
 
             WHAT THIS OBJECT REPRESENTS
                 This object represents a histogram intersection kernel kernel
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::type scalar_type;
@@ -501,6 +524,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a kernel with a fixed value offset
                 added to it.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::scalar_type scalar_type;
@@ -609,6 +635,15 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This is a function object that computes the derivative of a kernel 
                 function object.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    Instances of this object are allowed to have a mutable cache which is
+                    used by const member functions.  Therefore, it is not safe to use one
+                    instance of this object from multiple threads (unless protected by a
+                    mutex).
         !*/
 
         typedef typename kernel_type::scalar_type scalar_type;

dlib/svm/one_vs_all_decision_function_abstract.h

@@ -49,6 +49,16 @@ namespace dlib
                 within a one_vs_all_decision_function must be listed in the
                 template arguments if serialization and deserialization is to
                 be used.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads so long as all the decision functions contained in this object
+                    are also threadsafe.  This is because the const members are purely
+                    read-only operations.  However, any operation that modifies a
+                    one_vs_all_decision_function is not threadsafe.
         !*/
     public:
 

dlib/svm/one_vs_one_decision_function_abstract.h

@@ -50,6 +50,16 @@ namespace dlib
                 within a one_vs_one_decision_function must be listed in the
                 template arguments if serialization and deserialization is to
                 be used.
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads so long as all the decision functions contained in this object
+                    are also threadsafe.  This is because the const members are purely
+                    read-only operations.  However, any operation that modifies a
+                    one_vs_one_decision_function is not threadsafe.
         !*/
     public:
 

dlib/svm/sequence_labeler_abstract.h

@@ -258,6 +258,15 @@ namespace dlib
                     y == argmax_Y dot(get_weights(), PSI(x,Y))
                     Where PSI() is defined by the feature_extractor template
                     argument.  
+
+            THREAD SAFETY
+                It is always safe to use distinct instances of this object in different
+                threads.  However, when a single instance is shared between threads then
+                the following rules apply:
+                    It is safe to call the const members of this object from multiple
+                    threads so long as the feature_extractor is also threadsafe.  This is
+                    because the const members are purely read-only operations.  However,
+                    any operation that modifies a sequence_labeler is not threadsafe.
         !*/
 
     public:

dlib/svm/sparse_kernel_abstract.h

@@ -27,6 +27,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a radial basis function kernel
                 that works with sparse vectors.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::value_type::second_type scalar_type;
@@ -128,6 +131,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a sigmoid kernel
                 that works with sparse vectors.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::value_type::second_type scalar_type;
@@ -235,6 +241,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a polynomial kernel
                 that works with sparse vectors.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::value_type::second_type scalar_type;
@@ -347,6 +356,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a linear function kernel
                 that works with sparse vectors.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::value_type::second_type scalar_type;
@@ -410,6 +422,9 @@ namespace dlib
             WHAT THIS OBJECT REPRESENTS
                 This object represents a histogram intersection kernel 
                 that works with sparse vectors.
+
+            THREAD SAFETY
+                This kernel is threadsafe.  
         !*/
 
         typedef typename T::value_type::second_type scalar_type;