Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
ml.xml 44.5 KB
Newer Older
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37 
<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>

<doc>
   <title>Machine Learning</title>

   <!-- ************************************************************************* -->

   <body>
      <br/><br/>

         <p>
            This page documents all the machine learning algorithms present in
            the library.  In particular, there are algorithms for performing 
            binary classification, regression, clustering, anomaly detection, 
            and feature ranking, as well as algorithms for doing more 
            specialized computations.
         </p>

         <p> 
            A good tutorial and introduction to the general concepts used by most of the
            objects in this part of the library can be found in the <a href="svm_ex.cpp.html">svm example</a> program.
         </p>

         <p>   
            The major design goal of this portion of the library is to provide a highly modular and
            simple architecture for dealing with kernel algorithms. Towards this end, dlib takes a generic
            programming approach using C++ templates. In particular, each algorithm is parameterized
            to allow a user to supply either one of the predefined dlib kernels (e.g. <a 
            href="#radial_basis_kernel">RBF</a> operating
            on <a href="containers.html#matrix">column vectors</a>), or a new user defined kernel. 
            Moreover, the implementations of the algorithms are totally separated from the data on 
            which they operate. This makes the dlib implementation generic enough to operate on 
            any kind of data, be it column vectors, images, or some other form of structured data. 
            All that is necessary is an appropriate kernel.
         </p>
Davis King's avatar
updated the docs  
Davis King committed
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54 

         <br/> 
         <h3>Paper Describing dlib Machine Learning</h3>
         <pre>
Davis E. King. <a href="http://www.jmlr.org/papers/volume10/king09a/king09a.pdf">Dlib-ml: A Machine Learning Toolkit</a>. 
   <i>Journal of Machine Learning Research</i> 10, pp. 1755-1758, 2009

@Article{dlib09,
  author = {Davis E. King},
  title = {Dlib-ml: A Machine Learning Toolkit},
  journal = {Journal of Machine Learning Research},
  year = {2009},
  volume = {10},
  pages = {1755-1758},
}
         </pre>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85 
   </body>

   <!-- ************************************************************************* -->

   <menu width="150">
    <top>

      <section>
         <name>Primary Algorithms</name>
         <item>mlp</item> 
         <item>krls</item>
         <item>kcentroid</item>
         <item>linearly_independent_subset_finder</item>
         <item>kkmeans</item>
         <item>svm_nu_trainer</item> 
         <item>rvm_trainer</item> 
         <item>rvm_regression_trainer</item> 
         <item>rbf_network_trainer</item> 
         <item>rank_features</item> 
         <item>svm_pegasos</item> 
      </section>

      <section>
         <name>Trainer Adapters</name>
         <item>train_probabilistic_decision_function</item> 
         <item>reduced_decision_function_trainer</item> 
         <item>reduced</item> 
         <item>reduced_decision_function_trainer2</item> 
         <item>reduced2</item> 
         <item>batch</item> 
         <item>verbose_batch</item>
Davis King's avatar
updated the docs  
Davis King committed
86
87 
         <item>batch_cached</item> 
         <item>verbose_batch_cached</item>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
88 
         <item>batch_trainer</item>
Davis King's avatar
updated the docs  
Davis King committed
89
90 
         <item>null_trainer_type</item> 
         <item>null_trainer</item>
Davis King's avatar
updated the docs  
Davis King committed
91
92
93 
         <item>roc_trainer_type</item> 
         <item>roc_c1_trainer</item> 
         <item>roc_c2_trainer</item>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
94
95
96
97
98
99
100
101
102 
      </section>

      <section>
         <name>Kernels</name>
         <item>radial_basis_kernel</item>
         <item>polynomial_kernel</item>
         <item>sigmoid_kernel</item>
         <item>linear_kernel</item>
         <item>offset_kernel</item>
Davis King's avatar
updated the docs  
Davis King committed
103
104
105
106
107
108 

         <item>sparse_radial_basis_kernel</item>
         <item>sparse_polynomial_kernel</item>
         <item>sparse_sigmoid_kernel</item>
         <item>sparse_linear_kernel</item>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
109
110
111
112
113
114
115
116
117
118
119
120
121 
      </section>

      <section>
         <name>Function Objects</name>
         <item>decision_function</item>
         <item>distance_function</item>
         <item>probabilistic_decision_function</item>
         <item>normalized_function</item>
      </section>

      <section>
         <name>Miscellaneous</name>
         <item>vector_normalizer</item>
Davis King's avatar
updated the docs  
Davis King committed
122 
         <item>vector_normalizer_pca</item>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
123
124
125
126
127
128 
         <item>randomize_samples</item> 
         <item>is_binary_classification_problem</item> 
         <item>test_binary_decision_function</item> 
         <item>cross_validate_trainer</item> 
         <item>cross_validate_trainer_threaded</item> 
         <item>pick_initial_centers</item>
Davis King's avatar
updated the docs  
Davis King committed
129
130 
         <item>find_gamma_with_big_centroid_gap</item> 
         <item>verbose_find_gamma_with_big_centroid_gap</item>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243 
      </section>

    </top>  
   </menu>

   <!-- ************************************************************************* -->
   <!-- ************************************************************************* -->
   <!-- ************************************************************************* -->

   <components>
   
   <!-- ************************************************************************* -->
      
      <component>
         <name>pick_initial_centers</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kkmeans_abstract.h</spec_file>
         <description>
            This is a function that you can use to seed data clustering algorithms
            like the <a href="#kkmeans">kkmeans</a> clustering method.  What it 
            does is pick reasonable starting points for clustering by basically
            trying to find a set of points that are all far away from each other.
         </description>
         <examples>
            <example>kkmeans_ex.cpp.html</example>
         </examples>
                                 
      </component>

   <!-- ************************************************************************* -->
      
      <component checked="true">
         <name>mlp</name>
         <file>dlib/mlp.h</file>
         <spec_file>dlib/mlp/mlp_kernel_abstract.h</spec_file>
         <description>
            <p>
                This object represents a multilayer layer perceptron network that is
                trained using the back propagation algorithm.  The training algorithm also
                incorporates the momentum method.  That is, each round of back propagation
                training also adds a fraction of the previous update.  This fraction
                is controlled by the momentum term set in the constructor.  
            </p>
            <p>
               It is worth noting that a MLP is, in general, very inferior to modern
               kernel algorithms such as the support vector machine.  So if you haven't
               tried any other techniques with your data you really should.  
            </p>
         </description>

         <examples>
            <example>mlp_ex.cpp.html</example>
         </examples>
         
         <implementations>
            <implementation>
               <name>mlp_kernel_1</name>
               <file>dlib/mlp/mlp_kernel_1.h</file>
               <description> 
                  This is implemented in the obvious way.
               </description> 
    
               <typedefs>
                  <typedef>
                     <name>kernel_1a</name>
                     <description>is a typedef for mlp_kernel_1</description>
                  </typedef>
               </typedefs>                
               
            </implementation> 
                     
         </implementations>
                        
      </component>
            
   <!-- ************************************************************************* -->
      
      <component>
         <name>krls</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/krls_abstract.h</spec_file>
         <description>
                This is an implementation of the kernel recursive least squares algorithm 
                described in the paper The Kernel Recursive Least Squares Algorithm by Yaakov Engel.
            <p>
                The long and short of this algorithm is that it is an online kernel based 
                regression algorithm.  You give it samples (x,y) and it learns the function
                f(x) == y.  For a detailed description of the algorithm read the above paper.
            </p>
         </description>

         <examples>
            <example>krls_ex.cpp.html</example>
            <example>krls_filter_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>svm_pegasos</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
                This object implements an online algorithm for training a support 
                vector machine for solving binary classification problems.  

            <p>
                The implementation of the Pegasos algorithm used by this object is based
                on the following excellent paper:
               <blockquote>
                    Pegasos: Primal estimated sub-gradient solver for SVM (2007)
Davis King's avatar
Fixed incorrect citation.  
Davis King committed
244 
                    by Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263 
                    In ICML 
               </blockquote>
            </p>
            <p>
                This SVM training algorithm has two interesting properties.  First, the 
                pegasos algorithm itself converges to the solution in an amount of time
                unrelated to the size of the training set (in addition to being quite fast
                to begin with).  This makes it an appropriate algorithm for learning from
                very large datasets.  Second, this object uses the <a href="#kcentroid">kcentroid</a> object 
                to maintain a sparse approximation of the learned decision function.  
                This means that the number of support vectors in the resulting decision 
                function is also unrelated to the size of the dataset (in normal SVM
                training algorithms, the number of support vectors grows approximately 
                linearly with the size of the training set).  
            </p>
         </description>

         <examples>
            <example>svm_pegasos_ex.cpp.html</example>
Davis King's avatar
updated the docs  
Davis King committed
264 
            <example>svm_sparse_ex.cpp.html</example>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304 
         </examples>
      </component>
      
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>kkmeans</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kkmeans_abstract.h</spec_file>
         <description>
                This is an implementation of a kernelized k-means clustering algorithm.  
                It performs k-means clustering by using the <a href="#kcentroid">kcentroid</a> object.  
         </description>

         <examples>
            <example>kkmeans_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>vector_normalizer</name>
         <file>dlib/statistics.h</file>
         <spec_file link="true">dlib/statistics/statistics_abstract.h</spec_file>
         <description>
                This object represents something that can learn to normalize a set 
                of column vectors.  In particular, normalized column vectors should 
                have zero mean and a variance of one.  
         </description>

         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
Davis King's avatar
updated the docs  
Davis King committed
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322 
   <!-- ************************************************************************* -->

      
      <component>
         <name>vector_normalizer_pca</name>
         <file>dlib/statistics.h</file>
         <spec_file link="true">dlib/statistics/statistics_abstract.h</spec_file>
         <description>
                This object represents something that can learn to normalize a set 
                of column vectors.  In particular, normalized column vectors should 
                have zero mean and a variance of one.  

                This object also uses principal component analysis for the purposes 
                of reducing the number of elements in a vector.  
         </description>

      </component>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364 
   <!-- ************************************************************************* -->

      <component>
         <name>linearly_independent_subset_finder</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/linearly_independent_subset_finder_abstract.h</spec_file>
         <description>
            <p>
                This is an implementation of an online algorithm for recursively finding a
                set of linearly independent vectors in a kernel induced feature space.  To 
                use it you decide how large you would like the set to be and then you feed it 
                sample points.  
            </p>
            <p>
                
                Each time you present it with a new sample point it either 
                keeps the current set of independent points unchanged, or if the new point 
                is "more linearly independent" than one of the points it already has,  
                it replaces the weakly linearly independent point with the new one.
            </p>

            <p>
                
                This object uses the Approximately Linearly Dependent metric described in the paper 
                The Kernel Recursive Least Squares Algorithm by Yaakov Engel to decide which
                points are more linearly independent than others.
            </p>
         </description>

      </component>
      
      
   <!-- ************************************************************************* -->
      
      
      <component>
         <name>kcentroid</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kcentroid_abstract.h</spec_file>
         <description>

                This object represents a weighted sum of sample points in a kernel induced
Davis King's avatar
Fixed spelling error  
Davis King committed
365 
                feature space.  It can be used to kernelize any algorithm that requires only
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648 
                the ability to perform vector addition, subtraction, scalar multiplication,
                and inner products.  

                  <p>
                An example use of this object is as an online algorithm for recursively estimating 
                the centroid of a sequence of training points.  This object then allows you to 
                compute the distance between the centroid and any test points.  So you can use 
                this object to predict how similar a test point is to the data this object has 
                been trained on (larger distances from the centroid indicate dissimilarity/anomalous 
                points).  
                  </p>

                  <p>
                The object internally keeps a set of "dictionary vectors" 
                that are used to represent the centroid.  It manages these vectors using the 
                sparsification technique described in the paper The Kernel Recursive Least 
                Squares Algorithm by Yaakov Engel.  This technique allows us to keep the 
                number of dictionary vectors down to a minimum.  In fact, the object has a 
                user selectable tolerance parameter that controls the trade off between 
                accuracy and number of stored dictionary vectors.
                  </p>

         </description>

         <examples>
            <example>kcentroid_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>train_probabilistic_decision_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
            <p>
               Trains a <a href="#probabilistic_decision_function">probabilistic_decision_function</a> using 
               some sort of batch trainer object such as the <a href="#svm_nu_trainer">svm_nu_trainer</a> or
               <a href="#rbf_network_trainer">rbf_network_trainer</a>.
            </p>
            The probability model is created by using the technique described in the paper:
            <blockquote>
                Probabilistic Outputs for Support Vector Machines and
                Comparisons to Regularized Likelihood Methods by 
                John C. Platt.  Match 26, 1999
            </blockquote>
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>rbf_network_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/rbf_network_abstract.h</spec_file>
         <description>
               Trains a radial basis function network and outputs a <a href="#decision_function">decision_function</a>. 
         </description>
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>rvm_regression_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/rvm_abstract.h</spec_file>
         <description>
            <p>
               Trains a relevance vector machine for solving regression problems.  
               Outputs a <a href="#decision_function">decision_function</a> that represents the learned 
               regression function. 
            </p>
               The implementation of the RVM training algorithm used by this library is based
               on the following paper:
               <blockquote>
                Tipping, M. E. and A. C. Faul (2003). Fast marginal likelihood maximisation 
                for sparse Bayesian models. In C. M. Bishop and B. J. Frey (Eds.), Proceedings 
                of the Ninth International Workshop on Artificial Intelligence and Statistics, 
                Key West, FL, Jan 3-6.
               </blockquote>
         </description>
         <examples>
            <example>rvm_regression_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      
      <component>
         <name>rvm_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/rvm_abstract.h</spec_file>
         <description>
            <p>
               Trains a relevance vector machine for solving binary classification problems.  
               Outputs a <a href="#decision_function">decision_function</a> that represents the learned classifier. 
            </p>
               The implementation of the RVM training algorithm used by this library is based
               on the following paper:
               <blockquote>
                Tipping, M. E. and A. C. Faul (2003). Fast marginal likelihood maximisation 
                for sparse Bayesian models. In C. M. Bishop and B. J. Frey (Eds.), Proceedings 
                of the Ninth International Workshop on Artificial Intelligence and Statistics, 
                Key West, FL, Jan 3-6.
               </blockquote>
         </description>
         <examples>
            <example>rvm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      
      <component>
         <name>svm_nu_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
            <p>
               Trains a nu support vector classifier and outputs a <a href="#decision_function">decision_function</a>. 
            </p>
               The implementation of the nu-svm training algorithm used by this library is based
               on the following excellent papers:
               <ul>
                  <li>Chang and Lin, Training {nu}-Support Vector Classifiers: Theory and Algorithms</li>
                  <li>Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector 
                     machines, 2001. Software available at 
                     <a href="http://www.csie.ntu.edu.tw/~cjlin/libsvm">http://www.csie.ntu.edu.tw/~cjlin/libsvm</a></li>
               </ul>
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>normalized_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/function_abstract.h</spec_file>
         <description>
                This object represents a container for another function
                object and an instance of the <a href="#vector_normalizer">vector_normalizer</a> object.  

                It automatically normalizes all inputs before passing them
                off to the contained function object.
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->


      <component>
         <name>probabilistic_decision_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/function_abstract.h</spec_file>
         <description>
                This object represents a binary decision function for use with
                support vector machines.  It returns an 
                estimate of the probability that a given sample is in the +1 class. 
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>distance_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/function_abstract.h</spec_file>
         <description>
                This object represents a point in kernel induced feature space. 
                You may use this object to find the distance from the point it 
                represents to points in input space.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>decision_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/function_abstract.h</spec_file>
         <description>
                This object represents a decision or regression function that was 
                learned by a kernel based learning algorithm.  
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>offset_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kernel_abstract.h</spec_file>
         <description>
                This object represents a kernel with a fixed value offset
                added to it.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>linear_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kernel_abstract.h</spec_file>
         <description>
                This object represents a linear function kernel for use with
                kernel learning machines.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>sigmoid_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kernel_abstract.h</spec_file>
         <description>
                This object represents a sigmoid kernel for use with
                kernel learning machines.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>polynomial_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kernel_abstract.h</spec_file>
         <description>
                This object represents a polynomial kernel for use with
                kernel learning machines.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>radial_basis_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/kernel_abstract.h</spec_file>
         <description>
                This object represents a radial basis function kernel for use with
                kernel learning machines.
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
Davis King's avatar
updated the docs  
Davis King committed
649
650
651
652
653
654
655
656
657
658
659 
   <!-- ************************************************************************* -->
      
      <component>
         <name>sparse_linear_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/sparse_kernel_abstract.h</spec_file>
         <description>
                This object represents a linear function kernel for use with
                kernel learning machines that operate on 
                <a href="dlib/svm/sparse_vector_abstract.h.html#sparse_vectors">sparse vectors</a>.
         </description>
Davis King's avatar
updated the docs  
Davis King committed
660
661
662 
         <examples>
            <example>svm_sparse_ex.cpp.html</example>
         </examples>
Davis King's avatar
updated the docs  
Davis King committed
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708 
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>sparse_sigmoid_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/sparse_kernel_abstract.h</spec_file>
         <description>
                This object represents a sigmoid kernel for use with
                kernel learning machines that operate on 
                <a href="dlib/svm/sparse_vector_abstract.h.html#sparse_vectors">sparse vectors</a>.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->

      <component>
         <name>sparse_polynomial_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/sparse_kernel_abstract.h</spec_file>
         <description>
                This object represents a polynomial kernel for use with
                kernel learning machines that operate 
                <a href="dlib/svm/sparse_vector_abstract.h.html#sparse_vectors">sparse vectors</a>.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>sparse_radial_basis_kernel</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/sparse_kernel_abstract.h</spec_file>
         <description>
                This object represents a radial basis function kernel for use with
                kernel learning machines that operate 
                <a href="dlib/svm/sparse_vector_abstract.h.html#sparse_vectors">sparse vectors</a>.
         </description>
                                 
      </component>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749 
   <!-- ************************************************************************* -->
      
      <component>
         <name>is_binary_classification_problem</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
             This function simply takes two vectors, the first containing feature vectors and
             the second containing labels, and reports back if the two could possibly 
             contain data for a well formed classification problem.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>randomize_samples</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
               Randomizes the order of samples in a column vector containing sample data.
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>rank_features</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/feature_ranking_abstract.h</spec_file>
         <description>
             Finds a ranking of the top N (a user supplied parameter) features in a set of data 
             from a two class classification problem.  It  
              does this by computing the distance between the centroids of both classes in kernel defined 
              feature space.  Good features are then ones that result in the biggest separation between
              the two centroids.
Davis King's avatar
updated the docs  
Davis King committed
750
751
752
753
754
755
756
757
758
759 
         </description>
         <examples>
            <example>rank_features_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
Davis King's avatar
updated the docs  
Davis King committed
760 
         <name>find_gamma_with_big_centroid_gap</name>
Davis King's avatar
updated the docs  
Davis King committed
761
762
763 
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/feature_ranking_abstract.h</spec_file>
         <description>
Davis King's avatar
updated the docs  
Davis King committed
764
765
766
767 
            This is a function that tries to pick a reasonable default value for the
            gamma parameter of the <a href="#radial_basis_kernel">radial_basis_kernel</a>.  It
            picks the parameter that gives the largest separation between the centroids, in 
            kernel feature space, of two classes of data.
Davis King's avatar
updated the docs  
Davis King committed
768
769
770
771
772
773
774
775
776
777 
         </description>
         <examples>
            <example>rank_features_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
Davis King's avatar
updated the docs  
Davis King committed
778 
         <name>verbose_find_gamma_with_big_centroid_gap</name>
Davis King's avatar
updated the docs  
Davis King committed
779
780
781 
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/feature_ranking_abstract.h</spec_file>
         <description>
Davis King's avatar
updated the docs  
Davis King committed
782
783 
            This is just a version of 
            <a href="#find_gamma_with_big_centroid_gap">find_gamma_with_big_centroid_gap</a> that is
Davis King's avatar
updated the docs  
Davis King committed
784 
            verbose in the sense that it prints out status messages as it proceeds.
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801 
         </description>
         <examples>
            <example>rank_features_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>batch</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#batch_trainer">batch_trainer</a> objects.
         </description>
Davis King's avatar
updated the docs  
Davis King committed
802
803
804 
         <examples>
            <example>svm_pegasos_ex.cpp.html</example>
         </examples>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819 
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>verbose_batch</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#batch_trainer">batch_trainer</a> objects.  This function
            generates a batch_trainer that will print status messages to standard
            output so that you can observe the progress of a training algorithm.
         </description>
Davis King's avatar
updated the docs  
Davis King committed
820
821
822 
         <examples>
            <example>svm_pegasos_ex.cpp.html</example>
         </examples>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
823
824
825 
                                 
      </component>
Davis King's avatar
updated the docs  
Davis King committed
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855 
   <!-- ************************************************************************* -->
      
      <component>
         <name>batch_cached</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#batch_trainer">batch_trainer</a> objects that are setup
            to use a kernel matrix cache.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>verbose_batch_cached</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#batch_trainer">batch_trainer</a> objects.  This function
            generates a batch_trainer that will print status messages to standard
            output so that you can observe the progress of a training algorithm.
            It will also be configured to use a kernel matrix cache.
         </description>
                                 
      </component>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875 
   <!-- ************************************************************************* -->
      
      <component>
         <name>batch_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/pegasos_abstract.h</spec_file>
         <description>
               This is a batch trainer object that is meant to wrap online trainer objects 
               that create <a href="#decision_function">decision_functions</a>.  It
               turns an online learning algorithm such as <a href="#svm_pegasos">svm_pegasos</a>
               into a batch learning object.  This allows you to use objects like
               svm_pegasos with functions (e.g. <a href="#cross_validate_trainer">cross_validate_trainer</a>)
               that expect batch mode training objects. 
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
Davis King's avatar
updated the docs  
Davis King committed
876 
         <name>null_trainer_type</name>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
877 
         <file>dlib/svm.h</file>
Davis King's avatar
updated the docs  
Davis King committed
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905 
         <spec_file link="true">dlib/svm/null_trainer_abstract.h</spec_file>
         <description>
                This object is a simple tool for turning a <a href="#decision_function">decision_function</a> 
                (or any object with an interface compatible with decision_function)
                into a trainer object that always returns the original decision
                function when you try to train with it.  

               <p>
                dlib contains a few "training post processing" algorithms (e.g. 
                <a href="#reduced">reduced</a> and <a href="#reduced2">reduced2</a>).  These tools 
                take in a trainer object,
                tell it to perform training, and then they take the output decision
                function and do some kind of post processing to it.  The null_trainer_type 
                object is useful because you can use it to run an already
                learned decision function through the training post processing
                algorithms by turning a decision function into a null_trainer_type
                and then giving it to a post processor.  
               </p>
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>null_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/null_trainer_abstract.h</spec_file>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
906
907 
         <description>
            This is a convenience function for creating
Davis King's avatar
updated the docs  
Davis King committed
908 
            <a href="#null_trainer_type">null_trainer_type</a>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
909
910
911
912
913 
            objects.
         </description>
                                 
      </component>
Davis King's avatar
updated the docs  
Davis King committed
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963 
   <!-- ************************************************************************* -->
      
      <component>
         <name>roc_c1_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/roc_trainer_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#roc_trainer_type">roc_trainer_type</a> objects that are
            setup to pick a point on the ROC curve with respect to the +1 class.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>roc_c2_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/roc_trainer_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#roc_trainer_type">roc_trainer_type</a> objects that are
            setup to pick a point on the ROC curve with respect to the -1 class.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>roc_trainer_type</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/roc_trainer_abstract.h</spec_file>
         <description>
                This object is a simple trainer post processor that allows you to 
                easily adjust the bias term in a trained decision_function object.
                That is, this object lets you pick a point on the ROC curve and 
                it will adjust the bias term appropriately.  

               <p>
                So for example, suppose you wanted to set the bias term so that
                the accuracy of your decision function on +1 labeled samples was 99%.
                To do this you would use an instance of this object declared as follows:
                <tt>roc_trainer_type&lt;trainer_type&gt;(your_trainer, 0.99, +1);</tt>
               </p>
         </description>
                                 
      </component>
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981 
   <!-- ************************************************************************* -->
      
      <component>
         <name>reduced_decision_function_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/reduced_abstract.h</spec_file>
         <description>
               This is a batch trainer object that is meant to wrap other batch trainer objects 
               that create <a href="#decision_function">decision_function</a> objects.
               It performs post processing on the output decision_function objects 
               with the intent of representing the decision_function with fewer 
               support vectors.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
Davis King's avatar
updated the docs  
Davis King committed
982
983
984
985
986
987
988
989
990
991
992
993
994
995 
      <component>
         <name>reduced</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/reduced_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#reduced_decision_function_trainer">reduced_decision_function_trainer</a>
            objects.
         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
Davis King's avatar
Reorganized the stuff on the algorithms page. I pulled the machine learning  
Davis King committed
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095 
      <component>
         <name>reduced2</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/reduced_abstract.h</spec_file>
         <description>
            This is a convenience function for creating 
            <a href="#reduced_decision_function_trainer2">reduced_decision_function_trainer2</a>
            objects.
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>reduced_decision_function_trainer2</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/reduced_abstract.h</spec_file>
         <description>
               <p>
               This is a batch trainer object that is meant to wrap other batch trainer objects 
               that create <a href="#decision_function">decision_function</a> objects.
               It performs post processing on the output decision_function objects 
               with the intent of representing the decision_function with fewer 
               support vectors.  
               </p>
               <p>
               It begins by performing the same post processing as
               the <a href="#reduced_decision_function_trainer">reduced_decision_function_trainer</a>
               object but it also performs a global gradient based optimization 
               to further improve the results.
               </p>
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>test_binary_decision_function</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
            Tests a <a href="#decision_function">decision_function</a> that represents a binary decision function and
            returns the test accuracy.  

         </description>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
      
      <component>
         <name>cross_validate_trainer_threaded</name>
         <file>dlib/svm_threaded.h</file>
         <spec_file link="true">dlib/svm/svm_threaded_abstract.h</spec_file>
         <description>
               Performs k-fold cross validation on a user supplied trainer object such
               as the <a href="#svm_nu_trainer">svm_nu_trainer</a> or <a href="#rbf_network_trainer">rbf_network_trainer</a>.  
               This function does the same thing as <a href="#cross_validate_trainer">cross_validate_trainer</a>
               except this function also allows you to specify how many threads of execution to use.
               So you can use this function to take advantage of a multi-core system to perform
               cross validation faster.
         </description>
      </component>
      
   <!-- ************************************************************************* -->
      
      <component>
         <name>cross_validate_trainer</name>
         <file>dlib/svm.h</file>
         <spec_file link="true">dlib/svm/svm_abstract.h</spec_file>
         <description>
               Performs k-fold cross validation on a user supplied trainer object such
               as the <a href="#svm_nu_trainer">svm_nu_trainer</a> or <a href="#rbf_network_trainer">rbf_network_trainer</a>.
         </description>
         <examples>
            <example>svm_ex.cpp.html</example>
         </examples>
                                 
      </component>
      
   <!-- ************************************************************************* -->
      
   </components>

   <!-- ************************************************************************* -->


</doc>