<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>

<doc>
    <title>Suggested Books</title>



    <!-- ************************************************************************* -->

    <body>

        <br/><br/>
        <p>
         One of the major goals of dlib is to have documentation that enables
         someone to easily make use of its various components.  Ideally,
         you would read a short description of something, understand it immediately,
         and begin using it in your application without any difficulty.  Obviously, this
         depends partly on the background of the user.  For example, if you have 
         never written C++ software before then it probably isn't going to be this easy.
        </p>
        <p>
          This page is meant to complement the main library documentation by providing 
          references to books, along with my commentary, which explain most of 
          the background material needed to understand the various parts of the library.   
          In most cases these are the books I learned from during the process
          of creating dlib.  As always, if you disagree with anything or think I have left out 
          an important text then shoot me an <a href="mailto:davis@dlib.net">email</a>. 
        </p>
        <br/><br/>
        

      <h2>General Programming</h2>
      <ul>
        <h3>C++</h3>
         <ul>
            <li> <i>Programming: Principles and Practice Using C++</i> by Bjarne Stroustrup 
               <ul>  This is the sort of book you would use in a freshman introduction-to-programming class.  
                  So if you are just beginning to study programming and are interested in C++ then I think
               it is probably safe to say this is one of the best books you could read.   It is both
               recent (2009) and written by the creator of C++. </ul> <br/>
            </li>
            <li> <i>Effective C++: 55 Specific Ways to Improve Your Programs and Designs</i> (3rd Edition) by Scott Meyers  
               <ul>  This is a great intermediate level C++ book.  Most people have heard the jokes about 
                  how easy it is to shoot yourself in the foot with C++.  This book explains many things you
                  need to know about the language to avoid doing so on a regular basis.  So if you are
                  writing C++ software then this is a must-read.  I would even claim that
                  you are a danger to the C++ software you touch unless you know what is in this book.   
                  I'm not kidding.  Finally, the book isn't just about the quirks of C++. It also discusses many general 
                 software engineering ideas which have wide applicability.  So in this
                  respect it is a great book for any software developer to read.  
               </ul><br/>
            </li>
            <li> <i>More Effective C++: 35 New Ways to Improve Your Programs and Designs</i> by Scott Meyers  
               <ul> Consider this an expansion to Effective C++.  If you are going to read the above
                  book then you would almost certainly benefit from reading this one as well.
               </ul><br/>
            </li>
            <li> <i>The C++ Standard Library: A Tutorial and Reference</i> by Nicolai M. Josuttis   
               <ul> If you are going to buy a reference book on the C++ standard library then this 
                  is the one to get.  I think you
                  will find it is better than any of the available online references.    So if you find 
                  yourself frustrated with the online resources, then this is the book for you.
               </ul><br/>
            </li>
            <li> <a href="http://stdcxx.apache.org/doc/stdlibref/index.html">Apache C++ Standard Library Reference</a>
               <ul> What I said aside, this is a good online reference.   I often find myself referring to it
                  when I do not have the Josuttis book on hand.
               </ul><br/>
            </li>
         </ul>


         <h3>Multithreading</h3>
         <ul>
            <li> <i>Programming with POSIX Threads</i> by  David R. Butenhof  
               <ul> When I was an undergrad, this book was my main resource for learning about multithreading.  
                  It was enjoyable to read, as are all the books on this list, and covered everything
                  in great depth without becoming overbearing.  Also, despite what the title may suggest,
                  this book is useful for understanding multithreading broadly, not just multithreading
                  on POSIX systems.
               </ul><br/>
            </li>
         </ul>

        <h3>Network Programming</h3>
         <ul>
            <li> <i>Unix Network Programming, Volume 1: The Sockets Networking API</i> (3rd Edition) 
                by  W. Richard Stevens
               <ul> A lot of people call this book the network programming Bible and  
                  this praise is well deserved.  If you want a deep understanding of how computer networks
                  function, including the Internet, then this is the book to read.  As with 
                   the Butenhof book above, this is an excellent choice even for people who do not
                   intend to write software for Unix systems.   
               </ul><br/>
            </li>
         </ul>

        <h3>WIN32 Programming</h3>
              It has been a long time since I needed to refer to these two books.  However,
              they contained information I couldn't find elsewhere no matter
              how hard I looked.  So I recommend them in case you need to create or understand
              some low level win32 code. 
              <br/>
              <br/>
         <ul>  
            <li>  <i>Win32 Programming</i> by Brent E. Rector and Joseph M. Newcomer </li>
            <li> <i>Programming Windows</i> by  Charles Petzold </li>
            <li> <a href="http://msdn.microsoft.com/en-us/library/default.aspx">MSDN Library</a>  
               <ul>  This is Microsoft's online reference documentation.  It is very large and sometimes
                  confusing.  But at the end of the day you should be able to find the documentation
                  for just about every function in the entire Windows API.  
               </ul><br/>
            </li>
         </ul>
      </ul>





        <h2>Computer Science: Algorithms and Data Structures</h2>
         <ul>
            <li>  <i>Introduction to Algorithms</i> by  Cormen, Leiserson, Rivest and Stein
               <ul> You should get this book if you are looking for a good discussion of the classic computer science
                  algorithms and data structures (e.g. most of the components on the <a href="containers.html">containers</a>
                  page).  
               </ul><br/>
            </li>
            <li>  <i>Algorithms in C++, Parts 1-4: Fundamentals, Data Structure, Sorting, Searching</i> 
                  (3rd Edition) by Robert Sedgewick   
               <ul>  This is another good algorithms book.  If you are going to get only one book on this
                  subject then get the one above.  However, when I was learning about these topics I 
                  used both these books and on many occasions I found it helpful to read the description
                  of an algorithm or data structure in both.  Where one description was a little vague or 
                  confusing the other generally filled in the gaps.
               </ul><br/>
            </li>
         </ul>
        


        <h2>Lossless Data Compression</h2>
         <ul>
            <li> <i>Text Compression</i> by Bell, Cleary, and Witten 
               <ul> When I was studying data compression this was my most useful
                  resource.  If you are looking to understand how lossless data compression
                  algorithms work then this is the book you want.  It is completely self-contained 
                  and an absolute joy to read.  Note that contrary to one of the reviews on
                  amazon.com, the book <i>Managing Gigabytes</i> is not the second edition of this book;
                  if this topic interests you then be sure you get the 318 page
                  book published in 1990.
               </ul><br/>
            </li>
         </ul>



        <h2>General Math</h2>

         <ul>
            <li> <i>Linear Algebra Done Right</i> by Sheldon Jay Axler  
               <ul> If a matrix seems like an arbitrary grid of numbers or you find that 
                  you are confused by vectors, matrices, and the various things
                  that get done with them then this book will change your whole view of this subject.  
                  It doesn't teach you any algorithms.  Instead, it will give you a general 
                  framework in which to think about all this stuff.  Once you have that down
                  everything else will start to make a lot more sense.  If all goes well
                  you will even start to agree with the following: linear algebra is beautiful. :) 
               </ul><br/>
            </li>
            <li> <i>Numerical Linear Algebra</i> by Trefethen and Bau 
               <ul> While <i>Linear Algebra Done Right</i> is fairly abstract, this book by
                  Trefethen and Bau will 
                  explain some of the actual algorithms that are often used.
                  This is a great second book if you find that you want to know more about 
                  the SVD, LU decomposition, or various other algorithms involving linear algebra.   
               </ul><br/>
            </li>
            <li> <i>Calculus: Single and Multivariable</i> by Hughes-Hallett, Gleason, and McCallum 
               <ul>
                   Some of the books below will require and understanding of basic calculus.  So 
                   I'm recommending this book.  It was the book I used as an undergrad and I 
                   remember it being alright.  That isn't exactly a glowing review so if you
                   are really considering buying a calculus book you may want to check out
                   other reviews before picking this one. 
               </ul><br/>
            </li>
            <li> <i>Introduction to Real Analysis</i> (third edition) by Bartle and Sherbert  
               <ul> At some level real analysis is like a really rigorous repeat of calculus.  
                  So if you already have an undergraduate education in calculus and 
                  you are reading things that seem reminiscent of calculus but involve
                  stuff you haven't seen before (e.g. sup, inf, "sets of numbers", sequences of points) 
                  then you may be in need of a real analysis book.  This one is quite good and should
                  be accessible to someone with the usual undergraduate computer science math background.
               </ul><br/>
            </li>
         </ul>





        <h2>Optimization</h2>

        The subject of linear algebra is fundamental to optimization.  So you must be familiar
        with the contents of a book like <i>Linear Algebra Done Right</i> if you are going to study
        this area.  You will also need to know how to find the derivative of a function and
        understand what a derivative is all about.  So you will need to know a little bit of
        calculus.   Finally, once in a while you will need to know a little bit about real
        analysis.   Ultimately, what you need all depends on how deep you want to go.  

         <ul>
            <li> <i>Practical Methods of Optimization</i> (second edition) by R. Fletcher  1987 
               <ul> I love this book.  When I got it I literally spent my weekends sitting around
                  reading it for hours.  It is a fascinating and well written introduction to 
                  the subject of optimization.  This has been my most valuable resource for
                  learning the fundamentals of optimization and I cannot recommend it highly enough.    
               </ul><br/>
            </li>
            <li> <i>Numerical Optimization</i>  by Jorge Nocedal and Stephen Wright 2006  
               <ul>  This is a more recent text on optimization that is also very good.  It
                  covers many algorithms not covered by the above book.   
               </ul><br/>
            </li>
            <li> <i>Introduction to Derivative-Free Optimization</i>  by Conn, Scheinberg, and Vicente  
            <ul>  If you want to understand algorithms like <a href="optimization.html#find_min_bobyqa">BOBYQA</a>
            then this is a good recent book on the subject.  Note that a book like <i>Practical Methods of Optimization</i>
                 is almost certainly a prerequisite for reading this book.  As an aside, BOBYQA is not discussed in this book but 
                 its predecessor, NEWUOA is.  
               </ul><br/>
            </li>
         </ul>




        <h2>Machine Learning</h2>
         
         <ul>
            <li> <i>Artificial Intelligence: A Modern Approach </i> (3rd Edition) by Stuart Russell and Peter Norvig
               <ul> This book is about the much broader field of AI but it contains an excellent introduction
                  to machine learning and it also covers other useful topics like <a href="bayes.html">bayesian networks</a>.  
                  Moreover, it is very well written and self-contained.  So you don't need any particular 
                  background to be able to learn from it apart from a typical undergraduate background
                  in computer science. 
               </ul><br/>
            </li>
            <li> <i>Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond </i> 
               by Bernhard Schlkopf and Alexander J. Smola
               <ul> Most of the machine learning tools in dlib are implementations of various kernel methods.
                    So if you want a book that covers this topic in great depth as well as breadth then this is 
                    probably the book for you.  The most important prerequisite for this book is linear
                    algebra.  Virtually everything in this book depends on linear algebra in a fundamental way.  
                    <p>
                    The second important subject is optimization.  Whenever you see the text
                    mention the KKT conditions, duality, "primal variables", or quadratic programming it
                    is talking about ideas from optimization.  A good book which will explain all this to you
                    is <i>Practical Methods of Optimization</i>.  Note that this book calls the KKT conditions
                    just the "KT" conditions.  It is talking about the same thing.  Also, duality 
                    is something that comes up a lot in optimization but in the context of machine learning
                    usually people are talking about a particular form known as the Wolfe Dual.  
                    </p>
                    It would also be good (but maybe not critical depending on which parts you want to read) to 
                    be familiar with real analysis.  
               </ul><br/>
            </li>
            <li> <i>Kernel Methods for Pattern Analysis </i> by John Shawe-Taylor and Nello Cristianini 
               <ul> This is another good book about kernel methods.  If you have to choose between 
                  this book and <i>Learning with Kernels</i> I would go with <i>Learning with Kernels</i>.  However, it is
                  good to have both since reading different presentations of difficult subjects 
                  usually makes learning them easier.  
               </ul><br/>
            </li>

            <li> <i>Structured Prediction and Learning in Computer Vision</i> by Sebastian Nowozin and Christoph H. Lampert 2011
               <ul> If you are looking for a book discussing the background material necessary
                  for understanding things like the <a href="ml.html#structural_svm_problem">Structural SVM</a>
                  tools in dlib then this is a good book.  It is also available online 
                  in <a href="http://www.nowozin.net/sebastian/papers/nowozin2011structured-tutorial.pdf">PDF form</a>.
               </ul><br/>
            </li>

         </ul>

        <h2>Image Processing</h2>
         <ul>
            <li> <i>Digital Image Processing</i> by Rafael C. Gonzalez and Richard E. Woods  
               <ul> This is a terrific introduction to digital image processing.
                  By and large this book doesn't require any special prerequisites.  Sometimes
                  calculus shows up, but not too much.
               </ul><br/>
            </li>
         </ul>
    

    </body>



    <!-- ************************************************************************* -->

</doc>

