# Linear Algebra

This page documents the core linear algebra tools included in dlib. In particular, the three most important objects in this part of the library are the matrix, vector, and rectangle. All the other tools on this page are functions for manipulating these three objects. A good example and introduction can be found in the matrix example program.

Most of the linear algebra tools deal with dense matrices. However, there is also a limited amount of support for working with sparse matrices and vectors. In particular, the dlib tools represent sparse vectors using the container objects in the C++ STL. For details, see the notes at the top of dlib/svm/sparse_vector_abstract.h.

Finally, note that all the dense matrix tools can be obtained by #including <dlib/matrix.h> while the sparse vector tools can be obtained by #including <dlib/sparse_vector.h>. The geometry tools can be used by #including <dlib/geometry.h>.

Dense Matrix Tools
matrix
Basic_Math_Operators
exp dlib/matrix/matrix_math_functions_abstract.h.html#exp log10 dlib/matrix/matrix_math_functions_abstract.h.html#log10 log dlib/matrix/matrix_math_functions_abstract.h.html#log sqrt dlib/matrix/matrix_math_functions_abstract.h.html#sqrt pow dlib/matrix/matrix_math_functions_abstract.h.html#pow squared dlib/matrix/matrix_math_functions_abstract.h.html#squared cubed dlib/matrix/matrix_math_functions_abstract.h.html#cubed sigmoid dlib/matrix/matrix_math_functions_abstract.h.html#sigmoid abs dlib/matrix/matrix_math_functions_abstract.h.html#abs reciprocal dlib/matrix/matrix_math_functions_abstract.h.html#reciprocal reciprocal_max dlib/matrix/matrix_math_functions_abstract.h.html#reciprocal_max normalize dlib/matrix/matrix_math_functions_abstract.h.html#normalize round dlib/matrix/matrix_math_functions_abstract.h.html#round ceil dlib/matrix/matrix_math_functions_abstract.h.html#ceil floor dlib/matrix/matrix_math_functions_abstract.h.html#floor round_zeros dlib/matrix/matrix_math_functions_abstract.h.html#round_zeros conj dlib/matrix/matrix_math_functions_abstract.h.html#conj norm dlib/matrix/matrix_math_functions_abstract.h.html#norm imag dlib/matrix/matrix_math_functions_abstract.h.html#imag real dlib/matrix/matrix_math_functions_abstract.h.html#real complex_matrix dlib/matrix/matrix_math_functions_abstract.h.html#complex_matrix sin dlib/matrix/matrix_math_functions_abstract.h.html#sin cos dlib/matrix/matrix_math_functions_abstract.h.html#cos tan dlib/matrix/matrix_math_functions_abstract.h.html#tan asin dlib/matrix/matrix_math_functions_abstract.h.html#asin acos dlib/matrix/matrix_math_functions_abstract.h.html#acos atan dlib/matrix/matrix_math_functions_abstract.h.html#atan sinh dlib/matrix/matrix_math_functions_abstract.h.html#sinh cosh dlib/matrix/matrix_math_functions_abstract.h.html#cosh tanh dlib/matrix/matrix_math_functions_abstract.h.html#tanh
Linear_Algebra
inv dlib/matrix/matrix_la_abstract.h.html#inv pinv dlib/matrix/matrix_la_abstract.h.html#pinv svd dlib/matrix/matrix_la_abstract.h.html#svd svd2 dlib/matrix/matrix_la_abstract.h.html#svd2 svd3 dlib/matrix/matrix_la_abstract.h.html#svd3 det dlib/matrix/matrix_la_abstract.h.html#det trace dlib/matrix/matrix_la_abstract.h.html#trace dot dlib/matrix/matrix_utilities_abstract.h.html#dot length dlib/matrix/matrix_utilities_abstract.h.html#length length_squared dlib/matrix/matrix_utilities_abstract.h.html#length_squared trans dlib/matrix/matrix_utilities_abstract.h.html#trans diag dlib/matrix/matrix_utilities_abstract.h.html#diag diagm dlib/matrix/matrix_utilities_abstract.h.html#diagm lowerm dlib/matrix/matrix_utilities_abstract.h.html#lowerm upperm dlib/matrix/matrix_utilities_abstract.h.html#upperm chol dlib/matrix/matrix_la_abstract.h.html#chol inv_lower_triangular dlib/matrix/matrix_la_abstract.h.html#inv_lower_triangular inv_upper_triangular dlib/matrix/matrix_la_abstract.h.html#inv_upper_triangular lu_decomposition dlib/matrix/matrix_la_abstract.h.html#lu_decomposition qr_decomposition dlib/matrix/matrix_la_abstract.h.html#qr_decomposition cholesky_decomposition dlib/matrix/matrix_la_abstract.h.html#cholesky_decomposition eigenvalue_decomposition dlib/matrix/matrix_la_abstract.h.html#eigenvalue_decomposition real_eigenvalues dlib/matrix/matrix_la_abstract.h.html#real_eigenvalues
Conversions
mat matrix_cast dlib/matrix/matrix_utilities_abstract.h.html#matrix_cast pixel_to_vector dlib/matrix/matrix_utilities_abstract.h.html#pixel_to_vector vector_to_pixel dlib/matrix/matrix_utilities_abstract.h.html#vector_to_pixel
Sub_Matrix_Expressions
range dlib/matrix/matrix_subexp_abstract.h.html#range subm dlib/matrix/matrix_subexp_abstract.h.html#subm rowm dlib/matrix/matrix_subexp_abstract.h.html#rowm colm dlib/matrix/matrix_subexp_abstract.h.html#colm set_subm dlib/matrix/matrix_subexp_abstract.h.html#set_subm set_colm dlib/matrix/matrix_subexp_abstract.h.html#set_colm set_rowm dlib/matrix/matrix_subexp_abstract.h.html#set_rowm
Statistics
sum dlib/matrix/matrix_utilities_abstract.h.html#sum sum_rows dlib/matrix/matrix_utilities_abstract.h.html#sum_rows sum_cols dlib/matrix/matrix_utilities_abstract.h.html#sum_cols prod dlib/matrix/matrix_utilities_abstract.h.html#prod mean dlib/matrix/matrix_utilities_abstract.h.html#mean max dlib/matrix/matrix_utilities_abstract.h.html#max min dlib/matrix/matrix_utilities_abstract.h.html#min find_min_and_max dlib/matrix/matrix_utilities_abstract.h.html#find_min_and_max index_of_min dlib/matrix/matrix_utilities_abstract.h.html#index_of_min index_of_max dlib/matrix/matrix_utilities_abstract.h.html#index_of_max variance dlib/matrix/matrix_utilities_abstract.h.html#variance stddev dlib/matrix/matrix_utilities_abstract.h.html#stddev covariance dlib/matrix/matrix_utilities_abstract.h.html#covariance randm dlib/matrix/matrix_utilities_abstract.h.html#randm gaussian_randm dlib/matrix/matrix_utilities_abstract.h.html#gaussian_randm
Other_Utilities
is_col_vector dlib/matrix/matrix_utilities_abstract.h.html#is_col_vector is_row_vector dlib/matrix/matrix_utilities_abstract.h.html#is_row_vector is_vector dlib/matrix/matrix_utilities_abstract.h.html#is_vector const_temp_matrix dlib/matrix/matrix_abstract.h.html#const_temp_matrix symmetric_matrix_cache dlib/matrix/symmetric_matrix_cache_abstract.h.html conv dlib/matrix/matrix_conv_abstract.h.html#conv conv_same dlib/matrix/matrix_conv_abstract.h.html#conv_same conv_valid dlib/matrix/matrix_conv_abstract.h.html#conv_valid xcorr dlib/matrix/matrix_conv_abstract.h.html#xcorr xcorr_same dlib/matrix/matrix_conv_abstract.h.html#xcorr_same xcorr_valid dlib/matrix/matrix_conv_abstract.h.html#xcorr_valid flip dlib/matrix/matrix_utilities_abstract.h.html#flip flipud dlib/matrix/matrix_utilities_abstract.h.html#flipud fliplr dlib/matrix/matrix_utilities_abstract.h.html#fliplr make_symmetric dlib/matrix/matrix_utilities_abstract.h.html#make_symmetric ones_matrix dlib/matrix/matrix_utilities_abstract.h.html#ones_matrix zeros_matrix dlib/matrix/matrix_utilities_abstract.h.html#zeros_matrix uniform_matrix dlib/matrix/matrix_utilities_abstract.h.html#uniform_matrix identity_matrix dlib/matrix/matrix_utilities_abstract.h.html#identity_matrix rotate dlib/matrix/matrix_utilities_abstract.h.html#rotate reshape_to_column_vector dlib/matrix/matrix_utilities_abstract.h.html#reshape_to_column_vector reshape dlib/matrix/matrix_utilities_abstract.h.html#reshape removerc dlib/matrix/matrix_utilities_abstract.h.html#removerc remove_row dlib/matrix/matrix_utilities_abstract.h.html#remove_row remove_col dlib/matrix/matrix_utilities_abstract.h.html#remove_col set_all_elements dlib/matrix/matrix_utilities_abstract.h.html#set_all_elements hash dlib/matrix/matrix_utilities_abstract.h.html#hash tmp dlib/matrix/matrix_utilities_abstract.h.html#tmp equal dlib/matrix/matrix_utilities_abstract.h.html#equal pointwise_multiply dlib/matrix/matrix_utilities_abstract.h.html#pointwise_multiply join_rows dlib/matrix/matrix_utilities_abstract.h.html#join_rows join_cols dlib/matrix/matrix_utilities_abstract.h.html#join_cols tensor_product dlib/matrix/matrix_utilities_abstract.h.html#tensor_product scale_columns dlib/matrix/matrix_utilities_abstract.h.html#scale_columns scale_rows dlib/matrix/matrix_utilities_abstract.h.html#scale_rows sort_columns dlib/matrix/matrix_utilities_abstract.h.html#sort_columns rsort_columns dlib/matrix/matrix_utilities_abstract.h.html#rsort_columns clamp dlib/matrix/matrix_utilities_abstract.h.html#clamp lowerbound dlib/matrix/matrix_utilities_abstract.h.html#lowerbound upperbound dlib/matrix/matrix_utilities_abstract.h.html#upperbound linspace dlib/matrix/matrix_utilities_abstract.h.html#linspace logspace dlib/matrix/matrix_utilities_abstract.h.html#logspace cartesian_product dlib/matrix/matrix_utilities_abstract.h.html#cartesian_product
2D/3D Geometry
border_enumerator
rectangle
vector
point
rotate_point
point_rotator
point_transform
point_transform_affine
find_affine_transform
rotation_matrix
get_rect
centered_rect
center
dcenter
shrink_rect
grow_rect
translate_rect
resize_rect
resize_rect_width
resize_rect_height
move_rect
nearest_point
distance_to_rect_edge
Sparse Vector Tools
sparse_to_dense
distance_squared
dlib/svm/sparse_vector_abstract.h.html#distance_squared
distance
dlib/svm/sparse_vector_abstract.h.html#distance
assign
dlib/svm/sparse_vector_abstract.h.html#assign
length_squared
dlib/svm/sparse_vector_abstract.h.html#length_squared
length
dlib/svm/sparse_vector_abstract.h.html#length
scale_by
dlib/svm/sparse_vector_abstract.h.html#scale_by
add
dlib/svm/sparse_vector_abstract.h.html#add
subtract
dlib/svm/sparse_vector_abstract.h.html#subtract
max_index_plus_one
dlib/svm/sparse_vector_abstract.h.html#max_index_plus_one
add_to
dlib/svm/sparse_vector_abstract.h.html#add_to
subtract_from
dlib/svm/sparse_vector_abstract.h.html#subtract_from
min
dlib/svm/sparse_vector_abstract.h.html#min
max
dlib/svm/sparse_vector_abstract.h.html#max
make_sparse_vector
dlib/svm/sparse_vector_abstract.h.html#make_sparse_vector
sparse_matrix_vector_multiply
dlib/svm/sparse_vector_abstract.h.html#sparse_matrix_vector_multiply
sparse_to_dense
dlib/sparse_vector.h
dlib/svm/sparse_vector_abstract.h
This is a set of simple functions that take sparse vectors and converts them into equivalent dense vectors.
mat
dlib/matrix.h
dlib/matrix/matrix_mat_abstract.h
This is a set of simple functions that take objects like std::vector or array2d and convert them into matrix objects. Note that the conversion is done using template expressions so there is no runtime cost associated with calling mat().
matrix
dlib/matrix.h
dlib/matrix/matrix_abstract.h
This is a 2D matrix object that enables you to write code that deals with matrices using a simple syntax similar to what can be written in MATLAB. It is implemented using the expression templates technique which allows it to eliminate the temporary matrix objects that would normally be returned from expressions such as M = A+B+C+D; Normally each invocation of the + operator would construct and return a temporary matrix object but using this technique we can avoid creating all these temporary objects and receive a large speed boost.

This object is also capable of using BLAS and LAPACK libraries such as ATLAS or the Intel MKL when available. To enable BLAS support all you have to do is #define DLIB_USE_BLAS and then make sure you link your application with your BLAS library. Similarly, to enable LAPACK support just #define DLIB_USE_LAPACK and link to your LAPACK library. Finally, the use of BLAS and LAPACK is transparent to the user, that is, the dlib matrix object uses BLAS and LAPACK internally to optimize various operations while still allowing the user to use a simple MATLAB like syntax.

Note that the cmake files that come with dlib know how to link a project with ATLAS or the Intel MKL if you are building on a linux system. The cmake files may also work in a few other cases as well but I haven't tested any others. But in any case, by no means are you required to use the dlib cmake files.

It is also worth noting that all the preconditions of every function related to the matrix object are checked by DLIB_ASSERT statements and thus can be enabled by #defining ENABLE_ASSERTS or DEBUG. Doing this will cause your program to run slower but should catch any usage errors.

Finally, you don't need to #include anything to get the matrix extensions listed below. They are included by the dlib/matrix.h file for you.

matrix_ex.cpp.html
matrix_expressions_ex.cpp.html
matrix_utilities
dlib/matrix/matrix_utilities_abstract.h
This extension contains miscellaneous utility functions for manipulating matrix objects.
matrix_la
dlib/matrix/matrix_la_abstract.h
This extension contains linear algebra functions to calculate QR, LU, Cholesky, eigenvalue, and singular value decompositions. It also contains a few other miscellaneous functions that solve systems of equations or calculate values derived from the above decompositions.
matrix_math_functions
dlib/matrix/matrix_math_functions_abstract.h
This extension contains mathematical functions that operate on each element of a matrix independently.
matrix_sub_expressions
dlib/matrix/matrix_subexp_abstract.h
This extension contains a number of functions for dealing with sub-matrices.
border_enumerator
dlib/geometry.h
dlib/geometry/border_enumerator_abstract.h
This object is an enumerator over the border points of a rectangle.
nearest_point
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and a point and returns the point in the given rectangle that is nearest to the given point.
distance_to_rect_edge
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and a point and returns the Manhattan distance between the rectangle's edge and the point.
move_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and moves it so that it's upper left corner occupies the given location.
resize_rect_height
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and returns a new rectangle with the given height but otherwise with the same edge points as the original rectangle.
resize_rect_width
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and returns a new rectangle with the given width but otherwise with the same edge points as the original rectangle.
resize_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and returns a new rectangle with the given size but with the same upper left corner as the original rectangle.
translate_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle and moves it by a given number of units along the x and y axis relative to where it was before the move.
center
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
Returns the center point of a rectangle.
dcenter
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
Returns the center point of a rectangle. This is a version of center() which returns a double version of the point rather than one which uses integers to represent the result. Therefore, it is slightly more accurate.
centered_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
There are various overloads of this function but the basic idea is that it returns a rectangle with a given width and height and centered about a given point.
shrink_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle object, shrinks its borders by a given amount, and returns the result.
grow_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This function takes a rectangle object, grows its borders by a given amount, and returns the result.
rotate_point
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is a function that rotates a 2D vector or point object about a given point.
point_rotator
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is an object that rotates a 2D vector or point object about the origin.
point_transform
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is an object that rotates a 2D vector or point object about the origin and then adds a displacement vector.
point_transform_affine
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is an object that applies an affine transformation to a vector or point. Note that you can use find_affine_transform to easily create affine transforms from sets of point correspondences.
find_affine_transform
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is a routine that takes in two sets of points and finds the best affine transformation that maps between them.
rotation_matrix
dlib/geometry.h
dlib/geometry/point_transforms_abstract.h
This is a method for creating 2D rotation matrices.
get_rect
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This is a simple template function that returns a rectangle representing the size of a 2D container (e.g. matrix or array2d).
rectangle
dlib/geometry.h
dlib/geometry/rectangle_abstract.h
This object represents a rectangular region inside a Cartesian coordinate system. It allows you to easily represent and manipulate rectangles.
vector
dlib/geometry.h
dlib/geometry/vector_abstract.h
This object represents a two or three dimensional vector.

If you want to work with general N-dimensional column vectors then you should the matrix object. In particular, you should usually use a matrix with this type: dlib::matrix<double,0,1>.

point
dlib/geometry.h
dlib/geometry/vector_abstract.h
This object represents a point inside a Cartesian coordinate system. Note that a point is simply a typedef for a vector that is 2D and uses longs to represent coordinate values.