# Image Processing

This page documents the functionality present in this library that deals with the management and manipulation of images. One thing to note is that there is no explicit image object. Instead, everything deals with array2d objects that contain various kinds of pixels.

## Pixel Types

Most image handling routines in dlib will accept images containing any pixel type. This is made possible by defining a traits class, pixel_traits, for each possible pixel type. This traits class enables image processing routines to determine how to handle each kind of pixel and therefore only pixels which have a pixel_traits definition may be used. The following list defines all the pixel types which come with pixel_traits definitions.

- RGB
  There are two RGB pixel types in dlib, rgb_pixel and bgr_pixel. Each defines a 24bit RGB pixel type. The bgr_pixel is identical to rgb_pixel except that it lays the color channels down in memory in BGR order rather than RGB order and is therefore useful for interfacing with other image processing tools which expect this format (e.g. OpenCV).
- RGB Alpha
  The rgb_alpha_pixel is an 8bit per channel RGB pixel with an 8bit alpha channel.
- HSI
  The hsi_pixel is a 24bit pixel which represents a point in the Hue Saturation Intensity (HSI) color space.
- Grayscale
  Any built in scalar type may be used as a grayscale pixel type. For example, unsigned char, int, double, etc.

Pixels
rgb_pixel
bgr_pixel
rgb_alpha_pixel
hsi_pixel
pixel_traits
get_pixel_intensity
assign_pixel
assign_pixel_intensity
Image I/O
jpeg_loader
load_bmp
load_dng
load_image
load_jpeg
load_png
png_loader
save_bmp
save_dng
save_png
Keypoints and Descriptors
integral_image
hessian_pyramid
compute_dominant_angle
compute_surf_descriptor
hog_image
haar_x
haar_y
get_interest_points
get_surf_points
interest_point
surf_point
Edges and Thresholds
edge_orientation
hysteresis_threshold
sobel_edge_detector
suppress_non_maximum_edges
threshold_image
auto_threshold_image
Miscellaneous
cv_image
draw_line
assign_image
assign_image_scaled
assign_all_pixels
assign_border_pixels
equalize_histogram
get_histogram
spatially_filter_image
pyramid_down
zero_border_pixels
binary_dilation
binary_erosion
binary_open
binary_close
binary_intersection
binary_union
binary_difference
binary_complement
integral_image
dlib/image_transforms.h
dlib/image_transforms/integral_image_abstract.h
This object is an alternate way of representing image data that allows for very fast computations of sums of pixels in rectangular regions. To use this object you load it with a normal image and then you can use the get_sum_of_area() member function to compute sums of pixels in a given area in constant time.
haar_x
dlib/image_transforms.h
dlib/image_transforms/integral_image_abstract.h
This is a function that operates on an integral_image and allows you to compute the response of a Haar wavelet oriented along the X axis.
haar_y
dlib/image_transforms.h
dlib/image_transforms/integral_image_abstract.h
This is a function that operates on an integral_image and allows you to compute the response of a Haar wavelet oriented along the Y axis.
compute_dominant_angle
dlib/image_keypoint.h
dlib/image_keypoint/surf_abstract.h
Computes and returns the dominant angle (i.e. the angle of the dominant gradient) at a given point and scale in an image. This function is part of the main processing of the SURF algorithm.
get_surf_points
dlib/image_keypoint.h
dlib/image_keypoint/surf_abstract.h
This function runs the complete SURF algorithm on an input image and returns the points it found. For a description of what exactly the SURF algorithm does you should read the following paper:

> SURF: Speeded Up Robust Features By Herbert Bay, Tinne Tuytelaars, and Luc Van Gool

Also note that there are numerous flavors of the SURF algorithm you can put together using the functions in dlib. The get_surf_points() function is just an example of one way you might do so.

surf_ex.cpp.html
compute_surf_descriptor
dlib/image_keypoint.h
dlib/image_keypoint/surf_abstract.h
Computes the 64 dimensional SURF descriptor vector of a box centered at a given center point, tilted at a given angle, and sized according to a given scale.
hog_image
dlib/image_keypoint.h
dlib/image_keypoint/hog_abstract.h
This object is a tool for performing the image feature extraction algorithm described in the following paper:

> Histograms of Oriented Gradients for Human Detection by Navneet Dalal and Bill Triggs

hessian_pyramid
dlib/image_keypoint.h
dlib/image_keypoint/hessian_pyramid_abstract.h
This object represents an image pyramid where each level in the pyramid holds determinants of Hessian matrices for the original input image. This object can be used to find stable interest points in an image.

This object is an implementation of the fast Hessian pyramid as described in the paper:

> SURF: Speeded Up Robust Features By Herbert Bay, Tinne Tuytelaars, and Luc Van Gool

This implementation was also influenced by the very well documented OpenSURF library and its corresponding description of how the fast Hessian algorithm functions:

> Notes on the OpenSURF Library by Christopher Evans

get_interest_points
dlib/image_keypoint.h
dlib/image_keypoint/hessian_pyramid_abstract.h
This function extracts interest points from a hessian_pyramid.
interest_point
dlib/image_keypoint.h
dlib/image_keypoint/hessian_pyramid_abstract.h
This is a simple struct used to represent the interest points returned by the get_interest_points function.
surf_point
dlib/image_keypoint.h
dlib/image_keypoint/surf_abstract.h
This is a simple struct used to represent the SURF points returned by the get_surf_points function.
surf_ex.cpp.html
pixel_traits
dlib/pixel.h
dlib/pixel.h
As the name implies, this is a traits class for pixel types. It allows you to determine what sort of pixel type you are dealing with.
hsi_pixel
dlib/pixel.h
dlib/pixel.h
This is a simple struct that represents an HSI colored graphical pixel.
rgb_alpha_pixel
dlib/pixel.h
dlib/pixel.h
This is a simple struct that represents an RGB colored graphical pixel with an alpha channel.
rgb_pixel
dlib/pixel.h
dlib/pixel.h
This is a simple struct that represents an RGB colored graphical pixel.
bgr_pixel
dlib/pixel.h
dlib/pixel.h
This is a simple struct that represents a BGR colored graphical pixel.

The difference between this object and the rgb_pixel is just that this struct lays its pixels down in memory in BGR order rather than RGB order. You only care about this if you are doing something like using the cv_image object to map an OpenCV image into a more object oriented form.

cv_image
dlib/opencv.h
dlib/opencv/cv_image_abstract.h
This object is meant to be used as a simple wrapper around the OpenCV IplImage struct. Using this class template you can turn an IplImage object into something that looks like a normal dlib style image object.

So you should be able to use cv_image objects with many of the image processing functions in dlib as well as the GUI tools for displaying images on the screen.

assign_pixel
dlib/pixel.h
dlib/pixel.h
assign_pixel() is a templated function that can assign any pixel type to another pixel type. It will perform whatever conversion is necessary to make the assignment work. (E.g. color to grayscale conversion)
assign_pixel_intensity
dlib/pixel.h
dlib/pixel.h
assign_pixel_intensity() is a templated function that can change the intensity of a pixel. So if the pixel in question is a grayscale pixel then it simply assigns that pixel the given value. However, if the pixel is not a grayscale pixel then it converts the pixel to the HSI color space and sets the I channel to the given intensity and then converts this HSI value back to the original pixel's color space.
get_pixel_intensity
dlib/pixel.h
dlib/pixel.h
get_pixel_intensity() is a templated function that returns the grayscale intensity of a pixel. If the pixel isn't a grayscale pixel then it converts the pixel to grayscale and returns that value.
png_loader
dlib/image_io.h
dlib/image_loader/png_loader_abstract.h
This object loads a Portable Network Graphics (PNG) image file into an array2d of pixels.

Note that you must define DLIB_PNG_SUPPORT if you want to use this object. You must also set your build environment to link to the libpng library. However, if you use CMake and dlib's default CMakeLists.txt file then it will get setup automatically.

jpeg_loader
dlib/image_io.h
dlib/image_loader/jpeg_loader_abstract.h
This object loads a JPEG image file into an array2d of pixels.

Note that you must define DLIB_JPEG_SUPPORT if you want to use this object. You must also set your build environment to link to the libjpeg library. However, if you use CMake and dlib's default CMakeLists.txt file then it will get setup automatically.

load_jpeg
dlib/image_io.h
dlib/image_loader/jpeg_loader_abstract.h
This function loads a JPEG image file into an array2d of pixels.

Note that you must define DLIB_JPEG_SUPPORT if you want to use this object. You must also set your build environment to link to the libjpeg library. However, if you use CMake and dlib's default CMakeLists.txt file then it will get setup automatically.

load_dng
dlib/image_io.h
dlib/image_loader/image_loader_abstract.h
This global function loads a dlib DNG file (a lossless compressed image format) into an array2d of pixels.
save_dng
dlib/image_io.h
dlib/image_saver/image_saver_abstract.h
This global function saves an image as a dlib DNG file (a lossless compressed image format).

This routine can save images containing any type of pixel. However, the DNG format can natively store only the following pixel types: rgb_pixel, hsi_pixel, rgb_alpha_pixel, uint8, and uint16. All other pixel types will be converted into one of these types as appropriate before being saved to disk.

save_png
dlib/image_io.h
dlib/image_saver/save_png_abstract.h
This global function writes an image to disk as a PNG (Portable Network Graphics) file.

Note that you must define DLIB_PNG_SUPPORT if you want to use this function. You must also set your build environment to link to the libpng library. However, if you use CMake and dlib's default CMakeLists.txt file then it will get setup automatically.

This routine can save images containing any type of pixel. However, save_png() can only natively store the following pixel types: rgb_pixel, rgb_alpha_pixel, uint8, and uint16. All other pixel types will be converted into one of these types as appropriate before being saved to disk.

load_image
dlib/image_io.h
dlib/image_loader/load_image_abstract.h
This global function takes a file name, looks at its extension, and then loads it into an array2d of pixels using the appropriate image loading routine. The supported types are BMP, PNG, JPEG, and the dlib DNG file format.

Note that you can only load PNG and JPEG files if you link against libpng and libjpeg respectively.

image_ex.cpp.html
load_bmp
dlib/image_io.h
dlib/image_loader/image_loader_abstract.h
This global function loads a MS Windows BMP file into an array2d of pixels.
load_png
dlib/image_io.h
dlib/image_loader/png_loader_abstract.h
This function loads a Portable Network Graphics (PNG) image file into an array2d of pixels.

Note that you must define DLIB_PNG_SUPPORT if you want to use this object. You must also set your build environment to link to the libpng library. However, if you use CMake and dlib's default CMakeLists.txt file then it will get setup automatically.

save_bmp
dlib/image_io.h
dlib/image_saver/image_saver_abstract.h
This global function saves an image as a MS Windows BMP file.

This routine can save images containing any type of pixel. However, it will convert all color pixels into rgb_pixel and grayscale pixels into uint8 type before saving to disk.

draw_line
dlib/image_transforms.h
dlib/image_transforms/draw_abstract.h
This global function draws a line onto an image.
assign_border_pixels
dlib/image_transforms.h
dlib/image_transforms/assign_image_abstract.h
This global function assigns all the pixels in the border of an image to a specific value.
assign_all_pixels
dlib/image_transforms.h
dlib/image_transforms/assign_image_abstract.h
This global function assigns all the pixels in an image a specific value.
assign_image
dlib/image_transforms.h
dlib/image_transforms/assign_image_abstract.h
This global function copies one image into another and performs any necessary color space conversions to make it work right.
assign_image_scaled
dlib/image_transforms.h
dlib/image_transforms/assign_image_abstract.h
This global function copies one image into another and performs any necessary color space conversions to make it work right. Additionally, if the dynamic range of the source image is too big to fit into the destination image then it will attempt to perform the appropriate scaling.
get_histogram
dlib/image_transforms.h
dlib/image_transforms/equalize_histogram_abstract.h
This global function computes an image's histogram and returns it in the form of a column or row matrix object.
edge_orientation
dlib/image_transforms.h
dlib/image_transforms/edge_detector_abstract.h
This global function takes horizontal and vertical gradient magnitude values and returns the orientation of the gradient.
equalize_histogram
dlib/image_transforms.h
dlib/image_transforms/equalize_histogram_abstract.h
This global function performs histogram equalization on an image.
hysteresis_threshold
dlib/image_transforms.h
dlib/image_transforms/thresholding_abstract.h
This global function performs hysteresis thresholding on an image.
sobel_edge_detector
dlib/image_transforms.h
dlib/image_transforms/edge_detector_abstract.h
This global function performs spatial filtering on an image using the sobel edge detection filters.
image_ex.cpp.html
suppress_non_maximum_edges
dlib/image_transforms.h
dlib/image_transforms/edge_detector_abstract.h
This global function performs non-maximum suppression on a gradient image.
image_ex.cpp.html
zero_border_pixels
dlib/image_transforms.h
dlib/image_transforms/assign_image_abstract.h
This global function zeros the pixels on the border of an image.
spatially_filter_image
dlib/image_transforms.h
dlib/image_transforms/spatial_filtering_abstract.h
This global function performs spatial filtering on an image with a user supplied filter.
pyramid_down
dlib/image_transforms.h
dlib/image_transforms/image_pyramid_abstract.h
This is a simple function object to help create image pyramids.
auto_threshold_image
dlib/image_transforms.h
dlib/image_transforms/thresholding_abstract.h
This global function performs a simple binary thresholding on an image. Instead of taking a user supplied threshold it computes one from the image using k-means clustering.
threshold_image
dlib/image_transforms.h
dlib/image_transforms/thresholding_abstract.h
This global function performs a simple binary thresholding on an image with a user supplied threshold.
binary_dilation
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function performs the morphological operation of dilation on an image.
binary_erosion
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function performs the morphological operation of erosion on an image.
binary_open
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function performs a morphological opening on an image.
binary_close
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function performs a morphological closing on an image.
binary_intersection
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function computes the intersection of two binary images.
binary_union
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function computes the union of two binary images.
binary_difference
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function computes the difference of two binary images.
binary_complement
dlib/image_transforms.h
dlib/image_transforms/morphological_operations_abstract.h
This global function computes the complement of a binary image.