Summary about Face Recognition with OpenCV

Some ideas and approachments about Face Recognition

geometric feature

• marker points ( position of eyes, ears, noses, ...) are used to build the feature vector (the distance, the angle, ...)

Eigenfaces

• The idea is that a facial image is a point from a high-dim image space, and a high-dim dataset is often described by correlated variables and therefore only a few meaningful dim account for most of the information .

• alg

  • PCA alg finds the lower-dim subspace with maximum variance (with the mean) in data. ( that is, to turn a set of possibly correlated variables into a smaller set of uncorrelated variables) .

  • There is a TRICK here in computing the eigenvectors: a M x N matrix (M rows, N columns) with M > N can only have N - 1 non-zero eigenvalues, so we can take the eigenvalue decomposition S of size N x N instead .

  • The approach maximizes the total scatter, but if the variance is generated by an external source like light, components with a maximum variance over all classes aren't nessarily useful for classification.

• The API in OpenCV:

  • Ptr createEigenFaceRecognizer(int num_components, double threshold) (~\sources\modules\contrib\src\facerec.cpp)

    • Perform the PCA

    • (~\modules\core\src\matmul.cpp) pca(data, Mat(), CV_PCA_DATA_AS_ROW, _num_components)

      The parameter num_components means that how many principal components to retain (it is not larger than the number of training samples) .

    • Get the feature vectors (y here is the principal component ) Mat y = subspaceProject(_eigenvectors, _mean, data.row(sampleIdx)

      The parameter threshold here gives a upper distance limit in the prediction (It is the parameter minDist in the Eigenfaces::predict())

      Eigenfaces::predict(InputArray _src, int &minClass, double &minDist)

      Each test sample is projected into PCA subspace,and get the related label based on the NORM_L2 distance in the trained subspace.

Fisherfaces

Local Binary Patterns Histograms (LBPH)

• idea

  • The focus is only on extracting local features of an object, thus the features in this waywill have a low-dim implicitly.

  • Also, the local description has to be a bit robust against image illumination variations (things like scale,translation or rotation), so the Local Binary Patterns (LBP) is given to summarize the local structure in a image by comparing each pixel with its neighborhood .

• The API in OpenCV Ptr createLBPHFaceRecognizer(int radius,int neighbors, int grid_x, int grid_y, double threshold) Calculate lbp image

  • elbp(src[sampleIdx],_radius,_neighbors)

    • The parameters radius and neighbors are used in the local binary pattern creation

  • Get spatial histogram from this lbp image

      Mat p = spatial_histogram(
                lbp_image, /* lbp_image */
                /* number of possible patterns */
                static_cast<int>(std::pow(2.0, static_cast<double>(_neighbors))),
                _grid_x, /* grid size x */
                _grid_y, /* grid size y */
                true)

  The parameters *gird_x* and *grid_y* control the grid size of the spatial histograms.
          At last the feature vectors (p here is the spatial histogram) are given .

  LBPH::predict(InputArray _src, int &minClass, double &minDist)

  compareHist(_histograms[sampleIdx], query, CV_COMP_CHISQR)

  Chi-square test is used for the distance measure

The Performance

• Receiver operating characteristic (ROC)

  • A ROC space is defined by TAR (I.e., the rate of genuine attempts accepted) and FAR(I.e., the rate of impostor attempts accepted) as y and x axes respectively, which depicts relative trade-offs between true positive (benefits) and false positive (costs).

• Cumulative Match Characteristic (CMC)

  • It plots the probability of identification against the returned 1:N candidate list size. It shows the probability that a given user appears in different sized candidate lists.

Reference

• Face Recognition with OpenCV
• Eigenface
• FaceRecognizer API
• ROC