Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Analysis and Machine Intelligence,

"... Abstract—In this paper, a completed modeling of the LBP operator is proposed and an associated completed LBP (CLBP) scheme is developed for texture classification. A local region is represented by its center pixel and a local difference sign-magnitude transform (LDSMT). The center pixels represent t ..."

Abstract - Cited by 73 (6 self) - Add to MetaCart

Abstract—In this paper, a completed modeling of the LBP operator is proposed and an associated completed LBP (CLBP) scheme is developed for texture classification. A local region is represented by its center pixel and a local difference sign-magnitude transform (LDSMT). The center pixels represent the image gray level and they are converted into a binary code, namely CLBP-Center (CLBP_C), by global thresholding. LDSMT decomposes the image local differences into two complementary components: the signs and the magnitudes, and two operators, namely CLBP-Sign (CLBP_S) and CLBP-Magnitude (CLBP_M), are proposed to code them. The traditional LBP is equivalent to the CLBP_S part of CLBP, and we show that CLBP_S preserves more information of the local structure than CLBP_M, which explains why the simple LBP operator can extract the texture features reasonably well. By combining CLBP_S, CLBP_M, and CLBP_C features into joint or hybrid distributions, significant improvement can be made for rotation invariant texture classification.

(Show Context)

Towards Practical Smile Detection

by Jacob Whitehill, Gwen Littlewort, Ian Fasel, Marian Bartlett, Javier Movellan

"... Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting ..."

Abstract - Cited by 66 (11 self) - Add to MetaCart

Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting conditions on a relatively small number of subjects. This paper explores whether current machine learning methods can be used to develop an expression recognition system that operates reliably in more realistic conditions. We explore the necessary characteristics of the training dataset, image registration, feature representation, and machine learning algorithms. A new database, GENKI, is presented which contains pictures, photographed by the subjects themselves, from thousands of different people in many different real-world imaging conditions. Results suggest that human-level expression recognition accuracy in reallife illumination conditions is achievable with machine learning technology. However, the datasets currently used in the automatic expression recognition literature to evaluate progress may be overly constrained and could potentially lead research into locally optimal algorithmic solutions.

The first facial expression recognition and analysis challenge

by Michel F. Valstar, Bihan Jiang, Marc Mehu, Maja Pantic, Klaus Scherer - In Proc. IEEE Intl Conf. Automatic Face and Gesture Recognition, in print , 2011

"... discrete emotion detection, has been an active topic in computer science for over two decades. Standardisation and comparability has come some way; for instance, there exist a number of commonly used facial expression databases. However, lack of a common evaluation protocol and lack of sufficient de ..."

Abstract - Cited by 56 (9 self) - Add to MetaCart

discrete emotion detection, has been an active topic in computer science for over two decades. Standardisation and comparability has come some way; for instance, there exist a number of commonly used facial expression databases. However, lack of a common evaluation protocol and lack of sufficient details to reproduce the reported individual results make it difficult to compare systems to each other. This in turn hinders the progress of the field. A periodical challenge in Facial Expression Recognition and Analysis would allow this comparison in a fair manner. It would clarify how far the field has come, and would allow us to identify new goals, challenges and targets. In this paper we present the first challenge in automatic recognition of facial expressions to be held during the IEEE conference on Face and Gesture Recognition 2011, in Santa Barbara, California. Two sub-challenges are defined: one on AU detection and another on discrete emotion detection. It outlines the evaluation protocol, the data used, and the results of a baseline method for the two sub-challenges. I.

(Show Context)

WLD: A robust local image descriptor

by Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikäinen, Senior Member, Xilin Chen, Senior Member, Wen Gao - IEEE Transactions on Pattern Analysis and Machine Intelligence , 2010

"... Abstract—Inspired by Weber’s Law, this paper proposes a simple, yet very powerful and robust local descriptor, called the Weber Local Descriptor (WLD). It is based on the fact that human perception of a pattern depends not only on the change of a stimulus (such as sound, lighting) but also on the or ..."

Abstract - Cited by 51 (1 self) - Add to MetaCart

Abstract—Inspired by Weber’s Law, this paper proposes a simple, yet very powerful and robust local descriptor, called the Weber Local Descriptor (WLD). It is based on the fact that human perception of a pattern depends not only on the change of a stimulus (such as sound, lighting) but also on the original intensity of the stimulus. Specifically, WLD consists of two components: differential excitation and orientation. The differential excitation component is a function of the ratio between two terms: One is the relative intensity differences of a current pixel against its neighbors, the other is the intensity of the current pixel. The orientation component is the gradient orientation of the current pixel. For a given image, we use the two components to construct a concatenated WLD histogram. Experimental results on the Brodatz and KTH-TIPS2-a texture databases show that WLD impressively outperforms the other widely used descriptors (e.g., Gabor and SIFT). In addition, experimental results on human face detection also show a promising performance comparable to the best known results on the MIT+CMU frontal face test set, the AR face data set, and the CMU profile test set. Index Terms—Pattern recognition, Weber law, local descriptor, texture, face detection. Ç 1

(Show Context)

Local derivative pattern versus local binary pattern: Face recognition with higher-order local pattern descriptor

by Baochang Zhang, Yongsheng Gao, Sanqiang Zhao, Jianzhuang Liu - IEEE TRANS. IMAGE PROCESS , 2010

"... This paper proposes a novel high-order local pattern descriptor, local derivative pattern (LDP), for face recognition. LDP is a general framework to encode directional pattern features based on local derivative variations. The-order LDP is proposed to encode the -order local derivative direction ..."

Abstract - Cited by 46 (6 self) - Add to MetaCart

This paper proposes a novel high-order local pattern descriptor, local derivative pattern (LDP), for face recognition. LDP is a general framework to encode directional pattern features based on local derivative variations. The-order LDP is proposed to encode the -order local derivative direction variations, which can capture more detailed information than the first-order local pattern used in local binary pattern (LBP). Different from LBP encoding the relationship between the central point and its neighbors, the LDP templates extract high-order local information by encoding various distinctive spatial relationships contained in a given local region. Both gray-level images and Gabor feature images are used to evaluate the comparative performances of LDP and LBP. Extensive experimental results on FERET, CAS-PEAL, CMU-PIE, Extended Yale B, and FRGC databases show that the high-order LDP consistently performs much better than LBP for both face identification and face verifi-cation under various conditions.

(Show Context)

Motion Interchange Patterns for Action Recognition in Unconstrained Videos

by Orit Kliper-gross, Yaron Gurovich, Tal Hassner, Lior Wolf

"... Abstract. Action Recognition in videos is an active research field that is fueled by an acute need, spanning several application domains. Still, existing systems fall short of the applications ’ needs in real-world scenarios, where the quality of the video is less than optimal and the viewpoint is u ..."

Abstract - Cited by 44 (2 self) - Add to MetaCart

Abstract. Action Recognition in videos is an active research field that is fueled by an acute need, spanning several application domains. Still, existing systems fall short of the applications ’ needs in real-world scenarios, where the quality of the video is less than optimal and the viewpoint is uncontrolled and often not static. In this paper, we consider the key elements of motion encoding and focus on capturing local changes in motion directions. In addition, we decouple image edges from motion edges using a suppression mechanism, and compensate for global camera motion by using an especially fitted registration scheme. Combined with a standard bag-of-words technique, our methods achieves state-of-the-art performance in the most recent and challenging benchmarks. 1

(Show Context)

Multi-scale local binary pattern histograms for face recognition

by Chi Ho Chan - In Proc. of the 2nd International Conference on Biometrics , 2007

"... Recently, the research in face recognition has focused on developing a face representation that is capable of capturing the relevant information in a manner which is invariant to facial expression and illumination. Motivated by a simple but powerful texture descriptor, called Local Binary Pattern (L ..."

Abstract - Cited by 39 (7 self) - Add to MetaCart

Recently, the research in face recognition has focused on developing a face representation that is capable of capturing the relevant information in a manner which is invariant to facial expression and illumination. Motivated by a simple but powerful texture descriptor, called Local Binary Pattern (LBP), our proposed system extends this descriptor to evoke multiresolution and multispectral analysis for face recognition. The first descriptor, namely Multi-scale Local Binary Pattern Histogram (MLBPH), provides a robust system which is relatively insensitive to localisation errors because it benefits from the multiresolution information captured from the regional histogram. The second proposed descriptor, namely Multispectral Local Binary Pattern Histogram (MSLBP), captures the mutual relationships between neighbours at pixel level from each spectral channel. By measuring the spatial correlation between spectra, we expect to achieve higher recognition rate. The resulting LBP methods provide input to LDA and various classifier fusion methods for face recognition. These systems are implemented and compared with existing Local Binary Pattern face recognition systems and other state of art systems on Feret, XM2VTS and FRGC 2.0 databases, giving very promising results

(Show Context)

X.: Fusing robust face region descriptors via multiple metric learning for face recognition

by Zhen Cui, Wen Li, Dong Xu, Shiguang Shan, Xilin Chen - in the wild. In: Computer Vision and Pattern Recognition (CVPR), IEEE , 2013

"... In many real-world face recognition scenarios, face images can hardly be aligned accurately due to complex ap-pearance variations or low-quality images. To address this issue, we propose a new approach to extract robust face re-gion descriptors. Specifically, we divide each image (resp. video) into ..."

Abstract - Cited by 35 (4 self) - Add to MetaCart

In many real-world face recognition scenarios, face images can hardly be aligned accurately due to complex ap-pearance variations or low-quality images. To address this issue, we propose a new approach to extract robust face re-gion descriptors. Specifically, we divide each image (resp. video) into several spatial blocks (resp. spatial-temporal volumes) and then represent each block (resp. volume) by sum-pooling the nonnegative sparse codes of position-free patches sampled within the block (resp. volume). Whitened Principal Component Analysis (WPCA) is further utilized to reduce the feature dimension, which leads to our Spatial Face Region Descriptor (SFRD) (resp. Spatial-Temporal Face Region Descriptor, STFRD) for images (resp. videos). Moreover, we develop a new distance metric learning method for face verification called Pairwise-constrained Multiple Metric Learning (PMML) to effectively integrate the face region descriptors of all blocks (resp. volumes) from an image (resp. a video). Our work achieves the state-of-the-art performances on two real-world datasets LFW and YouTube Faces (YTF) according to the restricted pro-tocol. 1.

(Show Context)

Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Analysis and Machine Intelligence, (2007)

Tools