Gravity-center template based human face feature detection

Gravity-Center Template Based Human Face FeatureDetectionJun Miao1, Wen Gao1, Yiqiang Chen1, and Jie Lu 21Digital Media Lab, Institute of Computing Technology, Chinese Academy of Sciences,Beijing 100080, P. R. China{jmiao,wgao,yqchen}@2Institute of Electronic Information Engineering, Tongji University, Shanghai 200092, P. R.ChinaAbstract.This paper presents a simple and fast technique forgeometrical feature detection of several human face organs such as eyesand mouth. Human face gravity-center template is firstly used for facelocation, from which position information of face organs such aseyebrows, eyes, nose and mouth are obtained. Then the original imageis processed by extracting edges and the regions around the organs arescanned on the edge image to detect out 4 key points which determinethe size of the organs. From these key points, eyes and mouth's shapeare characterized by fitting curves. The results look well and theprocedure is fast.Keywords face detection, edge extraction, face feature detection1 IntroductionHuman face and its feature detection is much significant in various applications as human face identification, virtual human face synthesis, and MPEG-4 based human face model coding. As for face image processing , many people have done much work on them. The representative jobs include mosaic technique[1,2], fixed template matching[3], deformable template matching[4], gravity-center template matching [5], “eigenface”[6] or “eigenpicture”[7] scheme, neural network[8,9], and usage of multiple information including color, sound or motion[10,11].There also have been quite a few techniques reported about face contour and organ geo14metrical feature extraction, such as B-splines[12], adaptive Hough trasmform[13], cost minimiziation[14], deformable template[15,16] , region-based template[17], geometrical model[18]. However, most of them are either complex or time-consuming. Here, we present a simple and fast approach based on the technique of gravity-center template to locate face organs such as eyes and mouth, and detect their key points to characterize their scales and shape.2 Simplified geometrical model for eyes and mouthThe eyes and mouth have similar geometrical configuration as shown in fig. 1. There are four key points, the highest point P1, the lowest point P2, the left corner point P3 and the right corner point P4, which determine the size (a×b) and the arc-like(R1,R2) shape of the organs. Our goal is to detect out these key points to segment face regionand concisely describe the features of eyes and mouth.Fig. 1. Simplified model for eyes and mouth3 Detection System ArchitectureWe have implemented a facial geometrical feature extraction system for eyes and mouth based on human face gravity-center template technique the author recently suggested [5]. The system consists of there modules : the first one of gravity-center template based human face and its organs location, the second one of four key points extraction, and the last one of geometrical curves fitting.Inputting an original image, it will be processed by mosaicizing, mosaic edge extracting and feature simplifying for gravity-center template matching to locate human face in an unconstrained background and by adaptive Sobel edge extracting for feature points localization. From this information, geometrical shape of eyelids and lips can be easily fitted.image imageFig. 2. System architecture3.1 Face and facial organs localization based on face gravity-centertemplate matchThis procedure will produce the location information about face and its organs such aseyebrows, eyes, nose and mouth, which is necessary for further facial featuredetection. We use four face gravity-center templates [5] to detect frontal faces. Inthese templates, there are 4 to 6 small boxes which correspond to the facial organs:eyebrows, eyes, nose and mouth. Once a face in an image is located, simultaneously,its facial organs can be also located. The three steps b.(Image Mosaicizing), c.(MosaicHorizontal Edge Extracting) and d.(Feature Simplifying and Gravity-Cenetr TemplateMatching) in Fig.3 illustrate the procedure of face and its organs localization. Thedetail description on detecting procedures can be referred in reference [5].a. b.c. d.Fig. 3. Face and its organs localizationFig. 4. Face gravity-center templates3.2 Key feature points extractionFrom this step, four key points P1, P2, P3 and P4 illustrated in Fig.1 are expected to be extracted.The original image is firstly processed with an adaptive Sobel operator introduced in reference [19] to produce a binary edge image. Fig. 5 shows the result processed by this operator.Fig. 5. Binary edge image using an adaptive Sobel operatorAccording to the face organs’ location information obtained from gravity-center template matching, we can scanned in the edge image in areas around those organs to see if current position is one of four key feature point of P1, P2, P3 and P4 showed in Fig.1. If it is true, its coordinates is to be calculated out using a mechanism of median-pass from the coordinates of the points scanned in rows or columns. The procedure is shown in Fig.6.a. b.Fig. 6. Key feature points searching ( e.g. left eye)In the upper picture of Fig.6 the black area is cut from the area of the left eye in the Fig. 5, in which the position of the cross point of lines X, Y is obtained from the center of the small box corresponding to left eye. And lines X and Y are the initial positions in horizontal and vertical directions to scan and search the top and bottom sides, the left and right sides of the area. Scan starts from the initial X, Y lines and extend upwards and downwards, leftwards and rightwards, respectively in horizontal and vertical lines. A scan towards the one of the four directions will stop when the number of edge pixels it encounters in the scanning line that is orthogonal to the direction the scan towards is less than a threshold such as 3 edge pixels. In this case, the median value pixel is chosen as one key point which belongs to one of P1, P2, P3 and P4.3.3 Eyelids and lips shapes descriptionAccording to the simplified model for eyes and mouth’s, from four key feature points: P1, P2, P3 and P4, a upper and a lower circle arc can be generated to fit the upper and the lower eyelids or lips. The effect is shown in Fig.7.Fig. 7. Eyelids and lips fitting4 Some experimental examplesFig.8 to Fig.15 are part of the experiment results.Fig. 8.Fig. 9.Fig. 10. Fig. 11.Fig. 12.Fig. 13.Fig. 14. Fig. 15.5 Conclusion and future workCompared with some other detecting schemes such as deformable template technique, the approach introduced here is simple and fast ( 0.28 seconds in average on Pentium-550 PC ). Especially for faces with complex background, our system can work as well.During experiments, we still found some detection results of facial feature for those faces with complex texture and poor light conditions are not as well as those for good quality face images. And for the moment, this method can not characterize lips in more details yet.Further work will include eyeball, nose and cheek contour detection and the algorithm is expected to be enhanced to improve the system’s robustness. References1. L.D. Harmon, The recognition of faces, Scientific American, 229(5), 71-82, 1973.2. G. Z. Yang and T. S. Huang, Human face detection in a complex background,Pattern Recognition, 27(1), 43-63, 19943. W. Gao and M. B. Liu, A hierarchical approach to human face detection in acomplex background, Proceedings of the first International Conference on Multimodal Interface’96,289-292, 19964. A.L. Yuille, Deformable templates for face detection, J. Cogn. neurosci.3, 59-70,19915. J. Miao, B.C. Yin, K.Q. Wang, et al, A hierarchical multiscale and multianglesystem for human face detection in a complex background using gravity-center template, Pattern Recognition, 32(7), 19996. M. Turk and A. Pentland, Face recognition using eigenfaces, Proc. IEEE-CS-CCVPR, 586-591, 19917. M. Kirby and L. Sirovich, Application of the Karhunen-Loeve procedure for thecharacterization of human faces, IEEE Trans. PAMI, 12(1), 103-108, 19908. H.A. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection,IEEE-PAMI, 20(1), 23-38, 19989. K.K. Sung and T. Poggio, Example-based learning for view-based human facedetection, IEEE-PAMI, 20(1), 39-50, 199810.R. Brunelli and D. Falavigna, Person identification using multiple cues, IEEETrans. PAMI, 17(10), 955-966, 199511.C.H. Lee, J.S. Kim and K.H. Park, Automatic human face location in a complexbackground using motion and color information, Pattern Recognition, 29(11), 1877-1889, 199612.C.J. Wu and J.S. Huang, Human faces profiles recognition by computer, PatternRecognition, 23(3/4), 255-259, 199013.X. Li and N. Roeder, Face contour extraction from front-view images, Patternrecognition, 28(8), 1167-1179, 199514.K. Lam and H. Yan, Locating and extracting the eye in human face images,Pattern recognition, 29(5), 771-779, 199615.A.L. Yuille, D.S. Cohen and P.W. Hallinan, Feature extraction from faces usingdeformable templates, Proc. IEEE-CS-CCVPR, 104-109, 198916.C.L. Huang and C.W. Chen, Human face feature extraction for face interpretationand recognition, Pattern recognition, 25(12), 1435-1444, 199617.J.Y. Deng and F. Lai, Region-based template deformation and masking for eye-feature extraction and description, Pattern recognition, 30(3), 403-419, 199718.S.H. Jeng, H.Y.M. Liao, et al, Facial feature detection using geometrical facemodel: an efficient approach, Pattern recognition, 31(3), 273-282, 199819.S.Y. Lee, Y.K. Ham and R.H. Park, Recignition of human front faces usingknowledge-based feature extraction and neuro-fuzzy algorithm, Pattern recognition, 29(11), 1863-1876, 1996。