Research of Face Location System Based on Human Vision Simulations

Research of Face Location System Based on Human Vision SimulationsMo Huayi, Li Weijun, Lai Jiangliang, Dai LeiLab of Artificial Neural Networks, Institute of Semiconductors, CAS, Beijing 100083, Chinamohuayi@AbstractIn this paper we present a robust face location system based on human vision simulations to automatically locate faces in color static images. Our method is divided into four stages. In the first stage we use a gauss low-pass filter to remove the fine information of images, which is useless in the initial stage of human vision. During the second and the third stages, our technique approximately detects the image regions, which may contain faces. During the fourth stage, the existence of faces in the selected regions is verified. Having combined the advantages of Bottom-Up Feature Based Methods and Appearance-Based Methods, our algorithm performs well in various images, including those with highly complex backgrounds.Key words: face location, human vision, low-pass filter, image segmentation, region selecting and merging1. IntroductionAutomatic face location (also named face detection) is a very important task, which constitutes the "first step” of a large area of applications: face recognition, face retrieval by similarity, face tracking, surveillance, etc. Many face-location approaches have been proposed in the past 30 years, and according to the face information used, they can be roughly divided into tow categories [1, 2]: 1) Bottom-Up Feature Based Methods which include facial feature (such as eyes, nose, mouth, face contour) methods[3-9], skin color methods[10], active contour model methods[11], and template matching methods [3, 12]; 2) Appearance-Based Methods which include linear subspace methods [13], neural network methods[14], SVM methods[15], SNoW methods[16], and adaBoost methods[17]. Of the two categories, the bottom-up feature based methods have the advantages of vividness, easily being implemented and designed in various circumstances, but also have the disadvantages of being easily affected by noise and complex backgrounds of the images, meanwhile, Appearance-Based Methods generally have the advantages of strong robustness, great accuracy, but also have the disadvantages of high complexity and great difficulty in generalization and collecting training simples.Considering the advantages and disadvantages of these two kinds of methods, we design a novel face location system based on human vision simulations, which not only have the advantages of vividness, easily being implemented and designed in various circumstances of Bottom-up F eature Based Methods, but also have the advantages of strong robustness and great accuracy of Appearance-Based Methods mentioned.2. Human vision process of locating facesNowadays the human vision is still the best approach to locate faces in images, whose process consists of the following steps:Firstly, the human vision system perceives a scene image by means of the peripheral vision to get a low-resolution image[18, 19]. Secondly, before the eyes fixating on a certain region of the image, the human vision system segments the low-resolution image into several discrete regions[20]. Thirdly, the eyes focus on a certain region of interest and get the high-resolution image of the region (each represents an object) with the fovea vision, and in most cases, a face region can be focused on easily[21, 22]. Finally, the vision system identifies the high-resolution image, usually in accordance with the internal features of the region[20]. 3. Experiment steps based on human vision simulationsOur face location system is to simulate the human vision program and divided into four stages as follows.2008 International Conference on Intelligent Computation Technology and Automation3.1. Low-pass filteringThe input image is a colorful one with one or more faces in it. In our experiment, we use gauss low pass filter to process the initial image. The transfer function of Gauss low pass filter is:22(,)/2(,)Du v H u v e VHere ı is the standard deviation, let ı=D 0(Ceiling cutoff frequency), we can get the following expression:220(,)/2(,)D u v D H u v eWhile0(,)D u v D, the filter value drops from its maximum 1 to 0.607. In our experiment, we set01min(,)6D height widthFor in a human vision system, the sharpness of the fovea vision is about 6 multiples of that of the peripheral vision. Here height and width represents those of the image.After filtering the original image, we get a blurred image (fig. 2(b), and fig. 2(a) is the original image) as follows.3.2 Image segmentationHuman vision can segment the perceived image into several discrete regions. The way that the human vision perceives a scene image is similar to the way that artificial satellites perceive a terrain region imageon the earth from the sky. In order to get the segmented regions, we adopt a watershed algorithm based on immersion simulations[23] to segment the blurred image.(a) (b)F ig. 2. The original image (a) and the filtered image (b).To enhance the effect of the watershed transition, we use the gradient amplitude image of the filtered image. Here the sobel operator is employed to detect edges:121000121 §·¨¸¨¸¨¸©¹101202101 §·¨¸ ¨¸¨¸ ©¹G x G yF g Because the filtered image is a colorful one and has three color components (each is a 2-D image), we can get the gradient amplitude images of each component and combine them to form the final gradient amplitude image.g Here g r , g b, g g are the gradient amplitudes of the R,G,B components.To avoid the excess segment regions during the following watershed segmentation, we can make some changes in the gradient amplitude image:or a M×N gradient amplitude image g, let g(x,y)=0, while 0111(,),(,)M Ni j g x y c g i j MNd ¦¦ or1x c or 1y c or 1y N c ! .In our experiment, 00.8c ,1max(0.02min(,),2)c M N .After the watershed segmentation, the filtered image is segmented into several discrete regions (fig.3 (a)).In many cases (Fig. 3(a) for example), a face region will be segmented into several regions, and we need tomerge them into one. The desirable target is to merge the adjacent regions with high similarity. Here we employ the gray level average difference and texture difference to measure the similarity.(a) (b)F ig. 3. The segmenting result (a) and the merging result (b).As to the two adjacent regions R i and R j , each has three components of R, G, B, the gray level average difference of Rand R is defined as:M iR , M iG , M iB represent the gray level averages of the three components of R i , and M jR ,M jG ,M jB represent the gray level averages of the three components of R j . The texture difference is defined as:2(,)d i j iR iG iB V V V ǃǃare the standard deviations of the R,G, B components of R i , etc.The similarity between R and R is defined as:(,)d i j If d (i,j)<40, R i and R j are merged into one, and fig.3 (b) is the merging result.3.3. Face candidate region selectionAfter the image segmentation stage, the filtered image is segmented into several regions, and a face is contained by one of them. Since our target is to find out the regions containing faces, we can use the transcendental face information such as the complexion, size and shape of the face.Here we create a complexion model to detect the face region candidates. The color space we have chose n in our experiments is YCbCr space, since in YCbCr space the clustering result of the complexion points is good.YCbCr format can be transformed from RGB format in this way:0.299000.587000.1140000.168740.331260.500001280.500000.418690.08131128Y R Cb G Cr B ªºªºªºªº«»«»«»«» «»«»«»«»«»«»«»«» ¬¼¬¼¬¼¬¼The distribution of our complexion simple pixels (34800 pixels from 24 images) in YCbCr space and CbCr space is:(a) (b)F ig. 4. The distribution of our complexion simple pixels in Ycbcr space (a) and CbCr space (b).We use a 2-D gauss distribution to describe fig.4 (b)10000000110011(,)exp(()cov ())2(,),(,),11,,348001cov ()(),(,)T NN i i i i NTi i i i i i Cb Cr Cb Cr Cb Cr Cb Cb Cr Cr N N N Cb Cr N P P P P P P P P P P P ­½U °°°°°°°°®¾ °°°°°° °°¯¿¦¦ F rom the sample points we get ȝ0 =(111.7751.,148.2287)ˈcov=¸¸¹·¨¨©§137.30449.50659.506587.6061Select a proper threshold Į (in our experiment Į=0.2), while ȡ(Cb,Cr)>Į, we believe that the pixel is a complexion one. A face region candidate must have more than 50% complexion pixels.In the other hand, a face region candidate also must have enough pixels, for it is hard to verify a candidate with few pixels. In our experiment, the minimum of the pixel number of a face region candidate is 1/600 of that of the input image.At last, we attain the height and width of the minimum rectangle containing the a segmented region, if the region is a face region, the ratio of height to width will distribute in [a,b], in our experiment, a=0.8, b=1.6.The selecting result of fig.3 (b) is showed in fig5. (b), and fig. 5 (a) is the selecting result showed in origin input image.3.4 Verifying the face region candidates in the original image.1(,)d i jWhile human vision system perceives a certain region of the images with the fovea vision, the fine, high-resolution information of the region is attained and identified. We adopt an eyes-location method mentioned in [24](our previous work) and a template-matching method mentioned in[10] to verify the corresponding face region candidates (while verifying a region candidate, ignore other regions) in the original color image respectively , the final detection result is show in fig.5 (c).(a) (b) (c)Fig. 5. The selected regions showed in original image (a) and filtered image (b), the verifying result (c).4. Experiment ResultsWe have evaluated our algorithm on several face databases, including ceruleansky 1.0 face database of our lab, family collections, photos downloaded from websites with the hardware of a personal computer with a P4 1.8G CPU, 512M memories and software of windows XP OS and matlab 7.0.Each image is colorful and contains one or more complete, upright, moderate size faces in the normal light condition. In the image set with simple backgrounds (93 images with 108 faces, for example fig. 4(c), fig.5 (a)), we got True Positive Rate (TPR) of 98% and none False Detections (FD). In the image set with moderately complex backgrounds (84 images with 101 faces, fig. 5(b) for example), we got 94% TPR and 4 FD. In the image set with highly complex backgrounds (77 images with 95 faces, fig. 5(c), fig. 5(d) for example), we got TPR of 90% and 12 FD.5. DiscussionThis work proposes a four-stage approach, which is a “coarse to fine” course similar to that of human vision, to locate faces in color static images. Our approach combines the advantages of Bottom-Up F eature Based Methods and Appearance-Based Methods, and works well in various images even those with highly complex backgrounds.Therefore, we believe that the human vision, which works very well on image processing, will provide useful cues to the computer vision, and computer vision researchers should pay more attention to the researches of the rules of the human vision.(a) (b)(c) (d)Fig. 6. The experiment results. (a) simple background image set; (b) moderately complex background image set; (c), (d) highly complex background image set.6. AcknowledgementThis paper is supported by 863 project, 2006AA01Z12.7. Reference[1] Huang fuzhen and S. jianbo, Face Detection (book inChinese). Shanghai: Shanghai Jiao Tong Universitys press, 2006.[2] Y. Ming-Hsuan, D. J. Kriegman, and N. Ahuja,"Detecting faces in images: a survey," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 24, pp. 34-58, 2002.[3] A. Lanitis, C. J. Taylor, and T. F. Cootes, "Automaticface identification system using flexible appearance models," Image and Vision Computing, vol. 13, pp. 393-401, 1995.[4] D. W. Hansen and R. I. Hammoud, "An improvedlikelihood model for eye tracking," Computer Visionand Image Understanding, vol. 106, pp. 220-230, 2007.[5] I. Craw, H. Elis, and J. R. Lishman, "Automaticextraction of face-features," Pattern Recogn. Lett., vol.5, pp. 183-187, 1987.[6] A. L. Yuille, P. W. Hallinan, and D. S. Cohen, "Featureextraction from faces using deformable templates,"International Journal of Computer Vision, vol. 8, pp.99-111, 1992.[7] D. Maio and D. Maltoni, "Real-time face location ongray-scale static images," Pattern Recognition, vol. 33,pp. 1525-1539, 2000.[8] L. C. De Silva, K. Aizawa, and M. Hatori, "Detectionand tracking of facial features," in VisualCommunications and Image P rocessing '95, Taipei,Taiwan, 1995, pp. 1161-1172.[9] V. Govindaraju, S. N. Srihari, and D. B. Sher, "Acomputational model for face location," in ComputerVision, 1990. P roceedings, Third International Conference on, 1990, pp. 718-721.[10] R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, "F acedetection in color images," P attern Analysis andMachine Intelligence, IEEE Transactions on vol. 24,pp. 696-706, 2002.[11] S. R. Gunn and M. S. Nixon, "Global and local activecontours for head boundary extraction," InternationalJournal of Computer Vision, vol. 30, pp. 43-54, 1998. [12] H.-S. Kim, W.-S. Kang, J.-I. Shin, and S.-H. Park,"F ace detection using template matching and ellipsefitting," IEICE Transactions on Information andSystems, vol. 383-D, pp. 2008-2011, 2000.[13] M. S. Bartlett, M. S. Bartlett, J. R. Movellan, and T. J.Sejnowski, "F ace recognition by independent component analysis," Neural Networks, IEEE Transactions on, vol. 13, pp. 1450-1464, 2002.[14] S. Kah-Kay and P. Tomaso, "Example-Based Learningfor View-Based Human Face Detection," IEEE Trans.Pattern Anal. Mach. Intell., vol. 20, pp. 39-51, 1998. [15] E. Osuna, R. Freund, and F. Girosit, "Training supportvector machines: an application to face detection," inComputer Vision and P attern Recognition P roceedings., 1997 IEEE Computer SocietyConference on, 1997, pp. 130-136.[16] M. Nilsson, J. Nordberg, and I. Claesson, "F aceDetection using Local SMQT F eatures and Split upSnow Classifier," in Acoustics, Speech and SignalProcessing, 2007. IEEE International Conference on,2007, pp. II-589-II-592.[17] Z. ZhenQiu, Z. Long, S. Z. Li, and A. H. Z. HongJiangZhang, "Real-time multi-view face detection," inAutomatic Face and Gesture Recognition, 2002.Proceedings. Fifth IEEE International Conference on,2002, pp. 142-147.[18] K. Rayner, "Eye movements in reading andinformation processing: 20 years of research,"Psychological Bulletin, vol. 124, pp. 372-422, 1998. [19] J. M. Henderson and A. Hollingworth, "High-levelscene perception," Annual Review of P sychology, vol.50, pp. 243-271, 1999.[20] Jing qicheng, Jiao shulan, and J. guiping, HumanVision (book in Chinese). Beijing: Science Press, 1987. [21] S. K. Mannan, K. H. Ruddock, and D. S. Wooding,"The relationship between the locations of spatialfeatures and those of fixations made during visualexamination of briefly presented images " SpatialVision, vol. 10, pp. 165-188, 1996.[22] P. G. Schyns and A. Oliva, "F ROM BLOBS TOBOUNDARY EDGES:. Evidence for Time- andSpatial-Scale-Dependent Scene Recognition,"Psychological Science, vol. 5, pp. 195-200, 1994.[23] L. Vincent and P. Soille, "Watersheds in digital spaces:An efficient algorithm based on immersionsimulations," IEEE Transactions on P attern Analysisand Machine Intelligence, vol. 13, pp. 583-598, 1991. [24] Z.-H. Wang, H.-Y. Mo, H.-X. Lu, and A. W. S.-J.Wang Shou-Jue, "A method of biomimetic patternrecognition for face recognition," in Neural Networks,2003. P roceedings of the International JointConference on, 2003, pp. 2216-2221 vol.3.。