A survey of techniques for Face Reconstruction

1-4244-0991-8/07/$25.00/©2007 IEEE3554have:
u v 1 = f cos θ 0 − sin θ 0 f 0 f sin θ 0 cos θ ft ft tz x
x y
(1)
3-points method. First three points are found with the following properties:
A survey of techniques for Face Reconstruction
Ching Y. Suen 1, Arash Zaryabi Langaroudi2, Chunghua Feng1, Yuxing Mao 1
1
Centre for Pattern Recognition and Machine Intelligence (CENPARMI), Concordia University, Montreal, QC, Canada
2
Department of Computer Science and Software Engineering, Concordia University, Montreal, QC, Canada
Abstract— This paper presents a study on some approaches in face reconstruction and the methodology used. The results and drawbacks of each method are discussed. To construct the 3D face model there are some methods which select the features automatically or manually. First a method in textured 3D face reconstruction using two 2D images from any angle is discussed which does not need any particular database, but it has to define the feature points manually. Then we describe automatic 2D to 3D face reconstruction from a single frontal face image which needs the use of USF Human ID 3-D database. Afterwards we talk about a Model Based Face Reconstruction for Animation. We also describe briefly about 3D face modeling by fusing multiple 2D images which is fully automatic and via an EM approach which uses the shape and pose parameters. Finally we describe a Rapid Modeling of Animated Faces from Video method and 2D face reconstruction using a minimum set of feature points. Keywords: Face Reconstruction, feature point, texture generating, Pose estimation.
This work was supported in part by Bell Canada and the Natural Sciences and Engineering Research Council of Canada.
Fig. 1. System Overview [1]
A. Analytical 3D reconstruction algorithm for feature points By using orthogonal projections, the 3D Coordinates of the feature points are generated. If P=(x,y,z)T are 3D coordinates on the real face shape then its homogeneous coordinates will be P’=(x,y,z,1). Also Q=(u,v)T which are the corresponding coordinates of the image point in the camera are Q’=(u,v,1)T. By using camera pinhole model and assuming that the world coordinate system only rotates around the y-axis, we
of the database. In this paper we present a survey of some of the approaches to reconstruct the 3D face shape. This paper is organized as follows: First a method in textured 3D face reconstruction using two 2D images from any angle is discussed which does not need any particular database but it needs to define the feature points manually [1]. Then we describe an automatic 2D to 3D face reconstruction from a single frontal face image which needs the use of USF Human ID 3-D database [2]. Then we will talk about a Model Based Face Reconstruction for Animation [8], and briefly talk about 3D face modeling by fusing multiple 2D images which is fully automatic. Afterwards we discuss about a Rapid Modeling of Animated Faces based on the Video method [6] and 2D face reconstruction using a minimum set of feature points [4]. II. 3D FACE RECONSTRUCTION WITH TWO IMAGES AT ANY ANGLE USING 35 FEATURE POINTS [1] In this section we will describe a new approach for textured 3D face reconstruction based on two images taken at any angle. This technique uses 35 feature points corresponding to 2 images which are marked manually. First the analytical coordinates of the feature points are confirmed. Then by using Kriging interpolation method, missing vertices are produced. Finally the view-dependant or view-independent texture maps are generated.
y z 1
zA = zB
x c = (x A + x B )/2 z A ≠ zc
(2)
where f is the focal length of the camera and tx, ty, t z are the translation vector parameters in pinhole camera mode and is the angle of the image according to the camera. This technique assumes that the projection is orthogonal which requires the images of the face to be taken from a distance of two meters or more. We have two images with different angles 1 and 2. The feature points are marked on both images, so Q1 and Q2 show the same physical point on the real world (See Fig. 2). By solving equation 1 for x, y and z with 1 and 2, the 3D coordinates of the feature points will be obtained.
I. INTRODUCTION 3D modeling of human faces has applications ranging from 3D human face animation to video conferencing. The intuitive method to get face's 3D information is scanning it using a 3D scanner. In Laser Scanning for each pixel of the image, the range to the visible surface of the objects in the scene is known. Lee et al [23] digitized facial geometry through the use of scanning range sensors, but the approach based on 3D digitization and other methods are either cumbersome to operate or expensive to obtain. Cheaper and easier alternatives have been explored in the past two decades. It is a challenge to realize 3D face construction based on 2D face images. There are already numerous complicated techniques in this field. Various algorithms have been proposed for this problem, but the performances have been far from satisfactory. There are feature-based and appearance-based methods to construct a 3D face model. In the first method, a generic model in 3D is provided in advance, and a limited number of feature points are extracted either automatically or manually. Usually only the feature points with distinct characteristics can be located automatically. Feature points cooperate with the generic face information in the database to construct the face shape accurately. Note that the construction of an accurate face shape will be constrained by the characteristics