CN106203248A - Method and apparatus for face recognition - Google Patents

Abstract

A method and apparatus for facial recognition are provided. At least one exemplary embodiment discloses a facial recognition apparatus configured to: obtain a two-dimensional (2D) input image including a user's facial region; detect facial feature points from the 2D input image; adjust the pose of a stored three-dimensional (3D) facial model based on the detected facial feature points; generate a 2D projection image from the adjusted 3D facial model; perform facial recognition based on the facial region in the 2D input image and the facial region in the 2D projection image; and output a result of the facial recognition.

Description

Method and device for facial recognition

This application claims Korean Patent Application No. 10-2014-0118828 filed with the Korean Intellectual Property Office on September 5, 2014 and Korean Patent Application No. 10-2015-0001850 filed with the Korean Intellectual Property Office on January 7, 2015 application, the entire content of each of said Korean Patent Applications is hereby incorporated by reference in its entirety.

technical field

At least some example embodiments relate to face recognition technology for recognizing a user's face.

Background technique

Unlike identification technologies that require users to perform specific movements or actions, such as fingerprint recognition and iris recognition, facial recognition technology is considered as a convenient and competitive biometric technology that can authenticate a target without making contact with the target . Due to its convenience and effectiveness, this face recognition technology has been widely used in various application fields (eg, security system, mobile authentication, and multimedia search).

Contents of the invention

At least some example embodiments relate to a face recognition method.

In at least some example embodiments, the face recognition method may include: detecting facial feature points from a two-dimensional (2D) input image; adjusting a stored three-dimensional (3D) face model based on the detected facial feature points; The adjusted 3D face model produces a 2D projection image; face recognition is performed based on the 2D input image and the 2D projection image.

The step of adjusting the stored 3D facial model may include adjusting a facial pose and a facial expression of the stored 3D facial model by mapping the detected facial feature points to the stored 3D facial model.

The stored 3D face models include 3D shape models and 3D texture models. The 3D shape model and the 3D texture model may be deformable 3D models of facial pose and facial expression.

The adjusting the stored 3D face model may include: adjusting the stored 3D shape model based on facial feature points detected from the 2D input image; and adjusting the 3D texture model based on parameter information of the adjusted 3D shape model.

The step of adjusting the stored 3D shape model may include: adjusting pose parameters and expression parameters of the 3D shape model based on the detected facial feature points.

The step of generating a 2D projection image may include: generating a 2D projection image from the adjusted 3D texture model.

The stored 3D face model may be generated based on feature points detected from a plurality of 2D face images, which may be images obtained by capturing the user's face from a plurality of viewpoints.

The step of performing face recognition may include: determining a degree of similarity between the 2D input image and the 2D projected image; and outputting a result of face recognition based on whether the degree of similarity satisfies a condition.

The step of detecting facial feature points may include: extracting a face area from a 2D input image; detecting a face in at least one of eyebrows, eyes, nose, lips, chin, ears, and face contour from the extracted face area Part features.

At least other example embodiments relate to a method of generating a three-dimensional (3D) facial model.

In some example embodiments, the method may include: obtaining a 2D facial image of the user from a plurality of viewpoints; detecting facial feature points from the 2D facial image; generating a deformable 3D shape model based on the detected facial feature points and a deformable 3D texture model; storing the deformable 3D shape model and the deformable 3D texture model as the user's 3D face model.

The generating may include generating a deformable 3D texture model based on the deformable 3D shape model and texture information from at least one of the 2D face images.

The step of generating includes: determining parameters for mapping the detected facial feature points to feature points of the 3D standard model; and generating a deformable 3D shape model by applying the determined parameters to the 3D standard model.

At least other example embodiments relate to a method of generating a 3D face model.

In at least some example embodiments, the method may include: obtaining a 2D facial image and orientation data of the 2D facial image, the 2D facial image including a user's face; information; generate 3D data of the user's face based on the orientation data of the 2D face image and information on the matching points; convert the 3D standard model into the user's 3D face model using the 3D data.

The obtaining may include obtaining direction data of the 2D face image using motion data sensed by the motion sensor.

The 3D data on the user's face may be a collection of 3D points configuring the shape of the user's face.

The converting may include converting the 3D standard model into the user's 3D face model by matching the 3D standard model with 3D data of the user's face.

At least other example embodiments relate to a facial recognition device.

In at least some example embodiments, the face recognition device may include: an image acquirer configured to obtain a 2D input image including a user's face area; a 3D face model processor configured to The face pose of the user in the input image adjusts the face pose of the stored 3D face model and generates a 2D projection image from the adjusted 3D face model; a face recognizer configured to base the 2D input image and the 2D projection image Perform facial recognition.

The 3D face model processor may include: an area detector configured to detect a face area from the 2D input image; and a feature point detector configured to detect face feature points from the detected face area.

The 3D face model processor may adjust a face pose of the stored 3D face model by matching the detected face feature points with feature points of the stored 3D face model.

The 3D face model processor may adjust the face pose of the 3D shape model based on the user's face pose appearing in the 2D input image, and adjust the 3D texture model based on parameter information of the adjusted 3D shape model.

The face recognition device may further include: a display configured to display at least one of: one of the 2D input images, a 2D projected image, and a result of the face recognition.

At least other example embodiments relate to an apparatus for generating a 3D facial model.

In at least some example embodiments, the apparatus may include: an image acquirer configured to obtain a 2D facial image of the user from a plurality of viewpoints; a feature point detector configured to detect facial features from the 2D facial image Point; 3D facial model generator, is configured to generate deformable 3D shape model and deformable 3D texture model based on the detected facial feature points; 3D facial model register, is configured to convert deformable 3D shape The model and deformable 3D textured model are stored as the user's 3D face model.

The 3D face model generator may include: a 3D shape model generator configured to generate a deformable 3D shape model of the user's face based on the detected facial feature points; a 3D texture model generator configured to generate a deformable 3D shape model based on the deformable The 3D shape model and texture information from at least one of the 2D facial images generate a deformable 3D textured model.

At least other example embodiments relate to an apparatus for generating a 3D face model.

In at least some example embodiments, the apparatus may include: an image acquirer configured to obtain a 2D face image of the user from a plurality of viewpoints; a motion sensing unit configured to obtain direction data of the 2D face image; a 3D face model generator configured to generate 3D data of the user's face based on information about matching points between the 2D face images and direction data of the 2D face images, the 3D face model generator being configured to The 3D standard model is converted into a 3D face model of the user by using the 3D data; the 3D face model register is configured to store the 3D face model of the user.

Additional aspects of example embodiments will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.

Description of drawings

These and/or other aspects will become apparent and more readily understood from the following description of example embodiments in conjunction with the accompanying drawings, in which:

FIG. 1 is a diagram illustrating an overall operation of a face recognition system according to at least one example embodiment;

2 is a diagram illustrating a configuration of a three-dimensional (3D) face model generating device according to at least one example embodiment;

FIG. 3 is a diagram illustrating a configuration of a face recognition device according to at least one example embodiment;

4 illustrates a process of detecting feature points from a two-dimensional (2D) facial image, according to at least one example embodiment;

5 illustrates a process of generating a 3D face model using a 3D standard model according to at least one example embodiment;

6 illustrates a process of adjusting a 3D face model based on feature points detected from a 2D input image, according to at least one example embodiment;

7 illustrates a process of performing face recognition by comparing a 2D input image with a 2D projection image, according to at least one example embodiment;

FIG. 8 is a flowchart illustrating a method of generating a 3D face model according to at least one example embodiment;

FIG. 9 is a flowchart illustrating a face recognition method according to at least one example embodiment;

FIG. 10 is a diagram illustrating another configuration of a 3D face model generating device according to at least one example embodiment;

FIG. 11 is a flowchart illustrating another 3D face model generation method according to at least one example embodiment.

detailed description

Hereinafter, some example embodiments will be described in detail with reference to the accompanying drawings. With regard to the reference numerals assigned to elements in the drawings, it should be noted that like elements will be indicated by the same reference numerals whenever possible even if they are shown in different drawings. Also, in describing example embodiments, when a detailed description of a well-known related structure or function is considered to cause obscure interpretation of the present disclosure, the detailed description will be omitted.

It should be understood, however, that there is no intent to limit the disclosure to the disclosed example embodiments. On the contrary, example embodiments cover all modifications, equivalents, and alternatives falling within the scope of example embodiments. Like numbers refer to like elements throughout the description of the figures.

Also, terms such as first, second, A, B, (a), (b), etc. may be used herein to describe components. Each of these terms is not used to define the nature, sequence or order of the corresponding component but is only used to distinguish the corresponding component from other components.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, singular forms are intended to include plural forms unless the context clearly dictates otherwise. It should also be understood that when the terms "comprises" and/or "comprises" are used herein, it indicates the presence of stated features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, assemblies and/or combinations thereof

Unless specifically stated otherwise, or clear from the discussion, terms such as "process" or "calculate" or "determine" or "display" refer to the actions and processing of a computing system or similar electronic computing device that or a similar electronic computing device that manipulates and converts data expressed as physical quantities and electrical quantities within the computer system's registers and memories into computer system memory or registers or other such information storage devices, transmission devices, or display devices that are similarly represented as Other data of physical quantities.

It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.

Various example embodiments will now be described more fully with reference to the accompanying drawings in which some example embodiments are shown. In the drawings, the thicknesses of layers and regions may be exaggerated for clarity.

FIG. 1 is a diagram illustrating an overall operation of a face recognition system 100 according to at least one example embodiment. The face recognition system 100 may recognize a user's face from a two-dimensional (2D) input image for face recognition. The face recognition system 100 may extract and recognize a user's face appearing in a 2D input image by analyzing the 2D input image. The facial recognition system 100 can be used in various application areas such as security surveillance systems, mobile authentication, and multimedia data search.

The face recognition system 100 may register a three-dimensional (3D) face model of a user and perform face recognition using the registered 3D face model. The 3D face model may be a deformable 3D model deformable according to the user's facial pose or facial expression appearing in the 2D input image. For example, when a facial pose appearing in a 2D input image faces left, the face recognition system 100 may rotate the registered 3D face model to face left. In addition, the facial recognition system 100 may adjust the facial expression of the 3D facial model based on the user's facial expression appearing in the 2D input image. For example, the face recognition system 100 may analyze the user's facial expression based on facial feature points detected from the 2D input image, and adjust the shapes of the eyes, lips, and nose of the 3D face model so that the adjusted shapes correspond to the analyzed facial expression.

The face recognition system 100 may generate a 2D projection image from a registered 3D face model, and perform face recognition by comparing the 2D projection image with a 2D input image. Facial recognition can be performed in real time using 2D images. The 2D projected image refers to a 2D image obtained by projecting a 3D face model onto a plane. For example, the 2D projected image may be a 2D image obtained by projecting a 3D face model matched to the 2D input image at the same or similar viewpoint as that in the 2D input image, so that the pose of the face appearing in the 2D projected image May be the same or similar to the facial pose of the user appearing in the 2D input image. Facial recognition may be performed by matching a pre-stored 3D face model with facial poses that appear in the 2D input image and comparing the 2D projected image with the 2D input image. Although the user's facial pose appearing in the 2D input image is not facing forward, pose-responsive face recognition can be achieved by matching the 3D face model to the face pose appearing in the 2D input image and performing face recognition. Changed to improve the recognition rate.

Hereinafter, the operation of the face recognition system 100 will be described in detail. Face recognition performed by the face recognition system 100 may include a process 110 of registering a 3D face model of the user and a process 120 of recognizing the user's face from a 2D input image using the registered 3D face model.

Referring to FIG. 1 , at operation 130 of process 110 , the face recognition system 100 obtains a plurality of 2D face images of a user for face recognition. The 2D face image may include images of the user's face captured from various viewpoints. For example, the face recognition system 100 may obtain a 2D face image captured by a camera from the front and side of the user's face. The 2D face image may refer to an image including the user's face area, but may not necessarily include the entire area of the user's face. In operation 140 of process 110, the face recognition system 100 detects face feature points, eg, landmarks, from the 2D face image. For example, the facial recognition system 100 may detect feature points including eyebrows, eyes, nose, lips, chin, hair, ears, and/or facial contours from the user's 2D facial image.

In operation 150 of the process 110, the face recognition system 100 personalizes the 3D model by applying feature points extracted from the 2D face image for face recognition to a predetermined and/or selected 3D standard model. For example, the 3D standard model may be a deformable 3D shape model generated based on 3D face training data. A 3D standard model may include a 3D shape and a 3D texture, and parameters representing the 3D shape and the 3D texture. The face recognition system 100 may generate a 3D face model about a user's face by matching feature points of a 3D standard model with feature points extracted from a 2D face image. The generated 3D face model may be registered and stored as the user's 3D face model appearing in the 2D face image.

Alternatively, the face recognition system 100 may generate a 3D face model of the user using a 2D face image for face recognition, motion data of the 2D face image, and a 3D standard model. The face recognition system 100 may obtain a 2D face image and may obtain direction data of the 2D face image through a motion sensor, and generate 3D data about the user's face based on the direction data of the 2D face image and matching information. The 3D data about the user's face may be a collection of 3D points configuring the shape of the user's face. The face recognition system 100 may generate a 3D face model of the user by matching 3D data about the user's face with a 3D standard model. The generated 3D face model may be stored and registered as the user's 3D face model appearing in the 2D face image.

In process 120, the face recognition system 100 obtains a 2D input image including a user's face area through a camera. Although the face recognition system 100 may perform face recognition using a single 2D input image, example embodiments may not be limited thereto. At operation 160 of process 120, the face recognition system 100 adjusts the user's pre-stored 3D face model based on the facial pose or facial expression present in the 2D input image. The facial recognition system 100 may adjust the pose of the 3D face model to match the pose of the face appearing in the 2D input image, and adjust the expression of the 3D face model to match the facial expression appearing in the 2D input image.

The face recognition system 100 generates a 2D projection image from a 3D face model matched to a 2D input image for face recognition. In operation 170 of the process 120, the face recognition system 100 performs face recognition by comparing the 2D input image with the 2D projected image and outputs a result of the face recognition. For example, the face recognition system 100 may determine the degree of similarity between the face region in the 2D input image and the face region in the 2D projected image, and identify the face when the degree of similarity satisfies predetermined and/or desired conditions. The result of the recognition is output as "face recognition success", and in other cases "face recognition failure" is output.

The face recognition system 100 may include a 3D face model generation device (for example, the 3D face model generation device 200 of FIG. 2 , the 3D face model generation device 1000 of FIG. 10 ) and a face recognition device (for example, the face any one of the part recognition devices 300). The process 110 of registering a 3D face model of a user may be performed by the 3D face model generating apparatus 200 or the 3D face model generating apparatus 1000 . The process 120 of recognizing a user's face from a 2D input image may be performed by the face recognition device 300 .

FIG. 2 is a diagram illustrating a configuration of a 3D face model generating apparatus 200 according to at least one example embodiment. The 3D face model generating apparatus 200 may generate a 3D face model of the user's face from a plurality of 2D face images for face recognition. The 3D face model generating apparatus 200 may generate a 3D shape model and a 3D texture model as a 3D face model, and register the generated 3D shape model and the generated 3D texture model as the user's 3D face model. Referring to FIG. 2 , the 3D face model generating apparatus 200 includes an image acquirer 210 , a feature point detector 220 , a 3D face model generator 230 and a 3D face model register 260 . The image acquirer 210 , the feature point detector 220 , the 3D face model generator 230 and the 3D face model register 260 may be implemented using hardware components described below and/or hardware components running software components.

When at least one of the image acquirer 210, the feature point detector 220, the 3D face model generator 230, and the 3D face model register 260 is a hardware component running software, the hardware component is configured to run the The software in the (non-transitory computer readable medium) 270 is used to execute at least one of the functions of the image acquirer 210 , the feature point detector 220 , the 3D face model generator 230 and the 3D face model register 260 .

Although the memory 270 is shown as being external to the 3D face model generating apparatus 200 , the memory 270 may be included in the 3D face model generating apparatus 200 .

The image obtainer 210 obtains a 2D face image of the user for face recognition. The 2D facial image may include regions of the user's face including various facial poses. For example, the image obtainer 210 obtains 2D face images captured through a camera from a plurality of viewpoints, such as a front image or a side image. Information on the overall 2D shape of the user's face and texture information on the user's face may be extracted from the front image, and detailed information on the shape of the user's face may be extracted from the side image. For example, information about the 3D shape of the user's face may be determined by the 3D face model generating apparatus 200 by comparing the user's face area in the front image with the user's face area in the side image. According to example embodiments, the image acquirer 210 may capture a 2D face image through a camera to register a 3D face model, and the image acquirer 210 may store the 2D face image captured through the camera in the memory 270 .

The feature point detector 220 detects a face area and feature points or landmarks in the detected face area from the 2D face image. For example, the feature point detector 220 may detect feature points located on contours of eyebrows, eyes, nose, lips, and/or chin from a 2D face image. According to example embodiments, the feature point detector 220 may detect face feature points from a 2D face image using an active shape model (ASM), an active appearance model (AAM), or a supervised descent method (SDM).

The 3D face model generator 230 generates a 3D face model about the user's face based on the feature points detected from the 2D face image. A deformable 3D shape model and a deformable 3D texture model about the user's face may be generated as a 3D face model. The 3D face model generator 230 includes a 3D shape model generator 240 and a 3D texture model generator 250 .

The 3D shape model generator 240 generates a 3D shape model of the user's face using 2D face images captured from different viewpoints. A 3D shape model refers to a 3D model that has a shape without texture. The 3D shape model generator 240 generates a 3D shape model based on facial feature points detected from the 2D face image. The 3D shape model generator 240 determines parameters for mapping feature points detected from the 2D face image to feature points of the 3D standard model, and generates the 3D shape model by applying the determined parameters to the 3D standard model. For example, the 3D shape model generator 240 may generate a 3D representation of the user's face by matching feature points of eyebrows, eyes, nose, lips, and/or chin detected from a 2D face image with feature points of a 3D standard model. shape model.

Generating a 3D shape model using 2D facial images captured from different viewpoints may enable more detailed 3D shape models to be generated. In the case of generating a 3D shape model using only a front image obtained by capturing the user's face from the front, 3D shapes in the 3D shape model, such as the height of the nose and the shape of the cheekbones, may not be easily determined. However, in the case of generating a 3D shape model using a plurality of 2D face images captured from different viewpoints, a more detailed 3D shape model can be generated because the height of the nose and the shape of the cheekbones, for example, can be additionally considered. information.

The 3D texture model generator 250 generates a 3D texture model based on texture information extracted from at least one of the 2D face image and the 3D shape model. For example, the 3D texture model generator 250 may generate a 3D texture model by mapping texture extracted from the frontal image to a 3D shape model. A 3D textured model refers to a model having both the shape and the texture of a 3D model. A 3D texture model may have a higher level of detail than a 3D shape model and include vertices of the 3D shape model. 3D shape models and 3D texture models can have fixed-shape 3D models and deformable poses and expressions. The 3D shape model and the 3D texture model can have the same pose and expression. The 3D shape model and the 3D texture model can indicate the same or similar pose and expression with the same parameters.

The 3D face model registerer 260 registers and stores a 3D shape model and a 3D texture model as a 3D face model of the user. For example, when the user of the 2D face image obtained by the image acquirer 210 is "A", the 3D face model registerer 260 may register the 3D shape model and the 3D texture model generated for A as the 3D face model of A , the memory 270 can store the 3D shape model and the 3D texture model of A.

FIG. 3 is a diagram illustrating a configuration of a face recognition device 300 according to at least one example embodiment. The face recognition apparatus 300 may perform face recognition for a user appearing in a 2D input image for face recognition using the registered 3D face model. The face recognition apparatus 300 may generate a 2D projection image by rotating the 3D face model so that the 3D face model has the same or similar face pose as that of the user appearing in the 2D input image. The face recognition apparatus 300 may perform face recognition by comparing a 2D projected image with a 2D input image. The face recognition device 300 can provide a face recognition method robust to changes in a user's pose by matching a registered 3D face model with a face pose appearing in a 2D input image and performing face recognition. . Referring to FIG. 3 , the face recognition device 300 includes an image acquirer 310 , a 3D face model processor 320 and a face recognizer 350 . The 3D face model processor 320 includes a face recognition detector 330 and a feature point detector 340 .

Image acquirer 310, 3D face model processor 320 (including face region detector 330 and specific point detector 340) and face recognizer may be implemented using hardware components described below and/or hardware components running software components 350.

In the case where at least one of the image acquirer 310, the 3D face model processor 320 (including the face area detector 330 and the specific point detector 340) and the face recognizer 350 is a hardware component running software, the hardware component Configured to run software stored in memory (non-transitory computer readable medium) 370 to execute image acquirer 310, 3D face model processor 320 (including face region detector 330 and specific point detector 340) and the function of at least one of face recognizer 350 .

Although the memory 370 is shown as part of the face recognition device 300 , the memory 370 may be separate from the face recognition device 300 .

The image obtainer 310 obtains a 2D input image for recognizing a face including a user's face region. The image obtainer 310 obtains a 2D input image for identifying or authenticating a user through a camera or the like. Although the face recognition apparatus 300 may perform face recognition on a user using a single 2D input image, example embodiments are not limited thereto.

The face area detector 330 detects a user's face area from the 2D input image. The face region detector 330 recognizes a face region from the 2D input image using information on brightness distribution of the 2D input image, motion of an object, color distribution, eye positions, etc., and extracts location information of the face region. For example, the face region detector 330 detects a face region from a 2D input image using a Haar-based Adaboost cascade classifier commonly used in the prior art field.

The feature point detector 340 detects face feature points from a face area of a 2D face image. For example, the feature point detector 340 detects feature points including eyebrows, eyes, nose, lips, chin, hair, ears, and/or facial contours from the face area. According to example embodiments, the feature point detector 340 detects facial feature points from the 2D input image using ASM, AAM, or SDM.

The 3D face model processor 320 adjusts the pre-stored 3D face model based on the detected feature points. The 3D face model processor 320 matches the 3D face model with the 2D input image based on the detected feature points. Based on the results of the matching, the 3D facial model may be deformed to match the facial pose and expression present in the 2D input image. The 3D face model processor 320 adjusts the pose and expression of the 3D face model by mapping the feature points detected from the 2D input image to the 3D face model. The 3D face model may include a 3D shape model and a 3D texture model. The 3D shape model can be used to quickly match the facial poses present in the 2D input image, and the 3D texture model can be used to produce high-resolution 2D projection images.

The 3D face model processor 320 adjusts the pose of the 3D shape model based on the pose present in the 2D input image. The 3D face model processor 320 matches the pose of the 3D shape model with the pose appearing in the 2D input image by matching the feature points detected from the 2D input image with the feature points of the 3D shape model. The 3D face model processor 320 adjusts pose parameters and expression parameters of the 3D shape model based on the feature points detected from the 2D input image.

In addition, the 3D face model processor 320 adjusts the 3D texture model based on the parameter information of the 3D shape model. The 3D face model processor 320 applies pose parameters and expression parameters determined from the matching of the 3D shape model and the 2D input image to the 3D texture model. Based on the results of the application, the 3D texture model can be adjusted to have the same or similar pose and expression as that of the 3D shape model. After adjusting the 3D texture model, the 3D face model processor 320 may generate a 2D projection image by projecting the adjusted 3D texture model onto a plane.

The face recognizer 350 performs face recognition by comparing the 2D projected image with the 2D input image. The face recognizer 350 performs face recognition based on the degree of similarity between the face region appearing in the 2D input image and the face region appearing in the 2D projected image. The face recognizer 350 determines a degree of similarity between the 2D input image and the 2D projected image, and outputs a result of face recognition based on whether the determined degree of similarity satisfies a predetermined and/or desired condition.

The face recognizer 350 may determine a degree of similarity between the 2D input image and the 2D projected image using a feature value determination method generally used in the technical field of face recognition. For example, the face recognizer 350 may use feature extraction filters such as Gabor filters, Local Binary Patterns (LBP), Histograms of Oriented Gradients (HoG), Principal Component Analysis (PCA), and Linear Discriminant Analysis ( LDA)) to determine the similarity between the 2D input image and the 2D projected image. A Gabor filter refers to a filter used to extract features from an image using multiple filters with various sizes and angles. LBP refers to a filter for extracting the difference between a current pixel and adjacent pixels as a feature from an image. According to an example embodiment, the face recognizer 350 may divide the face area appearing in the 2D input image and the 2D projection image into cells of a predetermined and/or selected size and calculate a histogram associated with the LBP for each cell. , for example, on a histogram of the LBP index values included in the cell. The face recognizer 350 determines the vector obtained by linearly connecting the calculated histograms as the final feature value, and compares the final feature value of the 2D input image with the final feature value of the 2D projection image to determine the difference between the 2D input image and the 2D projection image. similarity between images.

According to an example embodiment, the face recognition device 300 also includes a display 360 . The display 360 displays a 2D input image, a 2D projected image, and/or a result of face recognition. In the event that the user determines that the user's face has not been properly captured based on the displayed 2D input image, or the display 360 displays the final result of the face recognition as a failure, the user can recapture the face and the face recognition device 300 can check the passed Re-capture the resulting 2D input image to re-perform face recognition.

FIG. 4 illustrates a process of detecting feature points from a 2D face image, according to at least one example embodiment. Referring to Fig. 4, the image 420 is a 2D face image obtained by capturing the user's face from the front by the 3D face model generating device, and the image 410 and the image 430 are generated by the 3D face model generating device (for example, 200 and 1000 ) is a 2D face image obtained by capturing the user's face from the side. Information about the overall 2D shape of the user's face and texture information of the user's face may be extracted from the image 420 by the 3D face model generating apparatus (eg, 200 and 1000 ). More detailed information about the shape of the face can be extracted from the image 410 and the image 430 . For example, a basic model about the user's face can be set based on the shape of the user's face extracted from the image 420, and the 3D face model generation device ( For example, 200 and 1000) to determine the 3D shape of the base model.

The feature point detector 220 of the 3D face model generating apparatus 200 of FIG. 2 may detect face feature points from 2D face images captured from a plurality of viewpoints, such as the image 410, the image 420, and the image 430. The facial feature points refer to feature points located in outline regions of eyebrows, eyes, nose, lips, chin, and the like. The feature point detector 220 may detect face feature points from the image 410, the image 420, and the image 430 using ASM, AAM, or SDM generally used in the related art field. Initialization of pose, scale or position of ASM, AAM or SDM may be performed based on the result of face detection.

An image 440 is an image obtained by detecting a feature point 444 within a face region 442 of the image 410 . An image 450 is an image obtained by detecting a feature point 454 within a face region 452 of the image 420 . Similarly, image 460 is a result image of feature points 464 detected within face region 462 of image 430 .

FIG. 5 illustrates a process of generating a 3D face model using a 3D standard model, according to at least one example embodiment. Referring to FIG. 5, a model 510 represents a 3D standard model. The 3D standard model, which is a deformable 3D shape model generated based on the 3D face training data, may be a parametric model representing characteristics of the user's face by an average shape and parameters.

As shown in Equation 1, the 3D standard model may include an average shape and a change amount of the shape. The amount of change in shape indicates a weighted sum of shape parameters and shape vectors.

[equation 1]

In Equation 1, An element representing a 3D shape that configures a 3D standard model, represents the elements associated with the average shape of the 3D standard model, denotes the shape element corresponding to the index factor "i", Indicates the shape parameter applied to the shape element corresponding to index factor i.

As shown in Equation 2, Can include coordinates of 3D points

In Equation 2, Represents an index indicating a 3D point (eg, and ), and "T" means "transpose".

The 3D shape model generator 240 of the 3D face model generating apparatus 200 of FIG. 2 may personalize a 3D standard model to register a user's face based on 2D face images captured from a plurality of viewpoints. The 3D shape model generator 240 may determine parameters for matching feature points included in the 3D standard model with face feature points detected from the 2D face image, and generate a 3D standard model by applying the determined parameters to the 3D standard model A 3D shape model of the user's face.

Referring to FIG. 5 , a model 520 and a model 530 are 3D shape models about a user's face generated from the model 510 which is a 3D standard model. Model 520 represents a 3D shape model viewed from the front, and model 530 represents a 3D shape model viewed from the side. The 3D shape model can have shape information without texture information, and can be used for matching with a 2D input image at high speed in the process of user authentication.

The 3D texture model generator 250 of FIG. 2 may generate a 3D texture model by mapping texture extracted from at least one 2D face image to a surface of a 3D shape model. For example, mapping a texture to the surface of a 3D shape model may mean adding depth information obtained from the 3D shape model to texture information extracted from a 2D face image captured from the front.

Model 540 and model 550 are 3D texture models generated based on the 3D shape model. Model 540 represents a 3D texture model viewed from the front, and model 550 represents a 3D texture model viewed from a diagonal direction. The 3D texture model may be a model including both shape information and texture information, and is used in the process of user authentication to generate a 2D projection image.

The 3D shape model and the 3D texture model are 3D models in which the shape of the face indicating the user's unique features is fixed and the posture or expression is deformable. A 3D texture model may have a higher level of detail and include a greater number of vertices than a 3D shape model. The vertices included in the 3D shape model may be a subset of the vertices included in the 3D texture pattern. The 3D shape model and the 3D texture model can indicate the same or similar pose and expression with the same parameters.

FIG. 6 illustrates a process of adjusting a 3D face model based on feature points detected from a 2D input image, according to at least one example embodiment. Referring to FIG. 6 , an image 610 is a 2D input image input to a face recognition device for face recognition or user authentication, and represents a face pose image captured by a camera.

The face region detector 330 of the face recognition device 300 of FIG. 3 can detect a face region from a 2D input image, and the feature point detector 340 of FIG. or feature points on the contour of the jaw. For example, the feature point detector 340 may detect facial feature points from a 2D input image using ASM, AAM, or SDM.

Image 620 is a resultant image obtained by detecting face area 630 from image 610 by face area detector 330 and detecting feature points 640 within face area 630 by feature point detector 340 .

The 3D face model processor 320 may match the pre-registered and stored 3D shape model with the 2D input image. The 3D face model processor 320 may adjust parameters of the 3D shape model based on the facial feature points detected from the 2D input image to adjust poses and expressions. The model 650 is a pre-registered and stored 3D shape model of the user's face, and the model 660 is a 3D shape model in which pose and expression are adjusted based on the feature points 660 detected from the image 610 . The 3D face model processor 320 may adjust the pose of the pre-stored 3D shape model to be the same as or similar to the face pose appearing in the 2D input image. In image 610, which is a 2D input image, the user's face presents a sideways posture, and the 3D shape model adjusted by the 3D face model processor 320 presents the same or similar sideways posture as the user's posture in image 610. attitude.

FIG. 7 illustrates a process of performing face recognition by comparing a 2D input image with a 2D projection image, according to at least one example embodiment. The 3D face model processor 320 of the face recognition apparatus 300 of FIG. 3 may adjust pose parameters and expression parameters of the 3D shape model based on face feature points detected from a 2D input image for face recognition. The 3D face model processor 320 may apply the adjusted pose parameters and the adjusted expression parameters of the 3D shape model to the 3D texture model to adjust the pose and expression of the 3D texture model to be the same as or similar to those of the 3D shape model. Then, the 3D face model processor 320 may generate a 2D projected image by projecting the 3D texture model to the image plane. The face recognizer 350 may perform face recognition based on the degree of similarity between the 2D input image and the 2D projected image, and output a result of the face recognition.

Referring to FIG. 7, an image 710 is a 2D input image for face recognition. The image 720 is a reference image to be compared with the image 710 as a 2D input image so that the face recognizer 350 performs face recognition. Region 730 included in image 720 represents a region reflecting a 2D projected image generated from the 3D texture model. For example, the image 730 is a face region obtained by projecting a texture model whose texture is mapped to the 3D shape model 660 of FIG. 6 to an image plane. The face recognizer 350 may perform face recognition by comparing the user's face area appearing in the 2D input image with the face area appearing in the 2D projected image. Alternatively, the face recognizer 350 may perform face recognition by comparing the entire area of the image 710 which is the 2D input image with the image 720 which is the result image obtained by reflecting the 2D projection image in the 2D input image .

FIG. 8 is a flowchart illustrating a 3D face model generation method according to at least one example embodiment.

Referring to FIG. 8 , in operation 810, the 3D face model generating apparatus obtains 2D face images of a user captured through cameras from different viewpoints. A 2D facial image can be used to register the user's face. For example, a 2D facial image may include an image including various facial poses, such as a frontal image and a profile image.

In operation 820, the 3D face model generating apparatus detects face feature points from the 2D face image. For example, the 3D face model generation device may detect facial feature points located on the contours of eyebrows, eyes, nose, lips, chin, etc. from a 2D input image using ASM, AAM, or SDM well known in the art.

In operation 830, the 3D face model generating apparatus generates a 3D shape model based on the detected feature points. The 3D face model generating apparatus may generate a 3D shape model by matching feature points of eyebrows, eyes, nose, lips, chin, etc. detected from a 2D face image with feature points of a 3D standard model. The 3D face model generation device determines parameters for mapping feature points detected from a 2D face image to feature points of a 3D standard model, and generates a 3D shape model by applying the determined parameters to the 3D standard model.

In operation 840, the 3D face model generating apparatus generates a 3D texture model based on the 3D shape model and texture information extracted from the 2D face image. The 3D face model generating apparatus may generate a 3D texture model on the user's face by mapping texture extracted from at least one 2D face image to a 3D shape model. The 3D texture model to which the parameters of the 3D shape model are applied may have the same or similar pose and expression as the 3D shape model.

In operation 850, the 3D face model generation device registers and stores the 3D shape model and the 3D texture model as the user's 3D face model. The stored 3D shape model and 3D texture model can be used in the user authentication process to authenticate the user appearing in the 2D input image.

FIG. 9 is a flowchart illustrating a face recognition method according to at least one example embodiment.

Referring to FIG. 9 , in operation 910, the face recognition apparatus detects face feature points from a 2D input image for face recognition. The face recognition device detects a face area from a 2D input image, and detects face feature points located on outlines of eyes, eyebrows, nose, chin, lips, etc. in the detected face area. For example, a face recognition device may use a Haar-based Adaboost cascade classifier to detect a face region from a 2D input image, and use ASM, AAM, or SDM to detect face feature points within the face region.

In operation 920, the face recognition device adjusts a pre-registered 3D face model of the user based on the feature points detected from the 2D input image. The face recognition device may match a pre-registered 3D face model with the 2D input image based on feature points detected from the 2D input image. The face recognition device may warp the 3D face model to match the facial pose and expression of the 3D face model to the facial pose and expression present in the 2D input image.

The 3D face model may include a 3D shape model and a 3D texture model. The face recognition apparatus may adjust the pose of the 3D shape model based on the feature points detected from the 2D input image, and adjust the 3D texture model based on parameter information of the pose-adjusted 3D shape model. The face recognition apparatus may adjust pose parameters and expression parameters of the 3D shape model based on feature points detected from the 2D input image, and may apply the adjusted parameters of the 3D shape model to the 3D texture model. Based on the results of the application of the parameters, the 3D texture model can be adjusted to have the same or similar pose and expression as that of the 3D shape model.

In operation 930, the face recognition device generates a 2D projection image from the 3D texture model. The face recognition device generates a 2D projected image by projecting the 3D texture model adjusted based on the 3D shape model in operation 920 to a plane. The facial pose appearing in the 2D projected image may be the same or similar to the facial pose appearing in the 2D input image. For example, when the user's facial posture appearing in the 2D input image is a side-facing posture, the 2D projection image generated through operation 910 to operation 930 may have the same or similar side-facing 3D texture model as the 2D input image. facial pose.

In operation 940, the face recognition device performs face recognition by comparing the 2D input image with the 2D projection image. The face recognition device performs face recognition based on a degree of similarity between a face region appearing in a 2D input image and a face region appearing in a 2D projected image. The face recognition device determines a degree of similarity between the 2D input image and the 2D projected image, and outputs a result of face recognition based on whether the determined degree of similarity satisfies a predetermined and/or desired condition. For example, if the similarity between the 2D input image and the 2D projected image satisfies predetermined and/or desired conditions, the face recognition device may output a result of "face recognition successful", and output "face recognition success" in other cases. Partial recognition failed" result.

FIG. 10 is a diagram illustrating another example of a configuration of a 3D face model generating apparatus 1000 according to at least one example embodiment. The 3D face model generating apparatus 1000 may generate a 3D face model of the user's face from a plurality of 2D face images for face registration. The 3D face model generating apparatus 1000 may generate a 3D face model of the user using 2D face images captured from different directions, motion data on the 2D face images, and a 3D standard model. Referring to FIG. 10 , a 3D face model generating apparatus 1000 includes an image acquirer 1010 , a motion sensing unit 1020 , a 3D face model generator 1030 and a 3D face model register 1040 .

The acquirer 1010, the motion sensing unit 1020, the 3D face model generator 1030, and the 3D face model register 1040 may be implemented using hardware components described below and/or hardware components running software components.

When at least one of the image acquirer 1010, motion sensing unit 1020, 3D face model generator 1030, and 3D face model register 1040 is a hardware component running software, the hardware component is configured to run the The software in the (non-transitory computer readable medium) 1070 is to perform at least one function of the image acquirer 1010 , the motion sensing unit 1020 , the 3D face model generator 1030 and the 3D face model register 1040 .

Although the memory 1070 is shown as being external to the 3D face model generating apparatus 1000 , the memory 1070 may be included in the 3D face model generating apparatus 1000 .

The image obtainer 1010 obtains 2D face images captured from different viewpoints for face registration. The image obtainer 1010 obtains 2D face images capturing the user's face from different directions through the camera. For example, the image obtainer 1010 may obtain 2D face images captured from different viewpoints, for example, a front image or a side image.

The motion sensing unit 1020 obtains direction data of the 2D face image. The motion sensing unit 1020 determines direction data of a 2D face image using motion data sensed through various sensors. The direction data of the 2D face images may include information on a direction in which each 2D face image is captured. For example, the motion sensing unit 1020 may determine orientation data for each 2D facial image using an inertial measurement unit (IMU), such as an accelerometer, gyroscope, and/or magnetometer.

For example, the user may capture the user's face by rotating the camera in different directions, and obtain 2D face images captured from various viewpoints as a captured result. During capturing of a 2D facial image, the motion sensing unit 1020 may calculate, based on the sensing signal output from the IMU, a function including, for example, a speed change, a direction change, a roll change, a pitch change, and a yaw change of the camera capturing the 2D facial image. motion data, and determine orientation data about the orientation in which the 2D facial image was captured.

The 3D face model generator 1030 generates a 3D face model of the user appearing in the 2D face image. The 3D face model generator 1030 detects facial feature points or landmarks from the 2D face image. For example, the 3D face model generator 1030 may detect feature points located on contours of eyebrows, eyes, nose, lips, chin, etc. from a 2D face image. The 3D face model generator 1030 determines information about matching points between 2D face images based on face feature points detected from the 2D face images.

The 3D face model generator 1030 generates 3D data of the user's face based on information on face feature points detected from the 2D face image, information on matching points, and direction data of the 2D face image. For example, the 3D face model generator 1030 may generate 3D data of the user's face using an existing stereo matching method. The 3D data of the user's face may be a collection of points configuring the shape or surface of the user's face.

The 3D face model generator 1030 converts the deformable 3D standard model into a 3D face model of the user using 3D data about the user's face. The 3D face model generator 1030 converts the 3D standard model into a 3D face model of the user by matching the 3D standard model with 3D data about the user's face. The 3D face model generator 1030 converts the 3D standard model into a 3D face model of the user by matching feature points of the 3D data with feature points of the 3D standard model. The user's 3D face model may include a 3D shape model associated with the shape of the user's face and/or a 3D texture model including texture information.

The 3D face model registerer 1040 registers and stores the 3D face model of the user generated by the 3D face model generator 1030 . The stored 3D face model of the user may be used to recognize the user's face, and the shape of the 3D face model may be converted in the process of face recognition.

FIG. 11 is a flowchart illustrating another 3D face model generation method according to at least one example embodiment.

Referring to FIG. 11 , in operation 1110, the 3D face model generating apparatus obtains a plurality of 2D face images and direction data of the 2D face images for face registration. The 3D face model generating device obtains 2D face images of the user captured by cameras from different viewpoints. The 3D face model generating device obtains 2D face images capturing the user's face from different directions, for example, a front image and a side image.

The 3D face model generating device obtains orientation data of a 2D face image using motion data sensed by a motion sensor. For example, the 3D face model generation device may obtain orientation data for each 2D face image using motion data sensed by an IMU including an accelerometer, a gyroscope, and/or a magnetometer. The direction data of the 2D face images may include information on a direction in which each 2D face image is captured.

In operation 1120, the 3D face model generating apparatus determines information about matching points between the 2D face images. The 3D face model generating device detects face feature points from the 2D face image, and detects matching points based on the detected feature points.

In operation 1130, the 3D face model generating apparatus generates 3D data about the user's face. For example, the 3D data of the user's face may be a collection of 3D points configuring the shape or surface of the user's face, and include a plurality of vertices. The 3D face model generation device generates 3D data of the user's face based on information on face feature points detected from the 2D face image, information on matching points, and direction data of the 2D face image. The 3D facial model generation device may use an existing stereo matching method to generate 3D data of the user's face.

In operation 1140, the 3D face model generating apparatus converts the 3D standard model into a 3D face model of the user using the 3D data generated in operation 1130. The 3D face model generating device converts the 3D standard model into the user's 3D face model by matching the 3D standard model with 3D data on the user's face. The 3D face model generation device generates a 3D face model of the user by matching the feature points of the 3D standard model with the feature points of the 3D data. The 3D face model generating device generates a 3D shape model and/or a 3D texture model as the user's 3D face model. The generated 3D face model of the user may be stored and registered, and used to recognize the user's face.

The units and/or modules described herein may be implemented using hardware components and/or hardware components executing software components. For example, hardware components may include microphones, amplifiers, bandpass filters, audio digitizers, and processing devices. The processing means may be implemented using one or more hardware devices configured to implement and/or execute program code by performing arithmetic, logic, and input/output operations. A processing device may include a processor, controller and arithmetic logic unit, digital signal processor, microcomputer, field programmable array, programmable logic unit, microprocessor, or any other device capable of responding to and executing instructions in a defined manner. The processing device may run an operating system (OS) and one or more software applications running on the OS. The processing device can also access, store, manipulate, process and create data in response to the execution of the software. For purposes of simplicity, a processing device is described in singular; however, one skilled in the art will appreciate that a processing device may include multiple processing elements and types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. Furthermore, different processing configurations are possible, such as parallel processors.

The software may include a computer program, a piece of code, instructions, or some combination thereof, to instruct and/or configure the processing means, individually or collectively, to operate as desired, thereby converting the processing means into a special-purpose processor. Software and data may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, computer storage medium or device, or in a transmission signal capable of providing instructions or data to or being interpreted by a processing device Wave implements software and data permanently or temporarily. The software can also be distributed over network-connected computer systems so that the software is stored and executed in a distributed fashion. Software and data can be stored by one or more non-transitory computer-readable recording media.

The methods according to the above-described example embodiments may be recorded in non-transitory computer-readable media including program instructions for implementing various operations of the above-described example embodiments. The media may also include individual program instructions, data files, data structures, etc., or a combination thereof. The program instructions recorded on the medium may be specially designed and constructed for the exemplary embodiments, or may be known and available to those skilled in the field of computer software. Examples of non-transitory computer-readable media include: magnetic media such as hard disks, floppy disks, magnetic tape; optical media such as CD-ROM discs, DVDs, and/or Blu-ray discs; magneto-optical media such as optical discs; and And hardware devices that execute program instructions, such as read only memory (ROM), random access memory (RAM), flash memory (eg, USB flash drive, memory card, memory stick, etc.), and the like. Examples of program instructions include both machine code, such as generated by a compiler, and files containing higher-level code executable by a computer using a compiler. The above-described apparatus may be configured to act as one or more software modules to perform the operations of the above-described example embodiments, or vice versa.

A number of example embodiments have been described above. However, it should be understood that various modifications may be made to these example embodiments. For example, if the described techniques are performed in a different order and/or if components in the described systems, architectures, devices, or circuits are combined in a different manner and/or are replaced or supplemented by other components or their equivalents, then the achieve the right result. Accordingly, other implementations are within the scope of the following claims.

Claims (33)

1. A face recognition method, comprising: Detect facial landmarks from a 2D input image; Adjusting the stored 3D face model based on the detected facial feature points; generating a 2D projection image from the adjusted 3D face model; Face recognition is performed based on the 2D input image and the 2D projected image. 2. The face recognition method according to claim 1, wherein the step of adjusting the stored 3D face model comprises: adjusting the stored 3D face model by mapping detected facial landmarks to the stored 3D face model Model's facial pose and facial expressions. 3. face recognition method as claimed in claim 1, wherein, the 3D face model of storage comprises 3D shape model and 3D texture model, The steps for adjusting the stored 3D face model include: Adjusting the stored 3D shape model based on facial landmarks detected from the 2D input image; The 3D texture model is adjusted based on the parameter information of the adjusted 3D shape model. 4. The face recognition method according to claim 3, wherein the step of adjusting the stored 3D shape model comprises: adjusting pose parameters and expression parameters of the 3D shape model based on the detected facial feature points. 5. The face recognition method according to claim 3, wherein the step of generating the 2D projection image comprises: generating the 2D projection image from the adjusted 3D texture model. 6. The face recognition method according to claim 1, wherein the 3D shape model and the 3D texture model are 3D models capable of deforming facial poses and facial expressions. 7. The face recognition method of claim 1, wherein the 2D projected image includes the same face pose as the face pose in the 2D input image. 8. The facial recognition method as claimed in claim 1, wherein the step of performing facial recognition comprises: Determine the degree of similarity between the 2D input image and the 2D projected image; The result of face recognition is output based on whether the degree of similarity satisfies the condition. 9. The facial recognition method as claimed in claim 1, wherein the step of detecting facial feature points comprises: Extract face regions from 2D input images; Facial feature points in at least one of eyebrows, eyes, nose, lips, chin, ears, and face contour are detected from the extracted face area. 10. The face recognition method of claim 1, wherein the performing comprises performing face recognition by comparing the 2D input image with the 2D projected image. 11. A method of generating a 3D face model comprising: Obtain a 2D facial image of the user from multiple viewpoints; Detect facial landmarks from 2D facial images; Generate a deformable 3D shape model and a deformable 3D texture model based on the detected facial feature points; The deformable 3D shape model and the deformable 3D texture model are stored as the user's 3D face model. 12. The method of claim 11 , wherein the step of generating comprises: generating a deformable 3D texture model based on the deformable 3D shape model and texture information from at least one of the 2D facial images . 13. The method of claim 11 , wherein the step of generating comprises: Determine the parameters used to map the detected face feature points to the feature points of the 3D standard model; A deformable 3D shape model is generated by applying determined parameters to a 3D standard model. 14. The method of claim 11, wherein the deformable 3D texture model comprises vertices of the deformable 3D shape model. 15. The method of claim 11, wherein the 2D facial image comprises images of the user's face from different viewpoints. 16. A method of generating a 3D facial model comprising: Obtaining a 2D face image and orientation data of the 2D face image, the 2D face image including the user's face; determining information about matching points between 2D facial images; generating 3D data of the user's face based on the orientation data of the 2D face image and information about the matching points; Use the 3D data to convert the 3D standard model into the user's 3D face model. 17. The method of claim 16, wherein the converting comprises converting the 3D standard model into the user's 3D face model by matching the 3D standard model with 3D data of the user's face. 18. The method of claim 16, wherein the 3D data of the user's face is a collection of 3D points configuring a shape of the user's face. 19. The method of claim 16, wherein the obtaining comprises obtaining the orientation data of the 2D facial image using motion data sensed by the motion sensor. 20. The method of claim 16, wherein the step of determining comprises: Detect facial landmarks from 2D facial images; Information on matching points is determined based on the detected facial feature points. 21. A facial recognition device comprising: an image obtainer configured to obtain a 2D input image including a user's face region; a 3D facial model processor configured to adjust the facial pose of the stored 3D facial model based on the user's facial pose as it appears in the 2D input image, and generate a 2D projection image from the adjusted 3D facial model; A face recognizer configured to perform face recognition based on the 2D input image and the 2D projected image. 22. The face recognition device as claimed in claim 21, wherein the 3D face model processor comprises: a face region detector configured to detect a face region from a 2D input image; A feature point detector configured to detect facial feature points from the detected face region. 23. The face recognition device as claimed in claim 22, wherein the 3D face model processor is configured to: detect facial feature points from a 2D input image, by combining the detected facial feature points with the stored 3D face The feature points of the model are matched to adjust the facial pose of the stored 3D face model. 24. The face recognition device as claimed in claim 21, wherein the stored 3D face model comprises a 3D shape model and a 3D texture model, The 3D face model processor is configured to: adjust the face pose of the 3D shape model based on the user's face pose appearing in the 2D input image, and adjust the 3D texture model based on parameter information of the adjusted 3D shape model. 25. The face recognition device of claim 24, wherein the 3D face model processor is configured to generate a 2D projection image by projecting the adjusted 3D texture model onto a plane. 26. The facial recognition device of claim 21 , further comprising: A display configured to display at least one of: one of the 2D input images, a 2D projected image, and a result of facial recognition. 27. The face recognition device of claim 21, wherein the face recognizer is configured to perform face recognition by comparing the 2D input image with the 2D projected image. 28. An apparatus for generating a 3D face model comprising: an image obtainer configured to obtain a two-dimensional 2D face image of the user from multiple viewpoints; a feature point detector configured to detect facial feature points from a 2D face image; A 3D face model generator configured to generate a deformable 3D shape model and a deformable 3D texture model based on the detected facial feature points; A 3D face model register configured to store the deformable 3D shape model and the deformable 3D texture model as the user's 3D face model. 29. The device according to claim 28, wherein the 3D facial model generator is configured to determine parameters for mapping detected facial feature points to feature points of a 3D standard model, and apply the determined parameters to to 3D standard models to generate deformable 3D shape models. 30. The device of claim 28, wherein the 3D face model generator comprises: a 3D shape model generator configured to generate a deformable 3D shape model of the user's face based on the detected facial feature points; A 3D texture model generator configured to generate a deformable 3D texture model based on the deformable 3D shape model and texture information from at least one of the 2D facial images. 31. An apparatus for generating a 3D face model comprising: an image obtainer configured to obtain a 2D face image of the user from multiple viewpoints; a motion sensing unit configured to obtain orientation data of a 2D face image; a 3D face model generator configured to generate 3D data of the user's face based on information about matching points between the 2D face images and direction data of the 2D face images, the 3D face model generator being configured to Use the 3D data to convert the 3D standard model into the user's 3D face model; A 3D face model register configured to store a 3D face model of the user. 32. The apparatus of claim 31, wherein the 3D face model generator is configured to detect face feature points from the 2D face image, and determine information on the matching points based on the detected face feature points. 33. The device of claim 30, wherein the 3D face model generator is configured to convert the 3D standard model into the user's 3D face model by matching the 3D standard model with the 3D face model of the user's face. face model.