WO2008056777A1 - Authentication system and authentication method - Google Patents

Abstract

Provided are an authentication system and an authentication method in which a plurality of 3D local regions are decided in a person face to be authenticated and 3D characteristic amounts of the face in the respective 3D local regions are calculated as the 3D face characteristic amounts. The 3D face characteristic amounts are compared to face characteristic amounts prepared in advance for comparison so as to authenticate the person. This reduces lowering the authentication accuracy and improves the authentication speed.

Description

 Specification

 Authentication system and authentication method

 Technical field

 [0001] The present invention relates to an authentication system and an authentication method for performing face authentication.

 Background art

 In recent years, networking of information devices has progressed, and various electronic services have become widespread. Along with this, the need for non-face-to-face non-face-to-face personal authentication (person authentication) is increasing. In this regard, research on biometrics authentication technology (biometric technology) that automatically identifies individuals based on the biometric characteristics of a person has been actively conducted. As one of biometric authentication technologies, face authentication technology that performs face authentication is known. This face authentication technology is a non-contact type authentication method, and the demand for it is very high in offices because of its convenience.

 [0003] By the way, in the face authentication described above, a change in shooting conditions greatly affects the authentication performance. This is because changes in shooting conditions often cause image fluctuations in the face image that exceed the identification information of the subject. The main causes that greatly affect the authentication performance are “posture change”, “light source change”, and “expression change”. Since the face has a three-dimensional shape (there is unevenness and depth), the face shape changes due to changes in the posture of the face and the illumination angle of the light source light on the face, or the facial expression changes. By changing, for example, a sharp hiding part of the nose, a partial hiding (shadow part, dark part) force S is generated on the face (this is called an occlusion). As a face authentication method, authentication (3D face authentication) power S using 3D shape information (3D information) can be mentioned. In general, this 3D face recognition uses the entire face, which causes the partial hiding problem. That is, the problem that the authentication accuracy is lowered due to the loss of data at the location where the concealment occurs in the authentication process cannot be solved. In addition, when dense 3D information is used, there is a problem that the authentication process takes time S.

In this regard, for example, Patent Document 1 discloses the following technique. First, the reference point of the face is extracted by examining the change in curvature of the face surface. The reference point of the face is These include points where the absolute value of curvature is maximum (for example, the tip of the nose) and points where the absolute value of curvature is maximum near the center of the side of the face (for example, ear hole points). Next, the face orientation (inclination), that is, the face orientation is corrected by calculating the reference posture based on these face reference points.

. Next, the corrected 3D shape data of the face is approximated to a plane with an arbitrary size, and the unit normal vector and area of this plane are obtained. Next, the normal distribution in which the size of the unit normal vector is expressed by this area is used as a feature amount, and authentication is performed.

[0005] However, the technique disclosed in Patent Document 1 is premised on using the entire three-dimensional shape, so-called global patch information. For this reason, since it is necessary to determine the reference direction of the face, it is not possible to determine this reference direction when the face is partially hidden due to the above-mentioned change in posture etc. The process cannot be executed.

 [0006] Further, for example, Patent Document 2 discloses the following technique. First, color information is used to extract 3D shape information and color information of only the face portion of a person, and face data is obtained by combining the 3D shape information and color information. Next, the center of the entire three-dimensional shape of this face data (collation face data) and dictionary face data prepared in advance is obtained, and translated so that the positions of these centroids coincide. At the same time, rotated face data is obtained by slightly rotating around the matched center of gravity. Then, a minimum error is obtained by calculating an error between the rotated face data and the dictionary face data, and determination (authentication) is performed based on the minimum error.

 [0007] However, with the technique disclosed in Patent Document 2, it takes time to obtain the minimum error by obtaining the center of gravity of the entire three-dimensional shape, and performing the translation and micro-rotation around the center of gravity. If dense 3D shape data is used, it takes a lot of processing time, and the authentication speed is slowed down. In addition, when the face is partially hidden (a tale version), the authentication accuracy is lowered. In addition, since the measured 3D shape itself must be retained, the storage capacity of the database becomes enormous. Patent Document 1: Japanese Patent Laid-Open No. 5-215531

 Patent Document 2: JP-A-9 259271

Disclosure of the invention [0008] The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide an authentication system and an authentication method capable of reducing a decrease in authentication accuracy and improving an authentication speed. Say it.

 [0009] In the authentication system and the authentication method according to the present invention, a plurality of local three-dimensional local regions in the person to be authenticated are determined, and the three-dimensional feature amount force of the face in each of these three-dimensional local regions The three-dimensional face feature value is calculated as a three-dimensional face feature value, and a comparison facial feature value prepared in advance is compared to perform an authentication operation for the person to be authenticated.

 [0010] In such an authentication system and authentication method, a plurality of local regions of the face to be extracted are extracted without using the information of the entire face as it is, and authentication is performed based on the extracted local regions. Is called. Therefore, even if partial hiding or the like occurs in the face, it is always possible to perform authentication using information in the local area excluding this portion without using the portion in which this hiding or the like has occurred. For this reason, a decrease in authentication accuracy is reduced, and the authentication speed can be improved.

 Brief Description of Drawings

 FIG. 1 is a schematic configuration diagram showing an example of an authentication system according to an embodiment of the present invention.

 FIG. 2 is a schematic diagram showing an example of the overall configuration of a controller in the authentication system.

 FIG. 3 is a functional block diagram for explaining a face authentication function provided in the controller.

FIG. 4 is a schematic diagram showing an example of coordinates of feature points in each feature part of a face.

 FIG. 5 is a schematic diagram for explaining calculation of three-dimensional coordinates of each characteristic part.

 FIG. 6 is a schematic diagram showing an example of a standard model.

 FIG. 7 is a three-dimensional graph for conceptually explaining the Gabor filter.

 FIG. 8 is a schematic diagram for explaining a method of setting a rectangular area from each feature point of 3D face part shape data.

FIG. 9 is a schematic diagram for explaining a method of extracting (determining) a local patch region from 3D face part shape data using the rectangular region information set in FIG. 8. FIG. 10 is a schematic diagram for explaining a method of setting a rectangular area from each feature point of 3D face part shape data.

 FIG. 11 is a schematic diagram showing an example of each 3D point and each local patch region in 3D face part shape data.

 [FIG. 12] (A) (B) and (C) are diagrams for explaining the intersection determination.

 FIG. 13 is a schematic diagram showing an example of a Bezier curved surface in extracting a three-dimensional face feature quantity.

 FIG. 14 is a flowchart showing an example of face authentication operation according to the present embodiment.

 FIG. 15 is a flowchart showing an example of the operation in step S9 of FIG.

 FIG. 16 is a functional block diagram for explaining the face authentication function provided in another controller.

FIG. 17 is a flowchart showing an example of the operation of the authentication system shown in FIG.

 BEST MODE FOR CARRYING OUT THE INVENTION

 Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted.

 FIG. 1 is a schematic configuration diagram showing an example of an authentication system 1 according to an embodiment of the present invention.

 FIG. 2 is a schematic diagram illustrating an example of the overall configuration of the controller 10. FIG. 3 is a functional block diagram for explaining the face authentication function provided in the controller 10. FIG. 4 is a schematic diagram showing an example of the coordinates of feature points in each feature part of the face.

 As shown in FIG. 1, the authentication system 1 performs personal authentication by face (hereinafter referred to as face authentication), and includes a controller 10 and two photographing cameras (two-dimensional camera; 2D camera) (hereinafter referred to as “face authentication”). Just “camera”! /, U) with CA1 and CA2! /

[0015] The cameras CA1 and CA2 are arranged so that the face of the authentication target person HM can be photographed from different positions (angles) with respect to the face position of the authentication target person HM. When the face image of the person HM to be authenticated is photographed by the cameras CA1 and CA2, the appearance information of the person HM to be authenticated HM obtained by this photographing, that is, two types of face images are transmitted to the controller 10 via the communication line. The image data communication method between the cameras CA1 and CA2 and the controller 10 is not limited to the wired method, and may be a wireless method. The face image may be an image including a background image as well as a face image. As shown in FIG. 2, the controller 10 is embodied by an information processing device such as a personal computer (PC), for example, and includes a CPU 2, a storage unit 3, a media drive 4, and a liquid crystal display, for example. Display unit 5, a keyboard 6a and an input unit 6 such as a mouse 6b as a pointing device, and a communication unit 7 such as a network card. The storage unit 3 includes a plurality of storage media such as a hard disk drive (HDD) 3a and a RAM (semiconductor memory) 3b. The media drive 4 also stores information recorded therein from a portable storage medium 8 such as a CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disk), flexible disk, or memory card. It is equipped with drive devices such as CD-ROM drive devices, DVD drive devices, flexible disk drive devices, and memory card drive devices that can be read. The information supplied to the controller 10 is not limited to being supplied via the recording medium 8, and may be supplied via a network such as a LAN (Local Area Network) or the Internet. . The controller 10 may be a dedicated controller (main unit control device) manufactured for this system. It has the functions described below!

 Further, as shown in FIG. 3, the controller 10 includes an image input unit 11, a face region detection unit 12, a face part detection unit 13, a face part 3D calculation unit 14, and a posture / light source correction unit 15. A standard model storage unit 16, a two-dimensional authentication unit 17, a face area 3D calculation unit 18, a three-dimensional authentication unit 19, a similarity calculation unit 20, a registered data storage unit 21, and a determination unit 22. It has.

[0018] The image input unit 11 inputs a face image of the person HM to be authenticated obtained by photographing with the cameras CA1 and CA2 from the cameras CA1 and CA2 to the controller 10. The image input unit 11 includes a first image input unit 11a and a second image input unit l ib corresponding to the cameras CA1 and CA2, and the face images transmitted from the cameras CA1 and CA2 respectively. Is entered. Therefore, a total of two face images are input from the cameras CA1 and CA2. Here, the authentication system 1 of this embodiment performs two-dimensional authentication (2D authentication) and three-dimensional authentication (3D authentication) using the input face image! / (If this is multiple-authenticated! / , U), and a configuration for making a determination based on these results. For this reason, in the authentication system 1 of this embodiment, a 2D image (2D image) and 3D shape data (3D shape data) are required. The The input device (2D image · 3D (3D) measurement input device) for acquiring the 2D image and 3D shape data is for multiple (2 to N) general 2D cameras (stereo cameras). There is a way. In this case, the 3D shape of the face (3D shape) is calculated from two or more 2D images.

 However, the present invention is not limited to this, and other 3D shape data acquisition methods can be adopted. For example, 3D shape data can be acquired by using a 3D measuring device (3D measuring device; 3D camera) such as a non-contact 3D digitizer using a light cutting method! /. In this embodiment, since two cameras CA1 and CA2 are used, it is necessary to calculate the 3D shape of the face from two 2D images (face images). However, the 3D such as the non-contact 3D digitizer described above is required. When using a measurement device (one camera and one 3D measurement device), 3D shape data can be acquired directly by the 3D measurement device, and there is no need to calculate it from a 2D image. Furthermore, in the type of 3D measurement device that combines a camera for acquiring 3D shape data and a camera for acquiring 2D images, it is not necessary to prepare a separate camera for acquiring 2D images as described above.

 The face area detection unit 12 detects (identifies and extracts) a face area from the face image input to the image input unit 11. The face area detection unit 12 includes a first face area detection unit 12a and a second face area detection unit 12b corresponding to the first image input unit 11a and the second image input unit l ib of the image input unit 11. Then, a face area (face area image) is detected from the face images transmitted from the first image input unit 11a and the second image input unit l ib, respectively. More specifically, the face area detection unit 12 extracts (cuts out) an area where a face image force face exists by performing template matching using a standard face image prepared in advance, for example. ) Perform processing.

 [0021] It should be noted that the method shown in the following 1. to 3. can be adopted as a method for detecting a face region, and other methods may be used.

1. [0022] 1. While scanning a window area (rectangular area) of a predetermined size with respect to a face image, it is determined whether or not an area representing a human face is included in the window area. A method that is performed by comparing a pixel value in the pixel and a predetermined threshold value (for example, Japanese Patent Application Laid-Open No. 2003-22441, Japanese Patent Application Laid-Open No. 8-339445). According to this method, a face detection algorithm that does not require motion information or color information can be used for high speed and high recognition from complex backgrounds. The face area can be detected based on the verification rate.

 [0023] 2. A so-called neural network that trains images of face regions of a plurality of people, stores the results as a learning dictionary, and compares the newly input face images with each other to determine face area detection. (For example, H. Rowley, S. Baluja, and T. anade. "New ral Networ K-Based Face Detection In IEE ^ Patt. Anal. Mach.intell ,. volume 20, pages 22-38, 1998. ).

 [0024] 3. A method using a detector proposed by Viola et al. (Viola-Jones detector), which stores various face area identifiers and uses them step by step, ie, comparison. The method of determining facial region detection while reducing the number of identifiers used as the system progresses (for example, P. Viola and M. Jones.Rapid object detection using a boosted cascade of simple features.In Proc. Of IEEE conference on Computer Vision and Pattern Recognition, Kauai, HI, December 2001.). Note that this method can be configured by combining a plurality of simple discriminant functions using simple image feature amounts.

 [0025] It should be noted that in the face area detection unit 12, the first face area detection unit 12a and the second face area detection unit 12b may individually detect the face area, but only one of them detects the face area. Also good. Alternatively, in the face area detection unit 12, the first face area detection unit 12a and the second face area detection unit 12b may individually detect the face areas, and may employ a highly accurate detection result. The face area can be detected with high accuracy by the corresponding area search process. The same applies to the face part detection unit 13.

The face part detection unit 13 detects (extracts or calculates) a characteristic part (referred to as a characteristic part) of the face with respect to the image power of the face area detected by the face area detection unit 12. Detecting a characteristic part of the face is called “face part detection”. Corresponding to the first face region detection unit 12a and the second face region detection unit 12b of the face region detection unit 12, the face part detection unit 13 includes a first face region detection unit 13a and a second face region detection unit 13b. The position of the characteristic part (coordinates on the image) is detected from the face area images transmitted from the first face area detecting unit 12a and the second face area detecting unit 12b, respectively. The facial features include the eyes (eg, the center of the pupil, the corner of the eye, the top of the eye, the upper and lower parts of the pupil), the eyebrows (eg, both ends and the middle of the eyebrows), and the nose (eg, the edges of the nose, the lower center of the nose, Hole), mouth (eg, left and right mouth edges, upper and lower lip center) or mandibular tip. In the present embodiment, for example, the face part detection unit 13 calculates the coordinates of the feature points Q1 to Q23 of each feature part as shown in FIG. Feature points Ql, Q3; Q2, Q4 are the ends of the left and right eyes, and feature points Q7, Q5; Q8, Q6 are the upper and lower parts of the left and right pupils, and feature points Q9, Q13; Q10, Q14 Are the left and right eyebrows, and feature points Ql l and Q12 are the approximate center of the left and right eyebrows, and feature points Q15 and Q16; Q17 and Q18 are the ends of the nose and feature points Q19 Is the lower center of the nose, the feature points Q20 and Q21 are both ends of the mouth, and the feature points Q22 and Q23 are the upper and lower portions of the center of the lips. It should be noted that the portion of the feature points to be extracted can be increased or decreased as necessary if it is set appropriately. Further, this feature part can be detected by various methods such as template matching using a standard template of the feature part.

 The coordinates of the calculated feature points Q1 to Q23 are expressed as two-dimensional coordinates on each image input from the cameras CA1 and CA2. For example, regarding the feature point Q20 corresponding to the right end of the mouth as viewed from the authentication target person HM, the coordinate value of the feature point Q20 is obtained in each of two images Gl and G2 (see FIG. 5 described later). More specifically, the coordinates (xl, yl) of the feature point Q20 on the image G1 are calculated with the end point of the images G1 and G2 as the origin O, and the coordinates of the feature point Q20 on the image G2 (x2 , Y2) is calculated.

 [0028] Further, the face part detection unit 13 calculates the coordinates of each feature point from the image of the face region, and calculates the luminance value of each pixel in the region having the feature point as a vertex (referred to as a feature region). This is acquired as information (called texture information) in this area. For example, in the case of the present embodiment, since two images are input, the face part detection unit 13 performs, for example, an average of corresponding pixels in corresponding feature regions in each of these images (images Gl and G2). The average brightness value of each pixel is used as texture information in the feature region.

[0029] It should be noted that the method of detecting the facial part is not limited to this! /. For example, as a face part detection method, a method as proposed in Japanese Patent Laid-Open No. 9 102043 “Detection of element positions in an image” may be employed. Further, for example, as a method for detecting a facial part, a method for detecting from the shape of a facial part by using auxiliary light, A method using learning by Rennet, or a method using frequency analysis by Gabor wavelet transform or normal wavelet transform which is not Gabor may be adopted.

[0030] The face part 3D calculation unit 14 calculates the three-dimensional coordinates of each feature part from the two-dimensional coordinates of the face feature part detected by the face part detection unit 13. More specifically, the face part 3D calculation unit 14 performs two-dimensional coordinates (2D coordinates) in each image Gi (i = l,..., N) of each feature point Qj detected by the face part detection unit 13. Based on Ui ( ^j ) and the camera parameters Pi (i = l, ..., N) of the camera that captured each image Gi, each feature part, i.e., each feature point Qj 3D coordinates (3D coordinates) M ( ^j ) (j = l, ..., M) f is calculated (so-called "three-dimensional reconstruction"). However, the symbol “N” indicates the number of cameras (N = 2 in this case), and the symbol “M” indicates the number of measurement points or feature points (M). Ff

The three-dimensional face data that combines the 3D coordinates M ^G) of each feature point Qj is called “3D face part shape data”.

[0031] A specific example of a method for calculating the 3D coordinate M ^(j) will be described below. The field coordinates (X, Υ, Ζ) ^{τ in} space and the coordinates (X, y) on the image have the relationship shown in the following formula (1 · 1).

 However, the symbol “w” in the equation (1 · 1) is a constant (w ≠ 0) that is not 0 (zero), and the symbol “P” represents the perspective projection matrix (camera parameter Pi).

[0033] As shown in the following equation (1 · 2), one-dimensionally more vectors are used to express the coordinates. This notation is called homogeneous coordinates. In homogeneous coordinates, a constant multiple other than 0 (zero) of the vector representing the coordinates, that is, (wx, wy, w) ^T ^ (x, y, 1 ^ etc.) above represents the same point. The homogeneous coordinates of a point are M = (X, Y, Z, 1) ^T , the homogeneous coordinates of the spatial point in the image are u = (x, y, 1) ^T, and `` = '' and `` ~ '' If the combination of the symbols is allowed to be a constant multiple, the left side is equal to 0 on the right side! /, Equal to the constant multiple! /, And! /, U The above expression (1 · 1) is expressed by the following expression (1 · 3).

u sPM (1-3) ここで、透視投影行列 Pは、 3 X 4の行歹 IJ式であり、その各成分を以下の（1·4)式に 示すものとすると、上記（1*1)式における「w」を消去することによって、空間と画像と の座標の関係は、以下の（1 · 5)及び（1 · 6)式に示すようになる。

u sPM (1-3) Here, the perspective projection matrix P is a 3 × 4 row IJ equation, and if each component is expressed by the following equation (1 · 4), the above (1 * 1 By eliminating “w” in equation (1), the relationship between the coordinates of the space and the image becomes as shown in the following equations (1 · 5) and (1 · 6).

• ·. (1.4)

PX ₊ P ₁₂ Y ₊ P ₁₃ Z ₊ Ρ „

 X 5)

+ i ₃₂ / + P ₃₃ Z + (1- P _M j JL + j. 24

 y = (1-6) Here, the cautionary point is that the constant multiple degree of freedom is allowed by the formula (1 · 3), so each component of Ρ is combined using each parameter. It is not independent.

 FIG. 5 is a schematic diagram for explaining the calculation of the three-dimensional coordinates of each feature part.

ただし、 (1-7)式中の記号「u」及び記号「M」は、それぞれ以下の（1 · 8)式に示すも のを表している。 ( For example, as shown in Fig. 5, two general cameras with different camera parameters (these are the first camera and the second camera), which are arranged at different free positions, are used. In this system, the world coordinates (X, Υ, Ζ) ^τ and the coordinates (xl, yl) on the images Gl and G2 of the first and second cameras corresponding to that point (world coordinate point) , (X2, y2) are expressed by the following formula (1.7) using the camera parameters P and P, respectively.

However, the symbols “u” and “M” in Equation (1-7) represent those shown in Equation (1 · 8) below. (

,

[0038] Therefore, when the perspective projection matrices P 1 and P 2 are known, the feature on the image

 1 2

 From the set of point coordinates (xl, yl), (x2, y2), the above formulas (1 · 7) and (1 · 8) are changed to w, w,

 1 2

By solving it as an equation of X, Υ, Ζ, the coordinates of the feature point in the space can be obtained, and 3D reconstruction can be performed by this. I.e., erasing w, w

 1 2

(1 · 5) and (1 · 6) are obtained by the expression, so the symbol ΓΡ ¹ ”represents the (i, j) component of Ρ

 ij 1

The symbol “P ² ” represents the (i, j) component of P.

 ¾ 2

 The following equation (1 · 9) is obtained. This equation (1 · 9) is a simultaneous linear equation of X, Y, and Ζ. By solving these equations, the coordinates (X, Υ, 特 徴) of the feature point in the three-dimensional space can be obtained. That power S. In (1 · 9), four equations are given for the three unknowns X, Υ, and Ζ. This means that the four components (xl, yl) and (x2, y2) are not independent. Similarly, the coordinates of the other feature points are calculated in space.

(1 · 9)

+ P -+-

+ P

Returning to FIG. 3, the posture / light source correction unit 15 performs posture variation correction and light source variation correction on the texture information calculated by the face part detection unit 13. Posture fluctuation correction corrects the effect on the texture due to the difference in face posture, that is, orientation (tilt). Light source fluctuation correction corrects the effect on the texture due to the difference in the direction (tilt) of the light source relative to the face. The posture / light source correction unit 15 is a standard model (standard stereo model; see Fig. 7 described later) that is a general (standard) face model prepared in advance for posture change correction and light source change correction for this texture information. ) Is used. [0040] <Attitude variation correction>

 (Shape information correction)

When correcting the posture variation with respect to the texture information, first, the shape of the 3D face part shape data (3D coordinates M ^{(j) of} each feature point Qj) is corrected. The posture 'light source correction unit 15 corrects the three-dimensional position so that the 3D face part shape data, that is, the 3D shape, most closely matches the 3D shape of the standard model (the shape of the 3D face part shape data itself changes). Shina! /,). In short, the posture / light source correction unit 15 performs so-called model fitting based on the standard model when the face according to the 3D face part shape data is facing sideways, and the face facing the side is the orientation of the standard model face ( Reference position), for example, correct the position so that it faces the front. This position is corrected based on the posture parameter t (pose parameter) shown in the following equation (2).

[0041] t = (s, φ, θ, φ, tx, ty, tz) ^Tt (2)

 However, the symbol “s” represents the scale conversion index, the symbol “φ, θ, φ” represents the transformation parameter indicating the rotational displacement (tilt), and the symbol “tx, ty, tz” is orthogonal 3 Represents a conversion parameter indicating translational displacement in the axial direction. The symbol “Tt” on the right shoulder represents “transposition”.

 [0042] (Texture information correction)

 Next, based on the position correction information obtained by correcting the face direction of the 3D face part shape data in the above direction, the two-dimensional texture (2D of each feature region acquired by the face part detection unit 13 is obtained. The texture information is corrected so that the (texture) is in the front direction (reference direction). As a result, the texture information corresponding to the case where the face was photographed from the front is reconstructed. In other words, a proper normalized texture image is created. By using the front textured face image reconstructed in this way, it becomes possible to handle V ヽ (independent dependence) texture information that is not affected by posture variations, that is, differences in shape.

[0043] The texture information correction is not limited to the above method! /. For example, the texture information correction is performed by pasting (mapping) the texture (texture image) of each feature region acquired by the face part detection unit 13 to the corresponding region (polygon described later) of the standard model. Similar to the above, there is a method of correcting so that a front textured face image can be obtained. It will be adopted. This makes it possible to handle texture information without being affected by the difference in posture! The front texture face image obtained by the correction may be projected onto cylindrical coordinates (cylindrical surface) arranged around the standard model so as to be easily compared with each other. The texture information of the projection image obtained by this projection is not affected by posture fluctuations, but is also not affected by changes in facial shape due to changes in facial expressions! /, Because it is pure facial texture information, It is very useful as information used for authentication.

 [0044] <Light source fluctuation correction>

 (Texture information correction)

 In the light source fluctuation correction for the texture information, for example, the brightness information of the texture is corrected. In this case, since the image captured by the camera generally includes the influence of shading due to the direction of the light source, the influence remains on the texture of each feature region in the input image. For this reason, the luminance is corrected for each feature region. More specifically, for example, the inside of the feature area is inclined so that the brightness of each pixel (node) in each feature area acquired by the face part detection unit 13 is equal to the brightness of the pixel corresponding to the standard model. In other words, the luminance is corrected by controlling the luminance value with the parameter of the tilt angle (orientation).

 [0045] The standard model storage unit 16 stores information on the standard model of the face in advance.

 FIG. 6 is a schematic diagram showing an example of a standard model. For example, as shown in FIG. 6, this standard model is composed of vertex data and polygon data. The vertex data is a set of coordinates of the vertex U of the feature part in the standard model, and has a one-to-one correspondence with the 3D coordinate of each feature point Qj. Polygon data is obtained by dividing the surface of a standard model into small polygons, for example, polygons such as triangles and quadrangles, and expressing these polygons as numerical data. Each polygon includes pixel luminance information and the like used in the light source fluctuation correction. The standard model may be average face data obtained by averaging the data of a plurality of people's faces. Further, the vertex of each polygon of the standard model may be configured by using an intermediate point other than the feature point Qj together with the feature point Qj. This midpoint is calculated by interpolation.

Returning to FIG. 3, the two-dimensional authentication unit (2D authentication unit) 17 is shown in the posture / light source correction unit 15. The 2D face feature value (2D face feature value: local 2D face feature value; local 2D face feature value) is calculated from the texture information of each feature region that has been corrected for the power fluctuation and the light source fluctuation. The 2D authentication unit 17 includes a corrected image acquisition unit 17a and a 2D feature quantity extraction unit 17b. The corrected image acquisition unit 17a acquires a corrected image (corrected texture image! /, U) obtained by the posture / light source correction unit 15 in which the texture image is subjected to posture change correction and light source change correction. That is, the corrected image from the attitude 'light source correction unit 15 is input to the corrected image acquisition unit 17a.

 [0047] The 2D feature amount extraction unit 17b extracts a 2D face feature amount from the corrected texture image acquired by the corrected image acquisition unit 17a. This 2D face feature extraction is performed by a method that uses Gabor wavelet transform, which is a technique that extracts local grayscale information (contour lines in a specific direction, etc.) as a feature. This Gabor wavelet transform can be used to detect the above-mentioned facial part, and can also be used to extract the grayscale information here. More specifically, the grayscale information obtained by applying the Gabor filter to the corrected texture image with the 2D coordinate point of the corrected texture image as a reference is extracted as a 2D face feature amount.

 FIG. 7 is a three-dimensional graph for conceptually explaining the Gabor filter. Here, as shown in FIG. 7, the Gabor filter is a spatial filter using a kernel in which a sin function (imaginary part) and a cos function (real part) are localized by a Gaussian function, and is a local filter of an image. It is a filter that performs transformation (Gabor wavelet transformation) that can extract information with high contrast.

 Since the filter processing by the Gabor filter is processing for local information, there is an advantage that it is not easily affected by fluctuations in illumination of the image. The Gabor wavelet transform creates a kernel with various periods by fixing the shape of the kernel, and expanding and contracting this kernel, and features corresponding to the spatial period (Gabor feature value; grayscale information here) This is a transformation that extracts.

[0049] The feature vector (two-dimensional feature vector; 2D feature vector) representing the feature quantity of the spatial period is an array of Gabor wavelet coefficients having different size and direction characteristics. The Gabor Wemblet transform is a function that minimizes the uncertainty of position and frequency, and is expressed by the following equation (3).

 The k vector in the above equation (3) is a constant that determines the wavelength and direction of the wave. The second term in [] is that the DC component of the function is 0 (zero) to satisfy the wavelet reconstruction condition, that is, It is a term added so that the following equation (4) can be obtained in the Fourier transform.

Ψ (0) = 0 (4)

[0051] When such a method using Gabor wavelet transform is applied to a face image, abundant feature information can be extracted by various directions and shading cycles, and therefore, it is adopted in a highly accurate face authentication system. The

 [0052] The 2D face feature amount can be calculated by convolving the Gabor filter shown in FIG. 7 with the corrected texture image. For example, directional force S, direction {0, π / 8, 2 π / 8, 3 π / 8, 4 π / 8, 5 π / 8, 6 π / 8, 7 π / 8, 8 π / 8} 40 (= 5 * 8 (2D face feature) is obtained by convolving multiple Gabor filters in 5 stages of 8 directions and Segenore force scale {4, 4 ^ 2, 8, 8 ^ 2, 16}. The symbol “*” indicates multiplication))) Dimensional feature vector (information on each shade period) can be obtained. Note that the extraction of 2D face feature values is not limited to this method using Gabor wavelet transform. Any method may be used for extracting the 2D face feature amount as long as the method uses other general texture information. Also, the direction and scale are not limited to 8 directions and 5 steps, and can be arbitrarily determined.

[0053] Returning to FIG. 3, the face area 3D calculation unit 18 obtains a high-density face from the image of the face area detected by the face area detection unit 12, that is, from the stereo image by the stereo camera in this embodiment. 3D shape (referred to as 3D close-fitting shape data) is calculated. However, the “high-density data” referred to here is data of only the face eyes and nose detected by the face part detection unit 13! /, And other feature parts (3D coordinates ^{特 徴 ΰ)} of feature points Qj ⁾ In other words, the data of the whole face including the amount of the face, etc., including the amount of data, only the data of this feature part, that is, “dense” with a large number of data acquisition points, for “rough (low density)” data with a small number of data acquisition points It is shown that the data is correct. The dense data acquisition points that make up the 3D dense face shape data This is called “3D point (3D point; or 3D measurement point)”. The 3D face shape data is face shape data composed of a plurality of 3D points.

 [0054] Calculation of a high-density 3D shape of a face from a stereo image is performed using, for example, a phase-only correlation method (POC: Phase-Only Correlation). The phase-only correlation method is one of the correlation calculation methods using Fourier transform. Two Fourier images are normalized for each spectrum and then synthesized. In other words, in the phase-only correlation method, when two images are given, the two-dimensional discrete Fourier transform of each image is normalized by the amplitude component, and a composite phase spectrum is obtained by calculating these products. Then, the inverse Fourier transform is performed on this. If the two images are similar, the POC function has a very sharp peak. The height of the correlation peak is useful as a measure of image similarity. The coordinates of the peak correspond to the relative misalignment of the two images. Since the phase-only correlation method has such characteristics, it is possible to obtain corresponding points between images with high precision under the influence of luminance fluctuation and noise. In other words, the phase-only correlation method is a process of searching for matching points between different images with high accuracy, that is, matching. In addition, highly accurate 3D close-fitting shape data is obtained by performing 3D reconstruction processing on the acquired corresponding points. As described above, since it is assumed that a plurality of 2D cameras are used in the present embodiment, when the force S and the 3D measurement device that calculates a high-density 3D shape by the phase-only correlation method are used. Since it is possible to obtain a high-density 3D shape without calculating from a plurality of images, it is not necessary to use such a method.

 [0055] Note that the corresponding point search using POC and multi-resolution includes, for example, the following method.

 First, a reduced image is created as a multi-resolution image. Secondly, in this reduced image, a corresponding point search is executed at the pixel level (pixel level). Thirdly, the reduced image is enlarged by a predetermined size by narrowing down the corresponding point candidates. Fourthly, in the reduced image enlarged by this predetermined size, corresponding point search is executed around the candidate at the pixel level. Fifth, the third and fourth are repeated until the same size as the original image before being reduced in the first one. Sixth, a sub-pixel level corresponding point search is performed with the same size as the original image.

[0056] The three-dimensional authentication unit (3D authentication unit) 19 is a 3D face dense shape calculated by the face region 3D calculation unit 18. 3D face feature values (local 3D face feature values; local 3D face feature values) are calculated based on the shape data and the 3D face part shape data calculated by the face part 3D calculation unit 14. The 3D authentication unit 19 includes a 3D local patch extraction unit (3D local patch extraction unit) 19a and a 3D feature quantity extraction unit (3D feature quantity extraction unit) 19b. The 3D local patch extraction unit 19a extracts (calculates) a 3D local patch region from the 3D face dense shape data and the 3D face part shape data (feature part). Hereinafter, the three-dimensional local patch region is simply referred to as “local patch region”.

 FIG. 8 is a schematic diagram for explaining a method of setting a rectangular area from each feature point of 3D face part shape data. FIG. 9 is a schematic diagram for explaining a method of extracting (determining) a local patch region from 3D face part shape data using the rectangular region information set in FIG. FIG. 10 is a schematic diagram for explaining a method of setting a rectangular area from each feature point of the 3D face part shape data. FIG. 11 is a schematic diagram showing an example of each 3D point and each local patch region in the 3D face part shape data.

[0058] The 3D coordinates M ( ^j ) (referred to as feature point coordinates) of each feature point Qj in each feature part of the 3D face part shape data exist on a high-density 3D shape (3D face part shape data). The local patch area is an area defined by a relative relationship from the feature point coordinates of the 3D face part shape data to the 3D face dense shape data. More specifically, for example, as shown in FIG. 8, on a plane T (local patch extraction plane) T defined by three points of the right eye a, right eye b, and right nose c, which are feature points Qj, For example, a rectangular region S defined by four points defined as a linear sum of vectors ca and cb is defined as a starboard region, for example. Then, as shown in FIG. 9, among a plurality of 3D points _α constituting the 3D face shape data, the 3D point _α force, and the vertical line that is dropped vertically perpendicular to the plane Τ (perpendicular line) A region formed by collecting 3D points in the rectangular region S (a region when viewed as an aggregate of the 3D points) is defined as a local notch region の of the starboard portion. The local patch region Ρ is, for example, a curved surface region (rectangular region S or local region) that substantially matches the actual face (right starboard) shape, such as the local patch region 場合 in the starboard case. It is not a complete planar shape defined by the patch extraction plane). FIG. 9 shows conceptual 3D points when 3D face shape data is looked down from above the face. Also, the local patch extraction plane is composed of 4 or more feature points. You may ask for it.

 In this way, as shown in FIG. 10, each local patch extraction plane is set from each feature point coordinate 201 of the 3D face part shape data, and a predetermined number is set on this local patch extraction plane. For example, a rectangular area 211 (portion part) and a rectangular area 212 (forehead part) are set. In addition to these, the rectangular area may be arbitrarily set to an area including facial features such as eyes, nose, mouth, and eyebrows as shown in rectangular areas 213, 214, and 215, for example. It is preferable that the facial feature part to be set is a part where the facial feature appears more prominently. Each rectangular area is set in this way, and as shown in FIG. 11, local patch areas 301, 302, 303,... Corresponding to these rectangular areas are determined. However, in FIG. 11, a plurality of points (plot points) 311 arranged on the entire face indicate each 3D point in the 3D close-fitting shape data), and in particular, the point indicated by reference numeral 312 (in the figure) The dark dots are the 3D points that make up the local patch area. The local patch region 302 corresponds to the local patch region P at the wrinkle point described in FIG.

 [0060] Since the face is symmetric, each extracted local patch region is preferably arranged at a symmetric position on the face. In addition, the eye area may be hidden by sunglasses, etc., and the mouth area may not be 3D-measurable due to the influence of wrinkles, etc., so the local patch area to be extracted is at least a part that is not easily hidden or 3D-measurable. It is desirable to include the nose and heel (the forehead is likely to be hidden by the hair).

[0061] The method of extracting the local patch region is not limited to this. For example, the local patch region can be extracted by preparing in advance a partial model shape (reference model shape) as a reference that the starboard portion has such a shape, and this partial model shape is a 3D face. A method may be used in which the force that is applied to the dense shape data is found, and the applied position is used as the local patch region of the starboard portion. More specifically, a reference three-dimensional (3D) patch shape (reference patch shape, reference partial model shape) corresponding to the local patch region to be extracted, that is, the average face (standard face) of the local patch itself, for example. The patch model obtained from the data is stored and saved in advance, and this patch model is compared with the 3D close-fitting shape data, for example, the similarity of each other's shape is compared, and the most A region having a shape similar (approximate) to the patch model shape is locally It may be a method of determining as a notch area.

 [0062] Further, for example, the local patch region extraction method may be a method of determining a region of 3D close-fitting shape data included in a region defined in advance on a two-dimensional image as a local patch region. More specifically, as shown in FIG. 10, a region that can be defined based on the feature point Qj detected by the face part detection unit 13 is defined as a selection region on the two-dimensional image, and this defined two-dimensional The 3D face shape data area of the selected area on the image is determined as the local patch area. In this method, the region on the 2D image is defined in advance by the calculation of the facial part 3D calculation unit 14, and all the 3D face shape data is measured. By searching for the corresponding points only in the upper region and performing 3D reconstruction, it is possible to measure the shape of only the local patch region, and to shorten the processing time.

[0063] Further, for example, the local patch region extraction method may be a method of determining the local patch region by performing an intersection determination with the shape of the standard model calculated from the average face. More specifically, first, a standard model is prepared in advance and extracted to this standard model! /, Local patch areas are defined, and these standard models and local patch areas on the standard model are stored and stored. . Next, the 3D position is corrected so that the 3D face part shape data best matches the 3D shape of the standard model. Next, after this position correction, a triangular patch that is a triangular area on the standard model is projected onto the triangular patch of the 3D face part shape data around the projection center point of the standard model. This triangular patch of 3D face part shape data is given as a patch composed of two points, the reference measurement point and the adjacent measurement point. Then, it is determined whether or not the projected triangular patch on the standard model and the triangular patch on the 3D face part shape data intersect, and if it intersects, the 3D face part shape is determined. A triangular patch on the data is determined as a local patch region. There are three cases of this intersection. If either case is satisfied, it is determined that the triangle patches intersect. FIG. 12 is a diagram for explaining the intersection determination. Figure 12 (A) shows the first case determined to be an intersection, Figure 12 (B) shows the second case determined to be an intersection, and Figure 12 (C) is determined to be an intersection. The third case is shown. In FIG. 12, the net pattern represents a standard model, and the hatched pattern represents measurement data. [0064] 1. When the triangular patch on the projected standard model is included in the triangular patch on the 3D face part shape data (FIG. 12 (A)).

 2. When the triangular patch on the projected standard model includes the triangular patch on the 3D face part shape data (Fig. 12 (B)).

 3. When the edge of the triangular patch on the projected standard model intersects the triangular patch on the 3D face part shape data (Fig. 12 (C)).

[0065] Further, for example, the local patch region extraction method may be a method of determining the local patch region by determining the shape and region of the standard model calculated from the average face. More specifically, first, a standard model is prepared in advance, and a map is created from the points of the standard model. That is, each three-dimensional point in the standard model point group prepared in advance is converted from the orthogonal coordinate system (x, y, z) to the polar coordinate system (spherical coordinate system) (r, θ, φ). Next, the spherical coordinates (Θ, φ) converted from the standard model point cloud are converted into uv plane coordinates by the conversion formula. Next, a map is created by labeling each local patch area to be extracted, which is composed of standard model point clouds. The conversion formula is (u, ν) ^Τ = (θ X (widt

x, yh—1) / (2πXwidth), (π / 2—φ) (height—πX height)) ¹ .

 , y, z

 Where Θ is the angle between OA and ζ axis at point A (r, θ, φ) on spherical coordinates,

 Φ is the intersection of the perpendicular line of point A (r, θ, φ) on the xy plane and the xy plane, z

 B is the angle between OB and the X axis. And width and height. This is the width of the projected image. Next, the point cloud of 3D face part shape data is processed in the same manner and projected onto the map image. Then, the region on the 3D face part shape data included in the labeled region is determined as the local patch region.

Returning to FIG. 3, the 3D feature quantity extraction unit 19b extracts a 3D face feature quantity from the information of the local patch region extracted by the 3D local patch extraction unit 19a. More specifically, a curved surface is calculated for each local patch based on information on a plurality of 3D points in each local patch region. The calculation of the curved surface is executed by a method using a curvature map, for example. In this case, first, normalization of the local patch area is performed. For example, in the case of a rectangular local patch area, the normalization is performed by performing three-dimensional affine transformation so that the vertex of the rectangular area becomes the vertex of a predetermined standard rectangular area (standard rectangular area). Is real Is done. In other words, a transformation (three-dimensional affine transformation) is performed in which the coordinate value indicating the 3D point of the local patch region is matched with the standard coordinate value. Then, the normalized local patch region is uniformly sampled, and the curvature at each sampling point is used as the shape feature (3D face feature amount) of the local patch region. It can be said that this method also uses the curvature map to compare the curvature of the local patch area and the standard rectangular area. For example, the curvature is “Face Identification Using 3D Curvature 3D Shape Feature Extraction 1”, IEICE Transactions Vol. J76-D2 No. 8 (August 1993) pp.1595- It can be calculated by using the method disclosed in 1603.

 [0067] The extraction of the 3D face feature value is not limited to the above method. For example, a method of extracting the 3D face feature value by surface approximation may be used. This curved surface approximation can use various curved surfaces such as a Bezier curved surface, a bicubic curved surface, a rational Bezier curved surface, a B-spline curved surface, and a NURBS (Non Uniform Rational B-Spline) curved surface. Here, the case where a Bezier curved surface is used will be described.

 FIG. 13 is a schematic diagram showing an example of a Bezier curved surface in the extraction of the three-dimensional face feature amount.

 As shown in FIG. 13, the Bezier curved surface is a curved surface F (Bézier curved surface F) defined by control points P arranged in a grid with P00, P01,. In this case, the control point P defines the four corner points and the rough shape of the curved surface F. A Bezier surface is a polynomial surface defined in the parameter region ue [0, l], ve [0, 1]. An n-order expression for u and an m-order expression for V is called an n X m-order surface, and is expressed by (n + 1) * (m + 1) control points. When n = m, it is called a bi-n-order surface. Such a Bezier surface is given by the following equation (5). ", Ν) ∑∑ρ ^ (Μ) Β ^ ν) · · · · (5)

[0069] A Bezier curved surface F shown in FIG. 13 represents a bicubic Bezier curved surface in the case of n = m = 3. By controlling this control point P (coordinate value thereof), the shape of the Bezier curved surface F changes, and the Bezier curved surface F is approximated to the shape of the local patch region. The shape information (curved surface information) of the approximated Bezier curved surface F is obtained as the patch shape information of the local patch region. Similarly, the patch shape information for each local patch area of the face, that is, 3 A dimensional feature vector (3D feature vector), that is, a 3D face feature value is obtained. Then, the total 3D face feature value is obtained by combining the patch shape information (3D face feature value) obtained for each local patch region. However, the present invention is not limited to this, and the information on the relative positional relationship between each local patch region (or each patch shape information), that is, the mutual distance, inclination, etc., is further added to the total 3D facial feature information. Information may be added. In this case, since it becomes possible to obtain “global shape information” indicating the overall characteristics of the face, the 3D face feature amount is more suitable for personal authentication.

 [0070] Note that the local patch region from which the 3D face feature value is extracted is preferably a three-local patch region including at least a part other than a facial feature part (the eyes, eyebrows, nose, mouth, etc.). In other words, the 3D face feature amount has no or little feature, that is, a part such as “forehead” or “頰”, which is a part where the feature is difficult to appear with the 2D feature quantity (feature part, 2D image) (surface irregularities) It is preferable that the patch is extracted from a local patch region including a flat portion with little change. As a result, when multiple verification with 2D authentication is performed, that is, at the time of authentication determination based on multiple similarity described later, feature quantities obtained from features (two-dimensional feature quantities) are of course used. In addition, it is possible to perform authentication with higher accuracy by using information on the feature amount (feature amount obtained three-dimensionally) of the part where the feature is difficult to appear.

 [0071] Since the 3D face feature quantity can be handled as a 3D feature vector (vector quantity) in this way, the calculated 3D face feature quantity (3D feature vector) or a comparison feature quantity prepared in advance (described later) When the 3D feature vector for comparison corresponding to the 3D feature vector of the 3D face feature amount (comparison vector amount) is registered (stored) in the storage unit 3 of the controller 10, for example, the above-mentioned 3D face dense shape data Compared to registering (coordinate information of each 3D point) itself, registering 3D feature vectors requires less registration data. In other words, data handling is improved, for example, memory capacity can be reduced.

[0072] The similarity calculation unit 20 compares the facial feature value of the comparison target person registered in advance (referred to as a comparison feature value) and the facial feature value of the authentication target person HM calculated above, that is, the 2D face feature value ( 2D feature vectors) and 3D face features (3D feature vectors) are evaluated for similarity. More specifically, the similarity calculation unit 20 performs similarity calculation based on the comparison feature quantity, the 2D face feature quantity, and the 3D face feature quantity, and each of the two-dimensional similarity degrees. (2D similarity; U and 3D similarity (3D similarity; Dij) are calculated, and multiple similarity is calculated using these 2D similarity and 3D similarity. First, calculation of 2D similarity explain.

 [0073] <Calculation of 2D similarity>

 As shown in the following equation (6), the 2D similarity L between the authentication target HM and the comparison target is calculated by the 2D feature quantity extraction unit 17b! It is given as the average value of the sum of the similarity SD (Ji, Ji ') of the F feature vectors extracted (generated).

 [0074] 1 F

 L (G, G ') ·' '(6)

 I = 0 However, in the above equation ½), the 2D similarity L is calculated by assuming that the feature quantity of the calculated feature vector is G and the registered feature quantity is G ′. ') Is expressed as. In addition, the symbol “i” in the above formula (i) represents the number of feature points, and i = l to F (pieces).

The similarity S (J, J ′) in the above equation (6) is defined by the following equation (7).

= sixS_D{j,J',d I . . . ( 7) ただし、上記（7)式中の記号「Ω」は、原点近傍 (0変位近傍）の局所領域を表す。ま た、記号 dベクトルは、位相差を表す。 D ii

= 6S _D {j, J ', d I... (7) However, the symbol “Ω” in the above equation (7) represents a local region near the origin (near zero displacement). The symbol d vector represents the phase difference.

[0076] In addition, S (J, J ', d vector) in the above equation (7) takes displacement correction into consideration for the phase similarity.

 D i i

 It is given by the following equation (8). This equation (8) has a form in which the correlation of amplitude is weighted by the similarity of the phase angle.

[0077] However, the symbol “J” in the above equation (8)

Yes, N is complex Gabor filter

is there. The symbol “a” represents the amplitude, and the symbol “φ” represents the phase. The k vector is jth A vector having the direction of the two-dimensional wave and the magnitude of the frequency, and is given by the following equation (9).

 (2π cos 9j,

 'k J.X, j (9)

 jy j

ノ

No

 [0078] When the Gabor filter is not used, the calculation of the 2D similarity can be performed by the Euclidean distance as in the 3D similarity calculation described later.

 [0079] <Calculation of 3D similarity>

 3D similarity between the HM subject to authentication and the comparison subject, that is, the similarity D of the shape feature

As shown by the following equation (10), the sum of the Euclidean distances between the vectors corresponding to each other (3D feature vector d ^S ) can be obtained.

.., d o)

[0080] <Calculation of multiple similarity>

 As shown in the following formula (11), the multiple similarity, which is the overall similarity between the HM (authentication object) and the comparison object (comparison object), is the 2D similarity and 3D similarity. It is calculated by the weighted sum for each degree of similarity. The multiple similarity is indicated by Re.

 [0081]

R e = WDi + {\ ˜W) S _D {J _i , J _i ^, ) (1 1) However, the symbol “W” in the above equation (11) represents a predetermined weighting coefficient. Here, this weighting factor W is given as a predetermined fixed value.

 The registration data storage unit 21 stores information on face feature amounts (comparison feature amounts and comparison face feature amounts) of a comparison target prepared in advance.

 The determination unit 22 performs authentication determination based on the multiple similarity Re. In the authentication judgment, the method differs between the case of face verification (Verification) and the case of face identification (Identification) as shown in (a) and (b) below.

[0084] (a) In face matching, the input face (input face; face of the person to be authenticated HM) is a specific registrant (special It is determined whether it is a regular registrant. In this face matching, the degree of similarity between the face feature amount of a specific registrant, that is, the comparison target person (comparison feature amount) and the face feature amount of the authentication target person HM is compared with a predetermined threshold value to compare with the authentication target person HM. Identity with the subject is determined. More specifically, when the multiple similarity Re is smaller than a predetermined threshold TH1, it is determined that the authentication target person HM is the same person (person) as the comparison target person. In this case, information on the threshold TH1 is stored in the determination unit 22. Alternatively, the information on the threshold TH 1 in this case is stored in the registration data storage unit 21! /.

 [0085] (b) Face identification is to determine who the input face belongs to. In this face identification, all the similarities between the face feature amount of the registered person (compared person) and the face feature amount of the person HM to be authenticated are calculated, and the person HM to be authenticated and each comparison object The identity of each person is determined. Then, the comparison target person having the highest identity among the plurality of comparison target persons is determined to be the same person as the authentication target person HM. More specifically, the comparison target person corresponding to the minimum multiple similarity Re (Remi n) among the multiple similarity degrees Re of the authentication target person HM and multiple comparison target persons is the authentication target person HM. It is determined that they are the same person.

FIG. 14 is a flowchart showing an example of the face authentication operation according to the present embodiment. First, the face image of the person HM to be authenticated is acquired by photographing with the cameras CA1 and CA2 (step Sl). Next, the two face images obtained by the photographing are input to the controller 10 (image input unit 11) (step S2). Next, the face area detection unit 12 detects a face area image from each face image input to the image input unit 11 (step S3). From the detected face area image, the face part detection unit 13 detects the facial feature part, that is, the coordinates of the feature point and the texture information of the feature area (step S4). Then, the face part 3D calculation unit 14 calculates the three-dimensional coordinates (3D face part shape data) of each feature part from the coordinates (feature point coordinates) of the feature part of the face detected by the face part detection unit 13. Calculated (step S5). Further, the posture / light source correction unit 15 performs posture variation correction and light source variation correction on the texture information detected by the face part detection unit 13 (step S6). The 2D authentication unit 17 calculates a 2D face feature amount from the corrected texture image of each feature region that has been corrected for the posture variation correction and the light source variation correction (step S7). On the other hand, the face area 3D calculation unit 18 calculates 3D face shape data composed of a plurality of 2D points from the face area image (stereo image) detected by the face area detection unit 12 (step). S8). Next, in the 3D authentication unit 19, the 3D local patch extraction unit 19a calculates the 3D face shape data calculated by the face region 3D calculation unit 18 and the face part 3D calculation unit 14 in step S5. A three-dimensional local patch region is calculated from the 3D face part shape data (step S9). Then, the 3D feature quantity extraction unit 19b calculates a 3D face feature quantity from the information of the local patch region calculated by the 3D local notch extraction unit 19a (step S10). Next, the similarity calculation unit 20 compares the face feature amount (comparison feature amount) of the comparison target registered in advance with the local 2D face feature amount and 3D face feature amount calculated in steps S7 and S10. The similarity is calculated based on the comparison feature quantity, the 2D face feature quantity, and the 3D face feature quantity, and the 2D similarity degree and 3D similarity degree. The multiple similarity is calculated from (Step Sl l). Then, based on the multiple similarity, the determination unit 22 performs face collation or face identification authentication determination (step S12).

 FIG. 15 is a flowchart showing an example of the operation in step S9 in FIG. In step S9, the local patch extraction unit 19a first extracts a local patch from each feature point (3D coordinate) (3D face part shape data) in each feature part calculated by the face part 3D calculation unit 14. The plane T for use is set (calculated) (step S21). Next, a rectangular area S (a partial area described later) is set on the set local patch extraction plane T (step S22). Then, a local patch area P corresponding to the rectangular area S is set, that is, a perpendicular line drawn perpendicularly to the local patch extraction plane の う ち out of a plurality of 3D points α constituting the 3D face part shape data is a rectangular area S. The 3D point α that falls within is identified, and the region that also has the identified 3D point α force is set as the local patch region Ρ (step S23).

[0089] As described above, according to the authentication system 1 according to the present embodiment, the entire face of the authentication target person's face is obtained by the three-dimensional shape acquisition unit (the face region detection unit 12, the face region 3D calculation unit 18). 3D shape information (overall 3D shape) is acquired, and the 3D shape information acquired by the 3D shape acquisition unit by the local region determination unit (3D local patch extraction unit 19a) is acquired. From (3D close-fitting shape data), a local region in the overall 3D shape A number of 3D local regions (3D local regions; local patch regions) are determined. In addition, the 3D feature amount calculation unit (3D feature amount extraction unit 19b) determines the shape of the 3D local region from the local 3D shape information (local 3D shape information) in the 3D local region determined by the local region determination unit. 3D face feature value, which is the local area shape information about the face and is a three-dimensional feature value of the face. Then, the 3D face feature amount calculated by the 3D feature amount calculation unit is prepared in advance by the feature amount comparison unit (similarity calculation unit 20 and determination unit 22) to perform the authentication operation for the authentication target person HM. The comparison face feature amount is compared.

[0090] Also, according to the authentication method of the present embodiment, in the first step, information on the entire 3D shape, which is the overall 3D shape of the face of the person to be authenticated, is acquired, and in the second step ! /, From the entire 3D shape information, a plurality of 3D local regions that are local regions in the entire 3D shape are determined. In the third step, from the local 3D shape information in the 3D local region, 3D face feature amount, which is local region shape information related to the shape of the 3D local region and is a three-dimensional feature amount of the face, is calculated. The Then, in the fourth step, the 3D face feature value is compared with the comparison face feature value prepared in advance to perform the authentication operation for the person to be authenticated HM.

 [0091] As described above, in the authentication system or the authentication method, a plurality of 3D local regions are determined from the entire 3D shape of the face of the person HM to be authenticated, and the 3D face feature amount is calculated from the local 3D shape information in the 3D local region. Then, the authentication operation for the person to be authenticated is performed by comparing the 3D face feature quantity with the comparison face feature quantity. Therefore, multiple 3D shape force local areas (3D local areas) of the entire face are extracted without using the information on the entire 3D shape of the face as it is, and based on these extracted 3D local areas! /, Since it is configured to perform authentication, even if partial hiding occurs in the face, it is not always necessary to use the part where the hiding occurs, and authentication is performed using information in a local area other than this part. Therefore, it is possible to reduce the decrease in authentication accuracy. In addition, since it is not necessary to handle the entire 3D shape (3D data) with a large amount of data as it is, that is, only partial 3D shape data in the local area needs to be used, the processing time is reduced and the authentication speed is reduced. Can be improved.

[0092] In the above authentication system, the 3D shape acquisition unit includes a 2D image acquisition unit (camera CA1, CA2) that acquires a 2D image of a face, and a feature region extraction unit (face region detection) The part 13) extracts a characteristic part that is a characteristic part of the face from the 2D image acquired by the two-dimensional image acquisition part. Then, the 3D coordinate calculation unit (facial part 3D calculation unit 14) calculates the 3D coordinates (Μ ^{ΰ) of} the feature part extracted by the feature part extraction unit, and the local region determination unit calculates the 3D coordinate. Based on the 3D coordinates of the feature part calculated by the calculation unit, a 3D local area is determined!

[0093] In the authentication method, the first step is a step including a fifth step of acquiring a 2D image of a face, and in the sixth step, a feature portion that is a characteristic portion of the face from the 2D image Is extracted. In the seventh step, the 3D coordinates of the characteristic part are calculated. Then, in the second step, a 3D local region is determined based on the 3D coordinates of the feature part.

As described above, in the authentication system or the authentication method, the characteristic part that is a characteristic part of the face is extracted from the 2D image, the 3D coordinate of the characteristic part is calculated, and the 3D local region is calculated based on the 3D coordinate. Therefore, when determining a 3D local area, it can be associated with the information of the two-dimensional feature part, and it is possible to perform high-accuracy authentication using the feature part information together with the 3D local area information. It becomes.

 [0095] In the authentication system, the local region determination unit sets a partial region (for example, a rectangular region S) having a predetermined shape in a plane (local patch extraction plane 平面) determined from the 3D coordinates, and The region corresponding to the partial region in the 3D shape is determined as the 3D local region. In this way, a partial area of a predetermined shape is set in the plane determined from the 3D coordinates of the feature part, and the area corresponding to this partial area in the overall 3D shape is determined as a 3D local area. The 3D local region can be easily determined from the 3D coordinates of the feature part.

[0096] Further, in the above authentication system, the entire 3D shape information is made into face shape data composed of a plurality of 3D points (α), and the local region determining unit determines the plane from the 3D points (<<). A region composed of 3D points where the perpendicular line drawn vertically to the region is included in the partial region is determined as the 3D local region (local patch region Ρ). In this way, the region composed of 3D points where the perpendicular line that is virtually perpendicularly dropped from the 3D point to the plane is included in the partial region is determined as the 3D local region. 3D local corresponding to partial area easily An area can be determined.

 [0097] In the authentication system, the local region determination unit compares the entire 3D shape with a reference three-dimensional partial model shape (reference 3D partial model shape; reference patch) prepared in advance. The part that is the most similar to the reference 3D partial model shape in the 3D shape is determined as the 3D local region. In this way, the overall 3D shape and the reference 3D partial model shape are compared, and the portion of the overall 3D shape that has the most similar shape to the reference 3D partial model shape is determined as the 3D local region. It is possible to easily determine a 3D local region in the entire 3D shape without acquiring a structure and operation for acquiring a feature part (2D face feature amount) from this 2D image.

 [0098] In the authentication system, the 3D feature amount calculation unit converts local 3D shape information in the 3D local region into predetermined curved surface information (for example, by a method using a Bezier curved surface). Is calculated as local region shape information. Thus, as the local region shape information of the 3D local region, information obtained by converting the local 3D shape information in the 3D local region into predetermined curved surface information is used, that is, the 3D shape information cannot be used as it is. Since this is converted to be treated as curved surface information (for example, curvature), dimensional compression is possible and the processing speed is increased.

 [0099] Further, in the authentication system, the 3D feature amount calculation unit calculates a 3D face feature amount including information on the relative positional relationship of each 3D local region as the 3D face feature amount. In this way, since the 3D face feature amount includes information on the relative positional relationship of each 3D local region, the entire face that is composed of only individual features in each 3D local region is determined by this 3D face feature amount. It is possible to represent the characteristics over a range of times (the global shape information of the face can be obtained), and more accurate authentication can be performed.

 [0100] Also, in the above authentication system, the 3D local area in the overall 3D shape is determined by the local area determining unit so that a plurality of 3D local areas are arranged at positions symmetrical with respect to the face. The In this way, since the 3D local area is placed in a symmetrical position on the face, the 3D local area (position) can be efficiently determined in the overall 3D shape, and the processing time is shortened. Data handling is improved.

[0101] Also, in the above authentication system, a plurality of 3D local regions are created by the local region determining unit. The 3D local region in the overall 3D shape is determined so that at least the nose and the heel region of the face are included. In this way, since the 3D local area in the overall 3D shape is determined so that at least the face nose and wrinkle are included, the 3D local area is measured by, for example, a part (for example, forehead) or a part that is hidden by hair. Difficult! /, It can be set avoiding parts (for example, mouth when having a mustache), and it is possible to calculate 3D facial features with high accuracy from this 3D local area. Can be performed.

 [0102] In the above authentication system, the two-dimensional feature amount of the face is obtained from the feature part information extracted by the feature part extraction unit by the two-dimensional feature quantity calculation unit (2D feature quantity extraction unit 17b). A 2D face feature is calculated. Then, the feature amount comparison unit combines the 2D face feature amount calculated by the 2D feature amount calculation unit and the 3D face feature amount calculated by the 3D feature amount calculation unit, for example, by a weighted sum. The feature quantity (multiple similarity) is compared with the comparison face feature quantity.

 [0103] In addition, in the eighth step, the 2D face feature value, which is a two-dimensional feature value of the face, is calculated from the information on the feature part. In step 4, the total face feature value that is a combination of the 2D face feature value and the 3D face feature value is compared with the comparison face feature value.

 [0104] As described above, in the authentication system or the authentication method, the 2D face feature amount of the face is calculated, and the comprehensive face feature amount combining the 2D face feature amount and the 3D face feature amount is compared with the comparison face. Since the feature quantities are compared, it is possible to perform more accurate authentication using the 2D face feature quantities and the 3D face feature quantities.

 [0105] Further, in the authentication system, the 3D feature quantity calculation unit calculates the 3D face feature quantity from the local 3D shape information in the 3D local region including at least a part other than the facial feature part. In this way, 3D face feature quantities are calculated from local 3D shape information in a 3D local area that includes at least parts other than facial feature parts. Therefore, authentication using multiple 2D face feature quantities and 3D face feature quantities (multiple authentication) ), Features of parts other than feature parts that are difficult to extract as 2D face feature quantities can be included as 3D face feature quantities, that is, feature quantities that cannot be covered by 2D face feature quantities. It can be covered with facial features, and as a result, more accurate authentication can be performed.

[0106] Also, in the above authentication system, information on feature parts for calculating 2D face feature values Is texture information, and the correction unit (posture / light source correction unit 15) performs posture fluctuation correction that is correction related to the posture of the face and light source fluctuation correction that is correction related to the direction of the light source relative to the face. Done. According to this, the posture variation correction, which is correction related to the posture of the face, and the correction of the direction of the light source with respect to the face are performed on the texture information of the feature part for calculating the 2D face feature amount. Based on the corrected texture information that has been corrected and corrected for light source fluctuations, it is possible to obtain appropriate 2D facial feature values, which in turn enables more accurate authentication.

[0107] In the authentication system, a 2D image of the face is captured by at least two imaging devices (cameras CA1 and CA2) in the 3D shape acquisition unit, and each of the imaging is performed by the 3D shape calculation unit. The 2D image force obtained from the device is calculated by performing 3D reconstruction on the corresponding points obtained by the computation process using the phase-only correlation method. According to this, since the entire 3D shape is calculated from the two 2D images obtained from at least two imaging devices by calculation using the phase-limited correlation method, the cost can be reduced without using an expensive 3D imaging device or the like. The power S can be used to calculate the entire 3D shape with high accuracy using the phase-only correlation method.

 [0108] In the authentication system, the 3D face feature amount calculated by the three-dimensional feature amount calculation unit is set as a vector amount (3D feature vector), and the storage unit (storage unit 3) sets the vector nore amount. The comparison vector feature (comparison 3D feature vector) is stored as the corresponding comparison facial feature (comparison feature), that is, the data stored as the comparison facial feature is measured. Since the amount of data is not so-called dense 3D shape data (3D face shape data), the amount of data to be stored can be reduced (the memory capacity can be reduced), and the data can be handled. It becomes easy.

 In the authentication system and the authentication method in the above-described embodiment, multiple similarity is calculated based on the 2D face feature amount and the 3D face feature amount, and face matching or face identification is performed based on the multiple similarity degree. However, the similarity is calculated based on the local area shape information and the global area shape information, and the face matching or face identification authentication determination is performed based on the similarity. It may be configured.

FIG. 16 is a functional block diagram for explaining the face authentication function provided in another controller. It is. FIG. 17 is a flowchart showing an example of the operation of the authentication system shown in FIG.

[0111] The authentication system of this embodiment is shown in FIGS. 1 to 3 in that it includes a controller 30 shown in FIG. 16 instead of the controller 10 in the authentication system 1 shown in FIG. Different from authentication system 1. Therefore, the description of the schematic configuration of the authentication system as shown in FIG. 1 and the overall configuration of the controller as shown in FIG. 2 will be omitted, and the functional blocks of the controller 30 will be described below.

 In FIG. 16, the controller 30 functionally includes an image input unit 31, a face region detection unit 32, a face region detection unit 33, a face region 3D calculation unit 34, and a face region 3D calculation unit 35. 3D local region extraction unit (3D local region extraction unit) 36, local region information calculation unit 38, global region information calculation unit 37, similarity calculation unit 39, registered data storage unit 40, and comprehensive judgment unit 41.

 [0113] These image input unit 31 (first and second image input units 31a and 31b), face region detection unit 32 (first and second face region detection units 32a and 32b), face part detection unit 33 (first And the second facial part detection unit 33a, 33b), the facial part 3D calculation unit 34, and the facial region 3D calculation unit 35 are the image input unit 11 (first and second image input units 11a, 1 lb) shown in FIG. , Face area detector 12 (first and second face area detectors 12a, 12b), face part detector 13 (first and second face part detectors 13a, 13b), face part 3D calculator 14 Since it is the same as that of the face area 3D calculation unit 18, the description thereof is omitted.

[0114] The 3D local region extraction unit 36 performs a three-dimensional analysis from the 3D face dense shape data calculated by the face region 3D calculation unit 35 and the 3D face region shape data (feature portion) calculated by the face region 3D calculation unit 34. A local region is extracted (calculated). That is, the 3D local region extraction unit 36 is the same as the 3D local patch extraction unit 19a of the 3D authentication unit 19 shown in FIG. 3, and the 3D face shape data and 3D face part shape data (feature part) 3 A dimensional local patch area is extracted (calculated). Therefore, as described above, this method of extracting a three-dimensional local patch region is performed by, for example, dropping a corresponding 3D close-fitting shape data region locally by dropping a perpendicular line to a partial region of a predetermined shape set in a plane. Extraction method as patch area, 3D close-fitting shape data area most similar to reference model shape is locally A method of extracting as a notch region, a method of determining a region of 3D facial shape data included in a predefined region on a two-dimensional image as a local patch region, and a shape of a standard model calculated from an average face Adopt the method of determining the local patch area by performing the intersection determination and the method of determining the local patch area by determining the shape and area of the standard model calculated from the average face. Can do.

[0115] The local region information calculation unit 37 also extracts (calculates) local region information from the information power of a single 3D local region (local patch region) extracted by the 3D local region extraction unit 36. In the present embodiment, the local region information calculation unit 37 uses the 3D local region (local patch region) alone information extracted by the 3D local region extraction unit 36, and the 3D unique feature amount in the facial feature part. (Local 3D facial feature) is extracted. As the local 3D facial feature amount extraction method, for example, an extraction method similar to the extraction method in the 3D feature amount extraction unit 19b of the three-dimensional authentication unit 19 shown in FIG. That is, the local 3D face feature amount extraction method is, for example, a method of extracting curvatures at a plurality of points in the curved surface of the local patch region as a local 3D face feature amount, and an approximation to the shape of the local patch region. Applying a method to extract the shape information (curved surface information) of the curved surface as a local 3D face feature, the force S is used.

 [0116] Also, for example, it is possible to employ a method of extracting the distance between the standard model and the local patch area after performing registration for each local patch area as a local 3D face feature amount using the standard model. . More specifically, first, a definition point h of a standard local model consisting of a plurality (N) defined in advance on the standard local region model used in the 3D local region extraction unit 36.

 The corresponding point S '(S' = (s, s, ..., s)) in the local patch region corresponding to H is obtained.

 1 2 Nh. Next, distances d (h, s) between a plurality of points H in the standard model and a plurality of points S ′ in the local patch region corresponding thereto are obtained. And these multiple distances d (h, s)

By calculating as a 3D feature vector, it can be extracted as a local 3D face feature.

[0117] The global area information calculation unit 38 also extracts (calculates) the global area shape information with respect to the information power of the three-dimensional local area (local patch area) extracted by the 3D local area extraction unit 36. In the present embodiment, the global region information calculation unit 38 is executed by the 3D local region extraction unit 36. From the extracted 3D local area (local patch area) information, 3D characteristic features (global 3D facial feature quantities) in the entire face are extracted (calculated).

 [0118] The global area shape information is a word for the local area shape information, and is a feature amount of the three-dimensional shape of the entire face of the person to be authenticated. The global area shape information is calculated based on, for example, the <1> face three-dimensional local area. Further, for example, the global area shape information is calculated based on <2> the shape of the entire face. Further, for example, the global area shape information is calculated based on <3> face three-dimensional feature points. Hereinafter, each case of <1> to <3> will be described more specifically.

 [0119] <1> Calculation method based on 3D local region of face

 Examples of the calculation method for calculating the global 3D facial feature quantity based on the local patch region of the face include the following <1-1> and <1-2>.

 [0120] <1-1>

 The global 3D face feature amount is calculated based on the center of gravity of the local patch region. That is, first, the centroid is calculated for each of a plurality (Nh) of local patch regions. Next, the plurality of calculated centroids S (S = (s, s, ... ', 3)) and the registered centroids

 1 2 Nh

 For T (T = (t, t, ... ', ΐ)), the distance d (t, s) between the corresponding centroids S and T is

1 2 Nh jj Calculated respectively. Then, the average of the calculated plurality of distances is obtained as the global 3D facial feature quantity di stb (Equation 12).

 [0121] <1-2>

 Further, the global 3D facial feature amount is calculated based on the normal of the local patch region. More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, RT registration makes it possible to more accurately align the local patch regions to be compared. In this way, the corresponding points are obtained by recalculating. A normal is obtained for each of the N corresponding points thus obtained.

 S

Therefore, the global 3D face feature distb is obtained. distb:

 N

[0122]>

 SRT fitting is a process that aligns the feature points in the measurement data with the feature points of the standard model. SRT fitting is a process of affine transformation of standard model data using Equations 14-1, 14-2 so that the distance energy between the feature points of the standard model and the feature points of the measurement data is minimized.

[0123] = ···-(14-1) Skin, C ∑ | M | ² ... (14.2) where P is the 3D point after conversion, and M is a 3 × 4 conversion Matrix and P is

 new old 3D point before conversion. M is the feature point of the standard model, and C is the measurement data.

 k k

 K is the number of feature points, and f (M, C) is the feature point of the standard model.

 k k

 It is the distance energy to the feature point of the measurement data.

 [0124] In SRT fitting, a transformation matrix that minimizes the distance energy is obtained by, for example, the least square method, and the position of the standard model data after the transformation is obtained. In addition, the projection center point of the standard model is also relocated.

 [0125]

 RT registration is the first measurement data T (T = {t I i≡

N} and the second measurement data t s consisting of multiple (N) points different from the number of the first measurement data M

 In S (S = {s I sEN}, the standard model consisting of N selected standard models

 s h

 Corresponding point cloud S '(S' = (s, s, ..., s)) and T '(T' = (t, t,

 1 2 Nh 1 2 Nh

)).

 [0126] The registration of two point groups is a rotation matrix RR and a translation vector Nort RT such that m = RRs + TT.

First, a covariance matrix B is calculated using the corresponding point group S ′, T ′. The covariance matrix B is given by Equation 15-1.

 The matrix A is given by Equation 15.2.

 [0128]

0 -s _? -M _T

s_ + m _∑ 0-<s' + m

 A

ここで、 s'は、 s' = (s '、s '、s ')であり、測定点の 3次元座標を表しており、 t'も同 様である。 _{^{- s'y; - ms +}} , 0 0 (15 · 2) s - m ', one _{^{m y' t s', -m}} ',, - s m,

Here, s 'is s' = (s ', s', s '), which represents the three-dimensional coordinates of the measurement point, and t' is the same.

 [0129] From the symmetric matrix of the covariance matrix B, eigenvalue decomposition is performed, for example, by the Jacobian method, and eigenvalues and eigenvectors are calculated.

 [0130] The smallest eigenvalue is obtained from the calculated eigenvalues (λ ≥ ··· ≥ λ ≥ ··· ≥ λ) j 1 j k

 And the eigenvector corresponding to this minimum eigenvalue is obtained, and the rows φ, φ,..., Φ] constituted by the obtained eigenvector are selected, and the rotation matrix

 1 2 k

 RR and translation vector RT are calculated.

 [0131] <2> Calculation method based on the shape of the entire face

 As a calculation method for calculating the global area shape information based on the entire face, for example, the following <2— ;! > To <2-6>.

 [0132] <2-1>

 The global 3D facial features are subjected to SRT fitting by using the local patch region extracted for the standard model, and are used as the deformation parameter S of the deformation S, movement T, and rotation R parameters of the SRT fitting. The

[0133] The deformation parameter S is a parameter for deforming the shape of the standard model so that the definition point on the standard model and the feature point on the local patch region are fitted to the shape of the local patch region. If the feature point used for SRT fitting matches the feature point of the face almost exactly, the deformation parameter S will be the same person's face size (width, height and depth). It is thought that it represents an individual because the travel, etc.) is unchanged. Also, deformation parameters

For S, for example, the same value is calculated even if shooting conditions such as magnification and exposure change. Further, the deformation parameter s may not use all the local patch regions. In other words, a plurality of local patch regions including the stably obtained nose may be collected and obtained by performing SRT fitting.

[0134] <2-2>

 For each local patch area, the global 3D facial feature value is a distance for each of a plurality of predefined points in the standard model and a plurality of measurement points on the local patch area respectively corresponding to the plurality of points of the standard model. It is given by finding the average of those distances.

[0135] More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, a plurality of measurements h on the local patch region corresponding to a plurality (N) of point groups H (H = (h, h,..., H)) defined in advance on the selected standard model. 1 2 Nh

 A fixed point group S ′ (S ′ = (s, s,..., S)) is acquired. Next, multiple standard models

 1 2 Nh

 The distances d (h, s) between the point group H and the plurality of point groups S in the corresponding local patch region are obtained. Then, an average value of the plurality of distances d (h, s) is obtained as the global 3D facial feature quantity di stb (see Equation 12). As long as the SRT fitting is used to make the alignment approximately correct, the corresponding point S 'is given to each subject.

[0136] <2-3>

 For each local patch area, the global 3D facial feature quantity is obtained by projecting a plurality of predefined points in the standard model onto the local patch area and processing the registered data in the same way as the projected points projected from the standard model. It is given by calculating the distance for each projected point and calculating the average of those distances.

[0137] More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, h 1 2 Nh is used to project a plurality of predefined point clouds H (H = (h, h, ···, h)) on the selected standard model onto the local patch region.

 Therefore, a plurality of projection point groups 3 ′ ′ = (3, s,...; ¾)) on the local patch region are acquired.

 1 2 Nh

The Next, multiple (N) registered data that has been processed in the same way for the selected standard model The distances d (t, s) between the point T and the corresponding projection points S ′ on the local patch region are obtained. Then, the average value of the plurality of distances d (t, s) is obtained as the global 3D face feature amount distb (see Equation 12). The corresponding point S ′ is given to each subject as long as they are aligned approximately by SRT fitting.

[0138] <2-4>

 The global 3D facial feature value is given for each local patch area by finding the average value of the distances between corresponding points in the local patch areas to be compared (measurement data and registration data).

 [0139] More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, the alignment between the local patch regions to be compared is performed with higher accuracy by the RT registration. Next, the N measurement point group S '(S, = (s, s, ..., s)) and the registered point group T' (cho, = (sh 1 2) used in this RT registration Nh 1

, S,..., S)), the distance between corresponding points is obtained. Next, this

2 Nh

 The average value of these multiple distances d (s, t) is obtained as the global 3D face feature distb (Equation 16)

distb― ~

 [0140] <2-5>

 The global 3D facial feature value is given by calculating an average value between points corresponding to each other in the local patch areas to be compared for each local patch area.

[0141] More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, the alignment between the local patch regions to be compared is performed with higher accuracy by the RT registration. N points after alignment

 Registration point cloud consisting of h, t = {t

 i I t ^ N} and a group of measurement points S t h s consisting of N points different from N

 Corresponding points with = {s I sEN} are recalculated. The distance from the point group T at each point s of the point group S is d (s, T) = mind (s, t) = d (s, m). Here, "mind (s, t)" is j l, ...

, Find the minimum value of d (s, t) in N. The point corresponding to s is m Then, the corresponding point cloud M of s becomes M = C (S, T) (C is a function for obtaining the nearest neighbor point). That is, in the point cloud S of the measurement data and the point cloud T of the measurement data, the nearest point from the point t of the point cloud T is obtained for the point s of the point cloud S, and the corresponding point cloud M is obtained. . Next, the distance d (s, m between the point m of the corresponding point cloud M and the point s of the corresponding point cloud S is obtained.

) Is required.

Calculated as b (Equation 17).

[0142] <2- 6>

 The global 3D facial feature value is given by obtaining a variance value between points corresponding to each other in the local patch areas to be compared for each local patch area.

[0143] More specifically, first, the standard model and the local patch region are aligned by SRT fitting. Next, the alignment between the local patch regions to be compared is performed with higher accuracy by the RT registration. Then, the registered point group T = {t I t ^ N} consisting of N points after alignment, and the measurement point group S consisting of N points different from N

= Corresponding points with {s I s EN} are recalculated. Then, the variance Sigma of each point s in the point group S and each point t in the point group T is obtained as the global 3D face feature distb (Equation 18).

Sigma

 [0144] In these calculation methods based on the shape of the whole face, the global patch shape information is compared as a dense data group in the local patch region and as a coarse data group in the other regions. It can be done.

 [0145] <3> Calculation method based on 3D feature points of face

The global 3D face feature is calculated based on the line (feature extraction line) defined in the local patch area. More specifically, a line (feature extraction line) is defined in a predetermined local patch region set in advance. The feature extraction line is, for example, 3D facial part shape data It is defined from multiple feature points. It is desirable that the feature extraction line be defined in the part where the 3D shape features of the face that enhance the authentication accuracy appear frequently. For example, the feature extraction line is a line including undulations of facial irregularities such as crossing the nose. Next, the feature extraction line defined in this local patch area is projected onto 3D facial shape data, and the 3D facial shape corresponding to multiple points on the feature extraction line defined in this local patch area. Multiple points on the data are obtained, and these point clouds are used as global 3D facial features. If there are no points on the 3D face shape data corresponding to the points on the feature extraction line defined in this local patch area, interpolation is performed from multiple points on the 3D face shape data. Sought by.

[0146] Note that the feature extraction line may be extended outside the local patch area from the local patch area that further increases the authentication accuracy. A feature extraction line is defined for each of a plurality of local patch regions that further improve the accuracy of authentication, and a plurality of point groups on the 3D face shape data are obtained from each feature extraction line. A group may be a global 3D facial feature. Further, the plurality of points on the feature extraction line may be equally spaced or equally spaced.

Similar to the similarity calculation unit 20 shown in FIG. 3, the similarity calculation unit 39 compares the feature quantity (comparison feature quantity) of the comparison target person registered in advance and the authentication target person HM calculated above. Similarity is evaluated by calculating similarity based on local region shape information and global region shape information. The similarity is calculated as local information similarity D ^sl and global information similarity D ^sb for local area shape information and global area shape information, respectively. d ^s ) can be obtained by calculating the total Euclidean distance between each other (see Equation 10).

The registration data storage unit 40 is prepared in advance in the same manner as the registration data storage unit 21 shown in FIG. 3 in order to calculate the local information similarity D ^sl and the global information similarity D ^sb by the similarity calculation unit 39. Information on the feature amount (comparison face feature amount) of the comparison target person is stored.

[0149] The overall judgment unit 41 assigns weights W and 1-W to the local information similarity D ^sl and the global area shape information D ^sb, and ^obtains the sum as multiple similarity Re (Re = WD ^sl + (1—

 ij

W) D ^sb ), authentication determination is performed based on this multiple similarity Re. The authentication decision is As described above, there are cases of face matching and face identification.

[0150] Note that the overall determination unit 41 first makes a determination based on the local information similarity D ^sl , and if the difference in the similarity is equal to or greater than a threshold in the determination result, determines that the other person Judgment may be made based on the global information similarity D ^sb only when the difference is less than the threshold value.

 [0151] In FIG. 17, in the face authentication using the global area shape information, first, a face image of the person HM to be authenticated is acquired by photographing with the cameras CA1 and CA2 (step S3 Do). The two face images obtained by the above are input to the controller 30 (image input unit 31) (step S32), and then from the face images input to the image input unit 31 by the face area detection unit 32. A face area image is detected (step S33), and from the detected face area image, the face part detection unit 33 detects the feature part of the face, that is, the coordinates of the feature point (step S34). A 3D coordinate (3D face part shape data) of each feature part is calculated by the 3D calculation part 34 from the coordinates of the feature part of the face (feature point coordinates) detected by the face part detection part 33 (step 3D). S35).

On the other hand, the face area 3D calculation unit 35 calculates 3D face shape data composed of a plurality of 2D points from the face area image (stereo image) detected by the face area detection unit 32 (step). S36). Next, the 3D facial dense shape data calculated by the facial region 3D calculation unit 35 by the 3D local region extraction unit 36 and the 3D facial part shape data calculated by the facial region 3D calculation unit 34 in step S35 described above. A three-dimensional local region (local patch region) is calculated from (step S37). Next, the local region information calculation unit 37 calculates the local region information, that is, the 3D face feature amount in this embodiment, from the information of the three-dimensional local region (local patch region) alone extracted by the 3D local region extraction unit 36. (Step 38). Next, the global region information calculation unit 38 calculates the information power of the 3D local region (local patch region) extracted by the 3D local region extraction unit 36, the global region shape information, and in this embodiment, the global 3D facial feature amount. (Step 39). Next, the similarity calculation unit 39 compares the feature quantity (comparison feature quantity) of the comparison target registered in advance with the local area shape information and the global area shape information calculated in steps S38 and S39. Similarity is evaluated (step S40). Then, based on the multiple similarity Re, the comprehensive determination unit 41 performs face collation or face identification authentication determination (step S41). [0153] In the case where the degree of coincidence of the shapes is compared for each local region by a method such as alignment, the shapes between the local regions are compared. Therefore, if the shape matching accuracy of the local area is high, the error becomes small even if the relative positional relationship between the local areas is greatly different, so that the error of other people also becomes small, resulting in a decrease in authentication accuracy. It will be. Therefore, when comparing the degree of matching of shapes by a method such as alignment using multiple local regions as one global region, in addition to the shape comparison for each local region, the relative position between the local regions Since related information is also included, the accuracy of authentication is expected to improve. Here, for example, the three-dimensional face authentication method based on the ICP algorithm, such as Japanese Patent Application Laid-Open No. 2007-164670, is an effective technique from this point. The force S and ICP algorithms are actually the processing time and characteristics. It was difficult in terms of quantification. In the above embodiment, the shape information of the global region of the face is divided into local regions and separated into the global region shape information and the local region shape information, so that the data amount and the processing time can be shortened. ing. Since global area shape information is also used, authentication can be performed with higher accuracy.

 [0154] The present embodiment can take the following aspects.

 [0155] (A) The area set on the local patch extraction plane T does not have to be rectangular like the rectangular area S. In short, it is a partial area (partial area) on the local patch extraction plane T. If so, the shape may be arbitrary. In addition, the shape of the feature portion may not be a rectangle but may be an arbitrary shape.

 [0156] (B) The method for determining the local patch area from the rectangular area S is as follows. The vertical line dropped perpendicularly to the local patch extraction plane T (the vertical leg force enters the rectangular area S 3D Various methods can be employed without being limited to the method of making a point, for example, it may be lowered at a predetermined angle with respect to the plane T without dropping from the 3D point perpendicular to the local patch extraction plane T. Alternatively, for example, a method may be used in which a virtual, for example, radial line is output from the rectangular area S in a predetermined direction, and a range on the 3D shape that intersects (contacts) the line is used as the local patch area.

(C) As shown in FIG. 1, the authentication system 1 does not have to be separated into the controller 10 and the cameras CA1 and CA2. For example, a configuration in which each camera is built directly in the controller 10 may be employed. However, in this case, each camera is built in such an arrangement that the subject person HM can be photographed at different angles! [0158] The present specification summarizes the main technologies, of which S discloses various technologies as described above.

 [0159] The authentication system that is one aspect is a local area determination unit that determines a plurality of three-dimensional local areas, which are local areas of the person to be authenticated, and the 3D determined by the local area determination unit. 3D feature quantity that is local area shape information related to the shape of each 3D local area, and that calculates the 3D facial feature quantity that is the 3D feature quantity of the face from the local 3D shape information in the local area Feature amount comparison for comparing the 3D face feature amount calculated by the calculation unit and the 3D feature amount calculation unit that performs the authentication operation for the person to be authenticated with a face feature amount for comparison prepared in advance. A part.

 [0160] In addition, in the authentication system that is powerful in another aspect, the authentication system further includes a three-dimensional shape acquisition unit that acquires information on the entire three-dimensional shape that is an overall three-dimensional shape of the face of the person to be authenticated. The local region determination unit determines a plurality of 3D local regions, which are local regions in the entire 3D shape, from the entire 3D shape information acquired by the 3D shape acquisition unit.

 [0161] According to the above configuration, the local region determination unit determines a plurality of three-dimensional local regions that are local regions of the person to be authenticated. For the plurality of 3D local regions, for example, the 3D shape acquisition unit acquires information on the entire 3D shape, which is the overall 3D shape of the face of the person to be authenticated, and the local region determination unit From the entire 3D shape information acquired by the 3D shape acquisition unit, the plurality of 3D local regions, which are local regions in the entire 3D shape, are determined. From the local 3D shape information in the 3D local area determined by the local area determination unit by the 3D feature quantity calculation unit, the local area shape information on the shape of each 3D local area is obtained. A 3D face feature value, which is a feature value, is calculated. Then, the feature quantity comparison unit compares the three-dimensional face feature quantity calculated by the three-dimensional feature quantity calculation unit with the comparison face feature quantity prepared in advance so as to perform the authentication operation for the person to be authenticated.

[0162] Therefore, according to this authentication system, local information in a plurality of 3D local regions selected from the entire 3D shape of the face of the person to be authenticated is not used as it is. Based on the 3D shape information! Even if partial occlusion occurs, it is not always necessary to use the part where this occlusion occurred. Authentication can be performed using information in other local areas excluding this part. Can be reduced. In addition, since it is not necessary to handle the entire 3D shape information with a large amount of data as it is, it is only necessary to handle partial 3D shape data in the local area, which shortens the processing time and improves the authentication speed. It becomes possible.

 [0163] In addition, in the authentication system that is effective in another aspect, the 3D shape acquisition unit includes a 2D image acquisition unit that acquires a 2D image of the face, and acquires the 2D image. The three-dimensional local region is determined based on a result of a feature part extraction unit that extracts a two-dimensional feature part that is a characteristic part of the face from the two-dimensional image acquired by the part.

 [0164] Also, in the authentication system that is effective in other aspects, the three-dimensional shape acquisition unit includes a three-dimensional coordinate calculation unit that calculates the three-dimensional coordinates of the feature part extracted by the feature part extraction unit. In addition, the local region determination unit determines the three-dimensional local region based on the three-dimensional coordinates of the feature portion calculated by the three-dimensional coordinate calculation unit.

 [0165] According to the above configuration, the 3D shape acquisition unit includes the 2D image acquisition unit that acquires the 2D image of the face, and the face is obtained from the 2D image acquired by the 2D image acquisition unit. A three-dimensional local region is determined based on the result of the feature part extraction unit that extracts a two-dimensional feature part that is a characteristic part of. For example, the feature part extraction unit extracts a feature part that is a characteristic part of the face from the two-dimensional image acquired by the two-dimensional image acquisition unit by the feature part extraction unit. The three-dimensional coordinates of the feature part extracted by the feature part extraction unit are calculated, and the three-dimensional local region is determined by the local region determination unit based on the three-dimensional coordinates of the feature part calculated by the three-dimensional coordinate calculation unit. It is done.

 [0166] Therefore, according to this authentication system, when determining a three-dimensional local region, it can be associated with information of a two-dimensional feature region, and the information of the feature region is used together with the information of the three-dimensional local region. High-precision authentication can be performed.

[0167] In addition, in the authentication system that can be applied to other aspects, the feature part extraction unit further includes a two-dimensional local region extraction unit that extracts a local region on a two-dimensional image from the extracted two-dimensional feature part. The 3D local region determining unit is calculated by the 2D local region extracting unit. The 3D local region is determined based on the generated 2D local region.

[0168] Also, in an authentication system that is powerful in another aspect, the local region determination unit calculates and extracts only a region corresponding to the two-dimensional local region as the three-dimensional local region.

[0169] In addition, in the authentication system that has power in another aspect, the local region determination unit sets a partial region of a predetermined shape in a plane determined from the three-dimensional coordinates, and in the overall three-dimensional shape. A region corresponding to the partial region is determined as the three-dimensional local region.

[0170] According to the above configuration, the local region determination unit sets a partial region of a predetermined shape in a plane determined from the three-dimensional coordinates, and the region corresponding to the partial region in the entire three-dimensional shape is three-dimensional. Determined as local region.

Therefore, according to this authentication system, a three-dimensional local region can be easily determined from the three-dimensional coordinates of a feature part using a simple method.

[0172] Further, in the authentication system that is effective in other aspects, the overall 3D shape information is face shape data composed of a plurality of 3D points, and the local region determination unit is configured so that the 3D points are Then, a region composed of three-dimensional points in which a perpendicular line that is virtually perpendicular to the plane is included in the partial region is determined as the three-dimensional local region.

[0173] According to the above configuration, the entire 3D shape information is the face shape data composed of a plurality of 3D points, and the local region determination unit descends the 3D points virtually perpendicular to the plane. A region composed of 3D points where the perpendicular line is included in the partial region is determined as a 3D local region.

 [0174] Therefore, according to this authentication system, the three-dimensional local region corresponding to the partial region can be easily determined using a simple method.

 [0175] Also, in the authentication system that is effective in other aspects, the local region determination unit compares the overall three-dimensional shape with a reference three-dimensional partial model shape prepared in advance, and A portion having a shape most similar to the reference three-dimensional partial model shape in the three-dimensional shape is determined as the three-dimensional local region.

[0176] According to the above configuration, the local region determination unit compares the entire three-dimensional shape with the reference three-dimensional partial model shape prepared in advance, and the reference three-dimensional partial model in the total three-dimensional shape. The partial force that is the shape most similar to the shape is determined as a ¾-dimensional local region [0177] Therefore, according to this authentication system, a configuration and operation for acquiring a two-dimensional image and extracting a feature part (two-dimensional facial feature amount) from the two-dimensional image are not required and easy. In addition, a 3D local region in the overall 3D shape can be determined.

 [0178] In addition, in the authentication system that works in another aspect, the local region determination unit includes the local region information defined on the entire three-dimensional shape and a reference three-dimensional partial model shape prepared in advance. A same space conversion unit for converting the same three-dimensional shape into the same space, and comparing the inclusion relationship between the entire three-dimensional shape and the reference three-dimensional partial model shape in the same space converted by the same space conversion unit. To determine the three-dimensional local region.

 [0179] In addition, in the authentication system that focuses on other aspects, the three-dimensional local region determination unit includes a three-dimensional surface on the reference three-dimensional model and a three-dimensional surface of the entire three-dimensional shape. The three-dimensional local region is determined by comparing the relationship.

 [0180] In addition, in the authentication system that focuses on other aspects, the 3D local region determination unit includes a 3D surface on the reference 3D model and a 3D coordinate point of the overall 3D shape. The three-dimensional local region is determined by comparing the inclusive relations.

 [0181] In addition, in the authentication system that focuses on other aspects, the 3D local region determination unit includes a 3D coordinate point on the reference 3D model and a 3D surface of the overall 3D shape. The three-dimensional local region is determined by comparing the inclusive relations.

 [0182] In addition, in the authentication system which is powerful in another aspect, the 3D local area determined by the local area determination unit is set as dense data, and the 3D local area determined to be other than the 3D local area is used. Keep the area as sparse data.

 [0183] With the above configuration, the local area determination unit converts the entire 3D shape and the local area information defined on the reference 3D partial model shape prepared in advance into the same space. The same space conversion unit compares the inclusion relation between the entire 3D shape and the reference 3D partial model shape in the converted same space, and according to the comparison result. A three-dimensional local region is determined. Therefore, according to this authentication system, the 3D local region in the entire 3D shape can be easily determined.

[0184] In addition, in the authentication system which has power in other aspects, the three-dimensional feature amount calculation unit includes the tertiary Original local region force Local 3D shape information is calculated as the local region shape information.

 [0185] Further, in the authentication system that is effective in other aspects, the three-dimensional feature amount calculation unit converts the local three-dimensional shape information in the three-dimensional local region into predetermined curved surface information, and the local region Calculate as shape information.

 [0186] In addition, in the authentication system that focuses on other aspects, the three-dimensional feature amount calculation unit uses local three-dimensional shape information in the three-dimensional local region as defined points defined on a standard model and a three-dimensional The local area shape information is calculated by converting the distance information of the corresponding points in the local area into an outer area.

 [0187] According to the above configuration, the three-dimensional feature amount calculation unit calculates local three-dimensional shape information from the three-dimensional local region as the local region shape information. For example, the 3D feature quantity calculation unit calculates the local area shape information obtained by converting the local 3D shape information in the 3D local area into predetermined curved surface information. Also, for example, the 3D feature value calculation unit converts the local 3D shape information in the 3D local region into the vector from the distance information between the defined points defined on the standard model and the corresponding points in the 3D local region. Calculated as local area shape information.

 Therefore, according to this authentication system, the 3D shape information is not used as it is. The 3D shape information calculated from the 3D local area, for example, the local 3D shape information is converted. Since the structure is handled as curved surface information (for example, curvature), dimensional compression is possible and the processing speed is increased.

 [0189] In addition, in the authentication system that can be applied to other aspects, the three-dimensional feature amount calculation unit includes, as the three-dimensional face feature amount, information on the positional relationship between three-dimensional local regions. Calculate the amount.

 [0190] According to the above configuration, the 3D feature amount calculation unit calculates a 3D face feature amount including information on the relative positional relationship of each three-dimensional local region as the 3D face feature amount.

[0191] Therefore, according to this authentication system, it is possible to represent the feature of the entire face by using only the individual features in each three-dimensional local area by using this three-dimensional facial feature amount (facial features). Global shape information can be obtained) and more accurate authentication can be performed. [0192] In addition, in the authentication system which is effective in another aspect, the local region determination unit is configured to form the entire three-dimensional shape so that the plurality of three-dimensional local regions are arranged at positions where the left and right sides of the face are symmetrical. Determine the 3D local region in the shape.

[0193] According to the above configuration, the local region determination unit determines the three-dimensional local region in the entire three-dimensional shape so that the plurality of three-dimensional local regions are arranged at positions that are symmetrical with respect to the face.

 [0194] Therefore, according to this authentication system, it is possible to efficiently determine the (position) of the three-dimensional local region in the entire three-dimensional shape, shorten the processing time, and handle the data. Will improve.

[0195] In addition, in the authentication system according to another aspect, the local region determination unit includes the entire three-dimensional shape so that the plurality of three-dimensional local regions include at least the nose and the heel portion of the face. The three-dimensional local region at is determined.

[0196] According to the above configuration, the three-dimensional local area in the entire three-dimensional shape is determined by the local area determination unit so that the plurality of three-dimensional local areas include at least the face nose and the heel region.

 [0197] Therefore, according to this authentication system, the three-dimensional local region is avoided, for example, by a part hidden by hair (for example, a forehead) or difficult to measure! /, Part (for example, a mouth when having a mustache). Since it can be set, it is possible to accurately calculate the 3D facial feature quantity from this 3D local area, and it is possible to perform highly accurate authentication.

 [0198] In addition, in an authentication system that is powerful in other aspects, a two-dimensional face feature value, which is a two-dimensional feature value of the face, is calculated from the feature part information extracted by the feature part extraction unit 2 A feature quantity calculation unit, wherein the feature quantity comparison unit includes a two-dimensional face feature value calculated by the two-dimensional feature value calculation unit and a three-dimensional face feature value calculated by the three-dimensional feature value calculation unit. Are compared with the comparative face feature value.

[0199] According to the above configuration, the two-dimensional feature quantity that is a two-dimensional feature quantity of the face is calculated by the two-dimensional feature quantity calculation unit from the feature part information extracted by the feature part extraction unit. Then, the 2D face feature calculated by the 2D feature amount calculation unit by the feature amount comparison unit The total facial feature quantity, which is a combination of the quantity and the 3D facial feature quantity calculated by the 3D feature quantity calculation unit, is compared with the comparison facial feature quantity.

[0200] Therefore, according to this authentication system, it is possible to perform more accurate authentication using the 2D face feature value and the 3D face feature value.

[0201] In addition, in the authentication system that is effective in other aspects, the three-dimensional feature amount calculation unit includes the three-dimensional shape information from local three-dimensional shape information in a three-dimensional local region including at least a part other than the facial feature part. A face feature amount is calculated.

[0202] According to the above configuration, the 3D feature quantity calculation unit calculates the 3D face feature quantity from the local 3D shape information in the 3D local region including at least a part other than the facial feature part.

 [0203] Therefore, according to this authentication system, it is difficult to extract features as 2D face feature quantities when performing authentication (multiple authentication) using 2D face feature quantities and 3D face feature quantities! /, Features of parts other than the feature part can be included as 3D facial feature quantities. For this reason, feature quantities that cannot be covered by two-dimensional face feature quantities can be covered by three-dimensional face feature quantities, and as a result, more accurate authentication can be performed.

 [0204] In addition, in the authentication system that is effective in other aspects, the feature part information for calculating the two-dimensional face feature amount is texture information, and the facial posture is related to the texture information. The image processing apparatus further includes a correction unit that performs posture variation correction that is correction and light source variation correction that is correction related to the direction of the light source relative to the face.

 [0205] According to the above configuration, the feature part information for calculating the two-dimensional face feature amount is texture information, and the correction unit corrects the posture of the face with respect to the texture information. The posture variation correction and the light source variation correction that are corrections related to the direction of the light source with respect to the face are performed.

 Therefore, according to this authentication system, it is possible to obtain an appropriate two-dimensional face feature amount based on the texture information subjected to the posture variation correction and the light source variation correction, and thus to perform more accurate authentication. Become.

[0207] In addition, in the authentication system that is effective in other aspects, the three-dimensional shape acquisition unit includes at least two photographing devices that photograph a two-dimensional image of the face, and two images obtained from the photographing devices. A three-dimensional shape calculation unit for calculating the whole three-dimensional shape by performing a high-precision corresponding point search process from the two-dimensional image by calculation using a phase-only correlation method and performing three-dimensional reconstruction.

 [0208] According to the above configuration, the 3D shape acquisition unit captures a 2D image of the face with at least two imaging devices, and the 3D shape calculation unit obtains 2 from each imaging device. Two-dimensional image force of the sheet A high-accuracy inspection process is performed by calculation using the phase-only correlation method, and the entire three-dimensional shape is calculated by performing three-dimensional reconstruction.

 Therefore, according to this authentication system, it is possible to calculate the entire three-dimensional shape with high accuracy by the phase-only correlation method at a low cost without using an expensive three-dimensional imaging apparatus or the like.

 [0210] In addition, in the authentication system that has power in another aspect, the three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit is a vector amount, and the comparison face feature corresponding to the vector amount A storage unit is further provided for storing a comparison vector quantity as a quantity.

 [0211] According to the above configuration, the three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit is a vector amount, and is stored as a comparison face feature amount corresponding to the vector amount by the storage unit. The comparison vector quantity is stored.

 [0212] Therefore, according to this authentication system, the data stored as the comparison facial feature quantity by the storage unit becomes the vector quantity that is obtained by the measured so-called dense three-dimensional shape data. The amount can be reduced (requires less memory capacity), and data handling becomes easier.

 [0213] Further, in the authentication system which is particularly effective in other aspects, it is based on the 3D local area determined by the local area determining unit! /, And is a global area in the overall 3D shape 3 A global region shape information relating to a shape of a three-dimensional global region, further comprising a global 3D feature amount calculation unit that calculates a global 3D face feature amount that is a 3D feature amount of the face, The comparison unit compares the global three-dimensional face feature amount calculated by the global three-dimensional feature amount calculation unit, which performs the authentication operation for the authentication target person, with a comparative global facial feature amount prepared in advance.

[0214] According to the above configuration, the global three-dimensional feature amount calculation unit determines the local region determination unit. Based on the specified 3D local area! /, The global area shape information on the shape of the 3D global area, which is the global area of the entire 3D shape, and the 3D feature of the face A global 3D face feature is calculated and prepared in advance by the feature comparison unit to perform an authentication operation on the authentication target by the global 3D feature calculation unit. The compared global facial feature quantity is compared.

 [0215] In addition, in the authentication system which is powerful in other aspects, based on the information of the three-dimensional local region determined by the local region determination unit! /, The global information in the entire three-dimensional shape is used. A global 3D facial feature quantity calculation unit for calculating a global 3D facial feature quantity of the face, wherein the feature quantity comparison unit performs the global 3D feature quantity to perform an authentication operation on the person to be authenticated. The global three-dimensional facial feature value calculated by the calculation unit is compared with a comparative global facial feature value prepared in advance.

 [0216] According to the above configuration, the global 3D feature value calculation unit is based on the information of the 3D local region determined by the local region determination unit! The global 3D facial feature value of the face is calculated, and the global 3D facial feature value calculated by the global 3D feature value calculation unit to perform the authentication operation for the authentication target person by the feature value comparison unit. The comparison global face feature quantity prepared in advance is compared.

 [0217] In addition, in the authentication system that works on other aspects, based on the three-dimensional feature point information defined on the three-dimensional local region determined by the local region determination unit! /, The entire three-dimensional It further includes a global 3D face feature quantity calculation unit that calculates a global 3D face feature quantity that is a global 3D feature quantity of the face, which is global information on the shape.

 [0218] In addition, in the authentication system that has power in another aspect, the global three-dimensional feature amount calculation unit calculates the deformation parameter of the standard model calculated based on the three-dimensional feature point information defined on the three-dimensional local region. Extract information.

 [0219] In addition, in the authentication system that focuses on other aspects, the global three-dimensional feature amount calculation unit calculates a three-dimensional local standard calculated based on three-dimensional feature point information defined on the three-dimensional local region. Extract distance information between model and 3D local area.

[0220] In addition, in the authentication system that has power in another aspect, the global three-dimensional feature quantity calculation unit calculates the three-dimensional station calculated based on the three-dimensional feature point information defined on the three-dimensional local area. Extract distance information between different areas.

 [0221] According to the above configuration, the global 3D feature value calculation unit is based on the 3D feature point information defined on the 3D local region determined by the local region determination unit! A global 3D facial feature value, which is a 3D feature value of the face, is calculated as global information in the 3D shape. Here, for example, information on the deformation parameter of the standard model calculated based on the 3D feature point information defined on the 3D local region is extracted by the global 3D feature amount calculation unit. Also, for example, the global 3D feature quantity calculation unit extracts distance information between the 3D local standard model calculated based on the 3D feature point information defined on the 3D local area and the 3D local area. Also, for example, the global 3D feature quantity calculation unit extracts the distance information between the 3D local areas calculated based on the 3D feature point information defined on the 3D local area! . Then, the feature value comparison unit compares the global three-dimensional face feature value calculated by the global three-dimensional feature value calculation unit with the prepared comparison global face feature value in order to perform an authentication operation on the person to be authenticated. Is done.

 [0222] In addition, in the authentication system that has power in other aspects, the three-dimensional local region determined by the local region determination unit is extracted in a line shape, and based on the extracted line-shaped three-dimensional local region, A global 3D feature quantity calculation unit is further provided that calculates a global 3D local facial feature quantity as a shape vector of the 3D global area, which is a global area in the overall 3D shape.

 [0223] According to the above configuration, the global three-dimensional feature amount calculation unit extracts the three-dimensional local region determined by the local region determination unit in a line shape, and based on the extracted line-shaped three-dimensional local region. Thus, a global 3D facial feature is calculated as a shape vector of the 3D global area, which is a global area in the overall 3D shape. Then, the feature amount comparison unit calculates the global three-dimensional face feature amount calculated by the global three-dimensional feature amount calculation unit so as to perform the authentication operation for the authentication target person, and the comparison-use global face feature amount prepared in advance. Compared

[0224] Therefore, according to these authentication systems, since the global 3D face feature value is used for face authentication, the authentication accuracy can be further improved. The global area shape information can be compressed by a so-called data compression technique, and the data amount can be reduced. In addition, global 3D facial features are calculated based on 3D local regions. Therefore, it is possible to calculate global information unique to the 3D local region. In addition, the amount of data in the 3D local area is less than the 3D shape data of the entire face. In particular, when the 3D local region is determined based on the 3D coordinates of the feature part, it is possible to select only global information between the feature points of the face part from the entire 3D shape data. Become.

[0225] In addition, in the authentication system that is effective in other aspects, the global three-dimensional feature amount calculation unit calculates center-of-gravity information regarding the three-dimensional local region as the global three-dimensional face feature amount.

[0226] Therefore, according to this authentication system, it is possible to calculate information on the center of gravity that is difficult to calculate from the entire three-dimensional shape data, and to calculate a global three-dimensional facial feature amount that is unique to the three-dimensional local region. It becomes possible.

[0227] In addition, in the authentication system that is effective in other aspects, the global three-dimensional feature amount calculation unit calculates normal information related to the three-dimensional local region as the global three-dimensional face feature amount.

[0228] Therefore, according to this authentication system, it is possible to calculate normal information that is difficult to calculate from the entire three-dimensional shape data, and to calculate a global three-dimensional facial feature that is unique to a three-dimensional local region. Is possible.

[0229] In addition, in the authentication system that focuses on other aspects, the feature amount comparison unit performs the three-dimensional feature according to a comparison result between the global three-dimensional face feature amount and the comparative global face feature amount. The 3D face feature value calculated by the quantity calculator is compared with the comparison face feature value prepared in advance.

 Therefore, according to this authentication system, according to the comparison result between the global 3D facial feature quantity and the comparative global facial feature quantity, for example, the comparison result is the global 3D facial feature quantity and the comparative global feature quantity. If the facial feature quantity is different, the comparison between the 3D facial feature quantity and the comparison facial feature quantity can be omitted, the authentication processing time is shortened, and the authentication can be performed faster. Become.

[0231] In addition, in the authentication system that focuses on other aspects, the feature amount comparison unit performs a global three-dimensional feature amount calculation unit that performs an authentication operation on the person to be authenticated. A global comparison result obtained by comparing a facial feature quantity with a comparative global facial feature quantity prepared in advance, and a 3D facial feature quantity calculated by the 3D feature quantity calculation unit The total comparison result is calculated by integrating the local comparison results that are compared with the compared facial feature quantities.

 [0232] Therefore, according to this authentication system, authentication is performed based on the overall comparison result obtained by integrating the global comparison result and the local comparison result, so that the comparison results can be interpolated with each other. Authentication can be performed with high accuracy.

 [0233] In addition, the authentication method which is powerful in other aspects includes a first step of acquiring information on the entire three-dimensional shape, which is the entire three-dimensional shape of the face of the person to be authenticated, and the entire three-dimensional shape. A second step of determining a plurality of three-dimensional local regions, which are local regions in the entire three-dimensional shape, from the shape information, and each three-dimensional local region from the local three-dimensional shape information in the three-dimensional local region Local area shape information related to the shape of the face, and a third step of calculating a three-dimensional facial feature quantity that is a three-dimensional feature quantity of the face; And a fourth step of comparing the face feature quantity with a comparison face feature quantity prepared in advance.

 [0234] According to the above configuration, in the first step, information on the entire three-dimensional shape that is the entire three-dimensional shape of the face of the person to be authenticated is acquired. In the second step, a plurality of three-dimensional local regions that are local regions in the whole three-dimensional shape are determined from the whole three-dimensional shape information. In the third step, from the local 3D shape information in the 3D local area, the local area shape information on the shape of each 3D local area, and the 3D face feature quantity that is the 3D feature quantity of the face Is calculated. Then, in the fourth step, the three-dimensional face feature value is compared with the comparison face feature value prepared in advance to perform the authentication operation for the person to be authenticated.

[0235] Therefore, according to this authentication method, a plurality of 3D local regions are determined from the entire 3D shape of the face of the person to be authenticated, and 3D facial features are obtained from the local 3D shape information in each 3D local region. Since the amount is calculated and the 3D face feature value is compared with the comparison face feature amount, the authentication operation is performed on the person to be authenticated, that is, the information on the entire 3D shape of the face is not used as it is. Since multiple local regions (3D local regions) are extracted from the 3D shape of the entire face and authentication is performed based on the extracted 3D local regions, the Even if a concealment etc. occurs, It is possible to perform authentication using information of a local region other than this part without using the part where the error occurs, and the reduction in authentication accuracy can be reduced. Also, it is not necessary to handle the entire 3D shape information (3D data) with a large amount of data as it is, that is, it is necessary to handle partial 3D shape data in the local area! / Time force S Increases the authentication speed without taking S.

 [0236] In addition, in the authentication method that is effective in another aspect, the first step includes a fifth step of acquiring a two-dimensional image of the face, and the face of the face is obtained from the two-dimensional image. A sixth step of extracting a characteristic part that is a characteristic part; and a seventh step of calculating a three-dimensional coordinate of the characteristic part, wherein the second step includes a three-dimensional part of the characteristic part. Determining the three-dimensional local region based on coordinates.

 [0237] According to this, the first step is a step including a fifth step of acquiring a two-dimensional image of the face, and in the sixth step, a characteristic part of the face is obtained from the two-dimensional image. Feature parts are extracted. In the seventh step, the three-dimensional coordinates of the feature part are calculated. In the second step, the determination is made based on the three-dimensional coordinates of the characteristic part.

 [0238] Therefore, according to this authentication method, when determining a three-dimensional local region, it can be associated with information of a two-dimensional feature region, and the information using the feature region information can be used together with the information of the three-dimensional local region. It is possible to perform accuracy authentication.

 [0239] In addition, in the authentication method which has power in another aspect, the method further includes an eighth step of calculating a two-dimensional face feature value that is a two-dimensional feature value of the face from the information on the feature part, The fourth step is a step of comparing a total face feature amount that is a combination of the two-dimensional face feature amount and the three-dimensional face feature amount with the comparison face feature amount.

 [0240] According to the above configuration, in the eighth step, a two-dimensional facial feature amount that is a two-dimensional facial feature amount is calculated from the feature part information, and in the fourth step, a two-dimensional facial feature amount is calculated. The total facial feature value that is a combination of the 3D face feature value and the face feature value for comparison is compared.

 [0241] Therefore, according to this authentication method, it is possible to perform more accurate authentication using the 2D face feature value and the 3D face feature value.

[0242] In the authentication method according to another aspect, based on the 3D local area determined in the second step! /, A 3D global area that is a global area in the overall 3D shape The method further comprises a ninth step of calculating global three-dimensional face feature amounts that are global region shape information relating to the shape of the face, which is a three-dimensional feature amount of the face, wherein the fourth step includes the authentication subject. The global three-dimensional facial feature value calculated in the above-mentioned nine steps to perform the authentication operation for the above is compared with a comparative global facial feature value prepared in advance.

[0243] According to the above configuration, the global region related to the shape of the three-dimensional global region, which is a global region in the overall three-dimensional shape, based on the three-dimensional local region determined in the second step by the ninth step A global 3D facial feature, which is shape information and is a 3D facial feature, is calculated, and then the global calculated by the 9th step to perform an authentication operation for the person to be authenticated in the 4th step. The three-dimensional face feature quantity is compared with a comparison-use global face feature quantity prepared in advance.

 [0244] Therefore, according to this authentication system, since the global 3D face feature value is used for face authentication, the authentication accuracy can be further improved. The global area shape information can be compressed by a so-called data compression technique, and the data amount can be reduced. In addition, since the global 3D facial feature value is calculated based on the 3D local region, it is possible to calculate global information unique to the 3D local region. In addition, the amount of data in the 3D local area is less than the 3D shape data of the entire face. In particular, when the 3D local region is determined based on the 3D coordinates of the feature part, it is possible to select only global information between the feature points of the face part from the entire 3D shape data. Become.

 [0245] In order to express the present invention, the present invention has been described appropriately and fully through the embodiments with reference to the drawings in the above. However, those skilled in the art may change and / or improve the above-described embodiments. It should be recognized that this can be done easily. Therefore, unless the modification or improvement implemented by a person skilled in the art is at a level that departs from the scope of the claims described in the claims, the modification or the improvement is within the scope of the claims. To be construed as inclusive.

 Industrial applicability

[0246] According to the present invention, it is possible to provide an authentication system and an authentication method for performing face authentication.

Claims

The scope of the claims  [1] A local region determination unit that determines a plurality of three-dimensional local regions that are local regions of the person to be authenticated;  From the local 3D shape information in the 3D local area determined by the local area determination unit, the local area shape information on the shape of each 3D local area, which is the 3D feature quantity of the face 3 A three-dimensional feature amount calculation unit for calculating a three-dimensional face feature amount;  A feature amount comparison unit that compares the three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit with a face feature value for comparison prepared in advance to perform an authentication operation on the person to be authenticated. A featured authentication system. [2] It further includes a 3D shape acquisition unit that acquires information on the entire 3D shape, which is the overall 3D shape of the face of the person to be authenticated,  The local region determination unit determines a plurality of three-dimensional local regions, which are local regions in the total three-dimensional shape, from the total three-dimensional shape information acquired by the three-dimensional shape acquisition unit. The authentication system according to claim 1. [3] The 3D shape acquisition unit includes a 2D image acquisition unit that acquires a 2D image of the face.  The three-dimensional local region is determined based on a result of a feature part extraction unit that extracts a two-dimensional feature part that is a characteristic part of the face from the two-dimensional image acquired by the two-dimensional image acquisition unit. The authentication system according to claim 2. [4] The three-dimensional shape acquisition unit further includes a three-dimensional coordinate calculation unit that calculates the three-dimensional coordinates of the feature part extracted by the feature part extraction unit,  The recognition / certification system according to claim 3, wherein the local region determination unit determines the three-dimensional local region based on the three-dimensional coordinates of the characteristic part calculated by the three-dimensional coordinate calculation unit. .  [5] The feature region extraction unit further includes a two-dimensional local region extraction unit that extracts a local region on the two-dimensional image from the extracted two-dimensional feature region. The three-dimensional local region determination unit determines the three-dimensional local region based on the two-dimensional local region calculated by the two-dimensional local region extraction unit. The described authentication system.  6. The authentication system according to claim 1, wherein the local region determination unit calculates and extracts only a region corresponding to the two-dimensional local region as the three-dimensional local region.  [7] The local region determination unit sets a partial region having a predetermined shape in a plane determined from the three-dimensional coordinates, and sets a region corresponding to the partial region in the entire three-dimensional shape as the three-dimensional local region. 5. The authentication system according to claim 4, wherein the authentication system is determined.  [8] The overall three-dimensional shape information is face shape data including a plurality of three-dimensional points, and the local region determination unit includes a perpendicular line that is virtually dropped from the three-dimensional point to the plane. 8. The authentication system according to claim 7, wherein an area composed of 3D points in the partial area is determined as the 3D local area.  [9] The local region determination unit compares the entire three-dimensional shape with a reference three-dimensional partial model shape prepared in advance, and most closely matches the reference three-dimensional partial model shape in the whole three-dimensional shape. 3. The authentication system according to claim 2, wherein a part having a similar shape is determined as the three-dimensional local region.  [10] The local region determination unit includes an identical space conversion unit that converts the local region information defined on the entire three-dimensional shape and a reference three-dimensional partial model shape prepared in advance into the same space. And determining the three-dimensional local region by comparing the inclusion relationship between the entire three-dimensional shape in the same space transformed by the same space transformation unit and the reference three-dimensional partial model shape. The authentication system according to claim 2.  [11] The three-dimensional local region determination unit determines the three-dimensional local region by comparing the inclusion relation between the three-dimensional surface on the reference three-dimensional model and the three-dimensional surface of the entire three-dimensional shape. The authentication system according to claim 10, wherein the authentication system is determined.  [12] The three-dimensional local region determination unit compares the inclusion relation between a three-dimensional surface on the reference three-dimensional model and a three-dimensional coordinate point of the entire three-dimensional shape, thereby determining the three-dimensional local region. The authentication system according to claim 10, wherein the authentication system is determined as: [13] The three-dimensional local region determination unit compares the inclusion relation between the three-dimensional coordinate point on the reference three-dimensional model and the three-dimensional surface of the entire three-dimensional shape, thereby determining the three-dimensional local region. 11. The authentication system according to claim 10, wherein the authentication system is determined as an area. [14] The three-dimensional local region determined by the local region determination unit is set as dense data, and the three-dimensional local outside region determined other than the three-dimensional local region is stored as sparse data. The authentication system according to claim 2. 15. The three-dimensional feature amount calculation unit according to claim 1, wherein the three-dimensional feature amount calculation unit calculates local three-dimensional shape information from the three-dimensional local region as the local region shape information. Eventually, the authentication system according to item 1. 16. The three-dimensional feature amount calculation unit calculates the local region shape information obtained by converting local three-dimensional shape information in the three-dimensional local region into predetermined curved surface information. The described authentication system. [17] The three-dimensional feature amount calculation unit converts local three-dimensional shape information in the three-dimensional local region into a vector from distance information between a defined point defined on the standard model and a corresponding point in the three-dimensional local region. The authentication system according to claim 16, wherein the local area shape information is calculated. 18. The three-dimensional feature quantity calculating unit calculates a three-dimensional face feature quantity including information on a relative positional relationship of each three-dimensional local region as the three-dimensional face feature quantity. The power according to claim 17, the authentication system according to claim 1. [19] The local region determining unit determines the three-dimensional local region in the entire three-dimensional shape so that the plurality of three-dimensional local regions are arranged at positions where the face is symmetrical. The authentication system according to any one of claims 1 to 18. [20] The local region determining unit determines the three-dimensional local region in the entire three-dimensional shape so that the plurality of three-dimensional local regions include at least the nose and the heel portion of the face. The authentication system according to any one of claims 1 to 19, wherein: [21] The apparatus further comprises a two-dimensional feature amount calculation unit that calculates a two-dimensional face feature amount, which is a two-dimensional feature amount of the face, from information on the feature portion extracted by the feature portion extraction unit, The feature amount comparison unit is a comprehensive face feature that is a combination of the two-dimensional face feature amount calculated by the two-dimensional feature amount calculation unit and the three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit. The amount is compared with the comparison face feature amount. Recognition, proof system.  [22] The three-dimensional feature quantity calculating unit calculates the three-dimensional face feature quantity from local three-dimensional shape information in a three-dimensional local region including at least a part other than the facial feature part. 24. The authentication system according to item 21.  [23] The feature part information for calculating the two-dimensional face feature amount is texture information, and with respect to the texture information, the posture variation correction that is correction related to the posture of the face and the direction of the light source with respect to the face. 23. The authentication system according to claim 21, further comprising a correction unit that performs light source fluctuation correction as correction. [24] The three-dimensional shape acquisition unit includes:  At least two photographing devices for photographing a two-dimensional image of the face;  From the two 2D images obtained from each imaging device, the corresponding point search process is performed by calculation using the phase-only correlation method, and the 3D reconstruction is performed to calculate the overall 3D shape. The authentication system according to claim 2, further comprising a calculation unit.  [25] The three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit is a vector amount, and stores a comparison vector amount as the comparison face feature amount corresponding to the vector amount. The authentication system according to any one of claims 1 to 24, further comprising a section.  [26] Global region shape information relating to a shape of a three-dimensional global region, which is a global region in the overall three-dimensional shape, based on the three-dimensional local region determined by the local region determination unit, the face A global 3D feature quantity calculation unit for calculating a global 3D face feature quantity, which is a 3D feature quantity of  The feature amount comparison unit includes a global three-dimensional face feature amount calculated by the global three-dimensional feature amount calculation unit to perform an authentication operation on the authentication target person, and a comparison global face feature amount prepared in advance. Comparing  15. The authentication system according to any one of claims 1 to 14, characterized in that: [27] Based on the information of the three-dimensional local region determined by the local region determination unit, A global 3D facial feature amount calculation unit for calculating the global 3D facial feature amount of the face, which is global information on the 3D shape of the body;  The feature amount comparison unit includes a global three-dimensional face feature amount calculated by the global three-dimensional feature amount calculation unit to perform an authentication operation on the authentication target person, and a comparison global face feature amount prepared in advance. Comparing  15. The authentication system according to any one of claims 2 to 14, characterized by: [28] Based on the three-dimensional feature point information defined on the three-dimensional local region determined by the local region determination unit, the three-dimensional information of the face is global information in the entire three-dimensional shape. A global 3D facial feature quantity calculation unit that calculates a global 3D facial feature quantity that is a feature quantity;  28. The authentication system according to claim 27. 29. The global three-dimensional feature quantity calculation unit extracts information on deformation parameters of a standard model calculated based on three-dimensional feature point information defined on the three-dimensional local area. The authentication system described in. [30] The global three-dimensional feature amount calculation unit extracts distance information between the three-dimensional local standard model calculated based on the three-dimensional feature point information defined on the three-dimensional local region and the three-dimensional local region. thing  The authentication system according to claim 28, wherein: [31] The global three-dimensional feature quantity calculation unit extracts distance information between the three-dimensional local areas calculated based on three-dimensional feature point information defined on the three-dimensional local area. The authentication system according to Item 28. [32] The 3D local area determined by the local area determining unit is extracted in a line shape, and based on the extracted line-shaped 3D local area! /, The global area in the overall 3D shape is extracted. A global 3D feature quantity calculation unit that calculates a global 3D local facial feature quantity as a shape vector of the 3D global area that is a global area.  28. The authentication system according to claim 27. [33] The global three-dimensional feature amount calculation unit calculates center of gravity information regarding the three-dimensional local region as the global three-dimensional face feature amount. 27. An authentication system according to claim 26. [34] The global 3D feature amount calculation unit calculates normal information regarding the 3D local region as the global 3D face feature amount.  27. An authentication system according to claim 26. [35] The feature quantity comparison unit includes the 3D face feature quantity calculated by the 3D feature quantity calculation unit according to a comparison result between the global 3D face feature quantity and the comparative global face feature quantity. Compare with the prepared facial features for comparison.  27. An authentication system according to claim 26. [36] The feature amount comparison unit performs the authentication operation on the person to be authenticated, the global three-dimensional face feature amount calculated by the global three-dimensional feature amount calculation unit, and a prepared comparison global face feature. The global comparison result comparing the amount and the local comparison result comparing the three-dimensional face feature amount calculated by the three-dimensional feature amount calculation unit and the comparison facial feature amount prepared in advance are integrated. Calculating comparison results  27. An authentication system according to claim 26. [37] a first step of acquiring information on an overall 3D shape, which is an overall 3D shape of the face of the person to be authenticated;  A second step of determining a plurality of three-dimensional local regions that are local regions in the whole three-dimensional shape from the whole three-dimensional shape information;  A local region shape information related to the shape of each three-dimensional local region is calculated from the local three-dimensional shape information in the three-dimensional local region, and a three-dimensional face feature amount that is a three-dimensional feature amount of the face is calculated. 3 processes,  An authentication method comprising: a fourth step of comparing the three-dimensional face feature quantity with a comparison face feature quantity prepared in advance to perform an authentication operation on the person to be authenticated. [38] The first step includes a fifth step of acquiring a two-dimensional image of the face, and a sixth portion that extracts a characteristic portion that is a characteristic portion of the face from the two-dimensional image. And the process of  A seventh step of calculating the three-dimensional coordinates of the characteristic part, The second step is based on the three-dimensional coordinates of the characteristic part! / 38. The authentication method according to claim 37, wherein the authentication method is a step of determining. [39] The method further includes an eighth step of calculating a two-dimensional facial feature quantity that is a two-dimensional feature quantity of the face from the information of the feature part,  The fourth step is a step of comparing an overall face feature amount combining the two-dimensional face feature amount and the three-dimensional face feature amount with the comparison face feature amount. The authentication method according to claim 38. [40] Global region shape information relating to the shape of a three-dimensional global region, which is a global region in the overall three-dimensional shape, based on the three-dimensional local region determined in the second step, A ninth step of calculating a global three-dimensional facial feature quantity that is a dimensional feature quantity;  In the fourth step, the global three-dimensional facial feature value calculated in the ninth step to perform an authentication operation for the subject to be authenticated is compared with a comparative global facial feature value prepared in advance.  38. The authentication method according to claim 37, wherein: