JP2008165314A - Personal identification device and personal identification method - Google Patents

Abstract

A personal identification device suitable for identifying an individual over a long period of time is provided.
Dictionary data for identifying an individual Pi (i = 1, 2,...) Is generated from feature data extracted from an identification element for identifying an individual in a plurality of different periods. A plurality of period dictionary data TDi, n (n = 1, 2,..., N) are provided. Identification of each individual different period is performed using the period dictionary data TDi, n of the corresponding period.
[Selection] Figure 8

Description

  The present invention relates to a personal identification device and a personal identification method for identifying an individual using an identification element for identifying the individual, such as an individual's face image or voice.

  For example, characteristic data of identification elements such as faces and voices of a plurality of persons to be identified are stored in advance as dictionary data, and the characteristic data of the identification elements of the individual to be identified and the dictionary data A personal identification method is known in which the identification element of a captured individual is identified by comparing it with characteristic data of the stored identification element.

  In this personal identification method, if a face image is taken as an example of an identification element, the dictionary data is generated using the feature data extracted from the face image of the individual to be identified. In this case, conventionally, only one dictionary data is generated. In general, in order to average capture errors such as imaging conditions and face orientation when capturing facial images as identification elements, feature data is averaged by capturing multiple facial images for the same individual. Is stored as dictionary data for identifying the individual (see Japanese Patent Application Laid-Open No. 2003-271958).

  Normally, when a face image for each individual is newly identified, the feature data of the identified individual face image is reflected in the one identification dictionary data, so that each individual The dictionary data of the face image for use is updated.

The above-mentioned patent documents are as follows.
JP 2003-271958 A

  The personal identification technique using the conventional face identification technique is mostly used for a security system, and the identification element such as a personal face image at the present time is compared with dictionary data. For this reason, face identification is performed by using one face identification dictionary data focused on improving the identification accuracy of the current face rather than the past face. The one dictionary data is updated by updating the dictionary data itself while deleting past registered images so as to follow the current face.

  By the way, recently, with the widespread use of digital still cameras and the increased storage capacity of personal computers, a large number of photographed images are taken into a large-capacity storage device of a personal computer to create an album. It can be managed. There are also sites that offer similar photo management services on the Web.

  In such a case, the photo management application used in the personal computer and the Web service are not limited to the case where the photograph image to be captured is always a new face image, and it is also expected that a past photograph image is captured.

  Since human faces and voices usually change over time, it seems to follow the current face image used in the conventional face identification technology in the above-mentioned photo management application and Web service. In the method of identifying a face using one dictionary data created by updating to the past, there is a possibility that past photo images cannot be identified as the same person.

  In addition, as described above, when one piece of conventional face identification dictionary data that is updated by averaging feature data of a plurality of face images captured for the same individual is used for the above-described photo management, a long period of time is required. When a person's face is identified in, a dictionary is created based on long-term face images from the past to the current photo, and the personal features are averaged, and no improvement in accuracy is expected. .

  It is known that not only a person's face but also a person's voice changes over time, and the same can be said when personal identification is performed using a person's voice.

  In view of the above points, an object of the present invention is to provide a personal identification device suitable for identifying an individual over a long period of time.

In order to solve the above problems, the personal identification device of the present invention is:
As dictionary data for identifying an individual, a plurality of period dictionary data generated from feature data extracted from an identification element for identifying the individual in each of a plurality of different periods,
The identification of the individual in each of the different periods is performed using the period dictionary data of the corresponding period.

  According to the personal identification device of the present invention having the above-described configuration, a plurality of period dictionary data for each of a plurality of different periods are provided, and the identification of each individual different period is performed by using the period dictionary for the corresponding period. Therefore, personal identification corresponding to changes in human identification elements is possible from the past to the present.

  According to the present invention, by using a plurality of period dictionary data in each of a plurality of different periods to identify an individual in a corresponding period, the identification element of a person from the past to the present can be obtained. Individual identification corresponding to changes is possible.

  Hereinafter, an embodiment of a personal identification device according to the present invention will be described with reference to the drawings, taking as an example a personal identification device as an identification element for identifying a face image.

[First Embodiment]
FIG. 1 is a block diagram illustrating a configuration example of a personal identification device based on face images according to this embodiment. The example of FIG. 1 is an example in the case where a personal identification device based on a face image is configured in a personal computer, for example, and some of the components shown in blocks can also have a software configuration.

  As shown in FIG. 1, the personal identification device based on a facial image of this embodiment is a ROM (Read Only Memory) 3 in which a program or the like is stored via a system bus 2 with respect to a CPU (Central Processing Unit) 1. A work area RAM (Random Access Memory) 4, an operation unit interface 5, a display interface 76, and an image data input interface 7 are connected.

  An operation input unit 8 such as a keyboard or a mouse is connected to the operation unit interface 5. The display interface 6 is connected to a display 9 such as an LCD (Liquid Crystal Display).

  The image data input interface 7 includes a USB terminal and a USB interface when receiving image data from an image output device such as an imaging camera (digital camera) having a USB (Universal Serial Bus) interface, for example.

  The image data input interface 7 includes a memory reader in which the card type memory is loaded when receiving image data from the card type memory storing the image data.

  Further, the image data input interface 7 is configured by an optical disk drive capable of reading image data from the optical disk when receiving image data from the optical disk storing the image data.

  Note that the image data input interface 7 captures not only image data but also information (for example, information in the format of an exchange image file (Exif)) that is associated with the image data and includes, for example, imaging time information and imaging conditions. Is.

  In this case, if the image data taken in through the image data input interface 7 does not include information on the imaging time, the CPU 1 does not provide information on the imaging time to the user, although illustration is omitted. Therefore, a message prompting the user to input the shooting time is displayed on the display 9, and information on the imaging time input by the user corresponding to this message is acquired in correspondence with the captured image data.

  The image data input interface 7 may be a scanner that scans and captures a photograph printed on photographic paper. In this case, the user inputs the shooting time information.

  In this embodiment, the system bus 2 includes a face identification dictionary database 10, a face identification image processing unit 11, a face identification processing unit 12, a display information generation unit 13, and a dictionary data update processing unit. 14 is connected.

  The face identification dictionary database 10 is configured by using all or a part of a storage area of a storage device such as a hard disk device. In this embodiment, a plurality of individuals Pi (i = 1, 2,..., Registered in the face identification dictionary database 10 for personal identification based on face images are the same. I corresponds to a personal number. For example, personal dictionary data Di for each of i = 1 is “Mr. A” and i = 2 is “Mr. B” and personal identification information (personal identification data) for identifying each individual. Correspondingly stored and held. In the following description, “face identification” is synonymous with “personal identification by face image”.

  As shown in FIG. 2, the face identifying image processing unit 11 performs face detection and eye detection from the image data of the original image taken in through the image data input interface 7 so that the detected positions of both eyes are horizontal. The face image is rotated according to the eye position. The face image is rotated according to the position of the eyes because, as will be described later, in face identification, the face image is a face image that faces the front and both eyes are horizontal. is there.

  Further, the face identifying image processing unit 11 performs a feature point plot (feature point sampling) for extracting feature data of feature parts (for example, eyebrows, eyes, nose, mouth, etc.) for identifying the face image. Then, a feature vector Vin as feature data for the captured face image Fin is acquired. Here, the feature vector Vin is composed of a function having a plurality of types of feature values as variables.

  In addition, regarding processing such as face detection, eye detection, and calculation of a feature vector Vin in the face identification image processing unit 11, Patent Document 2 (Japanese Patent Laid-Open No. 2006-72770) and Patent Document 3 (Japanese Patent Laid-Open No. 2005-44330). Gazette), non-patent literature (Kotaro Sabe, Kenichi Hidai, "Learning a Real-Time Arbitrary Posture Face Detector Using Pixel Difference Features", Proceedings of the 10th Image Sensing Symposium, pp. 547-552, 2004 (June of year) etc., and will be omitted here.

  The face identification image processing unit 11 supplies the feature vector Vin as feature data acquired for the captured face image Fin to the face identification processing unit 12.

  In addition, when face detection or eye detection cannot be performed in the face identification image processing unit 11, or when face detection or eye detection can be performed but a feature vector cannot be calculated, this is indicated. The data is sent to the identification processing unit 12 and the display information generation unit 13.

  The face identification processing unit 12 receives the feature vector Vin as the feature data of the face image Fin acquired from the image captured from the face identification image processing unit 11, and the feature vector Vin and the face identification dictionary database The personal dictionary data Di of a plurality of individuals stored in 10 are compared and collated, and the personal dictionary data Di that can be determined to match is detected.

  When the personal dictionary data Di that can be determined to match can be detected, the face identification processing unit 12 and the personal identification information (personal identification data) corresponding to the detected personal dictionary data Di. In this example, the personal name is sent to the display information generating unit 13.

  Further, when the personal dictionary data Di that can be determined to match is not detected, the face identification processing unit 12 sends a message to that effect to the display information generation unit 13.

  Note that the face identification processing unit 12 cannot detect the face or eyes from the face identification image processing unit 11, or does not calculate the feature vector even if face detection or eye detection is possible. Is received, the face identification process described above is not executed.

  When the display information generation unit 13 receives from the face identification processing unit 12 that the face has been identified and the name of the individual as personal identification information as the face identification result, the display information generation unit 13 displays the face identification result as shown in FIG. Display information of a face identification result confirmation screen for confirmation by the user is generated and displayed on the screen of the display 9 through the display interface 6.

  As shown in FIG. 3, a display image 21 of the captured face image Fin is displayed on the face identification result confirmation screen, and a message “This person asks whether the face identification result is correct” Is it ○○○? ”And button icons 23 and 24 for answers to the inquiry message 22 are displayed. The name of the face identification result is displayed in the portion of “XX” in the inquiry message 22.

  In the example of FIG. 3, the face identification processing unit 12 determines that the feature data of the captured face image matches the feature data of the face image in the personal dictionary data Di of “Taro Tanaka”. “Taro Tanaka” is displayed in the “XXX” part of

  The answer button icon 23 is for an affirmative answer, and the answer button icon 24 is for a negative answer.

  In the screen of FIG. 3, when the user clicks the button icon 23 for an affirmative answer using the operation input unit 8, the display information generation unit 13 stores the name information of the face identification result in the dictionary under the control of the CPU 1. The data is supplied to the data update processing unit 14. As will be described later, the dictionary data update processing unit 14 newly generates (newly registers) personal dictionary data Di stored in the face identification dictionary database 10 corresponding to the personal identification information specified by the received name. Update processing is performed.

  In addition, when the user clicks the negative answer button icon 24 using the operation input unit 8 on the screen of FIG. 3, the display information generation unit 13 displays a face identification result as shown in FIG. Display information of a face identification result incorrect answer screen that informs the user that the answer is correct is generated and displayed on the screen of the display 9 through the display interface 6.

  In this face identification result incorrect answer screen, as shown in FIG. 4, the display image 21 of the captured face image Fin and the name of the person who is the captured face image Fin are prompted to input a name. Message 25 is displayed. The user inputs the personal name displayed as the display image 21 in the name input field 26 in the message 25.

  When receiving the name input by the user, the display information generation unit 13 supplies the input name information to the dictionary data update processing unit 14 according to the control of the CPU 1. As will be described later, the dictionary data update processing unit 14 performs new generation or update processing of the personal dictionary data Di stored in the face identification dictionary database 10 corresponding to the personal identification information including the received name.

  When the user is instructed to end the face identification process without inputting the name in the name input field 26, the display information generating unit 13 does not transmit the name information, and the dictionary is controlled according to the control of the CPU 1. The dictionary data update processing unit 14 is notified that the update is not performed. Therefore, the dictionary data update processing unit 14 does not generate or update personal dictionary data.

  Further, when the display information generation unit 13 receives a notification from the face identification processing unit 12 that there is no personal dictionary data Di that matches the captured face image in the face identification dictionary database 10 as a face identification result. As shown in FIG. 5, display information of a face identification result non-match screen for informing the user of “no match” as the face identification result is generated and displayed on the screen of the display 9 through the display interface 6. .

  On this face identification result non-match screen, as shown in FIG. 5, the display image 21 of the captured face image Fin and the name of the person who is the captured face image Fin are prompted to input a name. Message 27 is displayed. The user inputs the personal name displayed as the display image 21 in the name input field 28 in the message 27.

  When receiving the input of the name by the user, the display information generation unit 13 supplies the input name information to the dictionary data update processing unit 14. As will be described later, the dictionary data update processing unit 14 performs new generation or update processing of personal dictionary data Di using the received name as personal identification information.

  When the user is instructed to end the face identification process without inputting the name in the name input field 28, the display information generating unit 13 does not transmit the name information and does not update the dictionary. Is notified to the dictionary data update processing unit 14. Therefore, the dictionary data update processing unit 14 does not generate or update dictionary data.

  When the display information generation unit 13 receives a notification from the face identification image processing unit 11 that face detection, eye detection, and finally calculation of a feature vector cannot be performed, as shown in FIG. Further, the display information of the non-verification notification screen is generated and displayed on the screen of the display 9 through the display interface 6.

  As shown in FIG. 6, a message 29 indicating that the captured face image is not used for updating the dictionary data is displayed on this incompatibility notification screen. Then, according to the control of the CPU 1, the display information generation unit 13 notifies the dictionary data update processing unit 14 of a notification that the dictionary data is not updated.

  In this embodiment, the dictionary data update processing unit 14 performs update processing of the personal dictionary data Di corresponding to the personal identification information when the personal identification information is specified based on the identification result in the face identification processing unit 13. To do.

  In this case, as can be seen from the above description, when the personal identification information is specified based on the identification result in the face identification processing unit 13, the face identification result is correct and the personal identification information is personal identification. When the device is specified, the face identification result is incorrect, and the personal identification information is specified by the personal name entered by the user. This includes the case of capturing a face image and the case where the individual specifying information is specified by the name of the individual input by the user.

  When the face identification image processing unit 11 receives notification that face detection, eye detection, and finally feature vector calculation cannot be performed, the dictionary data update processing unit 14 stores the personal dictionary data Di. Update processing is not performed. In addition, even when the personal identification information about the captured face image cannot be finally specified, the dictionary data update processing unit 14 does not perform the update process of the personal dictionary data Di.

  Conventionally, as personal dictionary data Di of the face identification dictionary database 10, one face identification dictionary data is registered for each individual, and one face identification dictionary data for each individual is registered. And the feature data extracted from the captured individual identification element are identified to identify the individual having the captured identification element.

  On the other hand, in the face identification dictionary database 10 of this embodiment, each personal dictionary data Di of “Mr. A”, “Mr. B”, “Mr. C”,. A plurality of period dictionary data are provided.

  That is, in this embodiment, as shown in FIG. 8, the face identification dictionary database 10 includes a plurality of different periods such as period 1, period 2,... As personal dictionary data Di per person. A plurality of period dictionary data TDi, 1, TDi, 2,..., TDi, N are provided for period N, and the plurality of period dictionary data are personal identification information that is individual identification information of each individual. It is stored in association with data IDi.

  In this embodiment, a personal name (name) is registered as the personal identification information (personal identification data IDi). The personal identification information is not limited to the personal name (name), but may be personal identification numbers, symbols, or other personal information, or a combination of information for identifying those individuals. Also good.

  In this embodiment, as shown in FIG. 8, each period of a plurality of period dictionary data TDi, j (j = 1, 2,..., N) is used as a part of the personal dictionary data Di. Reference time data Rt serving as a reference to be set is stored. In this embodiment, as the reference time data, data of the photographing time of the face image first captured for the corresponding individual Pi is used. In this case, the shooting time includes the year, month, day, hour, minute, and second when the shooting was performed. However, in this embodiment, when the period is set, the day is set as the minimum unit, so the period is set. As a reference time, only the date is used.

  Each of the plurality of period dictionary data TDi, j of each individual is captured when the corresponding individual face image photographed within the period corresponding to each period dictionary data TDi, j is captured. The feature data extracted from the extracted face image is used to generate a new one.

  Then, after the generation of the period dictionary data TDi, j, when there is further capture of the corresponding individual face image photographed within the corresponding period, it is extracted from the further captured face image. The period dictionary data TDi, j is updated using the feature data.

  Therefore, in this example, the number N of the plurality of period dictionary data TDi, 1, TDi, 2,..., TDi, N constituting the personal dictionary data Di of each individual is usually set for each individual Pi. It will be different.

  In the personal identification device of this embodiment, when the length of the period corresponding to the period dictionary data TDi, j is a fixed length that can be set by the user (fixed length mode), the age of the individual to be identified is set. Accordingly, the user can select and set one of two period modes when the variable length is set (variable length mode).

  In the fixed length mode in which the length of the period corresponding to the period dictionary data TDi, j is constant, for example, when it is assumed that the degree of change of the face is constant, for example, only the middle age period is set as the individual identification target period. This is effective when

  On the other hand, when the individual identification target period is considered to be long, a method of changing the period depending on the age is considered effective. In this method, the period dictionary data corresponding to the early growth period is shortened, and the period corresponding to the period dictionary data is gradually increased with the growth.

  In this embodiment, an overlap period is provided when two period dictionary data periods are adjacent to each other. The reason for providing the overlapping period in this way is to prevent abrupt change in adjacent periods in a plurality of period dictionary data for one individual.

  For example, if the period dictionary data TDa is created in 2004 and the period dictionary data TDb is created in 2005 based on the feature data of the face image of a certain individual Px, the year 2004 The feature data of the face image taken on December 31 is used only to update only the period dictionary data TDa if there is no overlapping period. However, since December 31, 2004 is close to 2005 in terms of period, it should be used to update the period dictionary data TDb. Therefore, in this embodiment, for example, one month of December 2004 is set as an overlapping period in the corresponding period of the period dictionary data TDa and the corresponding period of the period dictionary data TDb.

  Then, the feature data of the face image photographed for one month in December 2004 becomes the update target data for both the period dictionary data TDa and the period dictionary data TDb, and abrupt changes in the feature data of the dictionary data Can be avoided.

  In the first embodiment, either the fixed length mode or the variable length mode is set in advance as the period mode for the entire face identification dictionary database 10. This period mode is used when period dictionary data is newly created (new registration).

  That is, when a face image as a personal identification element is newly acquired and personal identification is performed, if no dictionary data is found for a period including the time point of the face image (for example, the time of shooting the face image), Period dictionary data is newly created and registered (new registration) in the identification dictionary database. At that time, referring to this period mode, the period dictionary data to be newly created corresponds to the period and overlap period. To set.

  In this embodiment, the user can select and set a fixed period length when selecting the fixed length mode. That is, in this embodiment, for example, a fixed dictionary period table TBLx as shown in FIG. 9A is prepared, and the user can select and set a fixed period length from the fixed dictionary period table TBLx. It is configured as follows.

  In the fixed dictionary period table TBLx in the example of FIG. 9A, the length of the dictionary period (hereinafter referred to as the dictionary period width) WD and the overlap period corresponding to the dictionary period width WD are associated with the table number. A period length (hereinafter referred to as overlapping period width) Wov is set in a one-to-one correspondence. In this case, the overlap period width Wov is selected to gradually increase as the dictionary period width WD increases.

  As will be described later, in this embodiment, when the user selects the fixed-length mode, the fixed dictionary period table TBLx is presented to the user, and the user selects and designates the table number of the table TBLx, so that the corresponding dictionary is displayed. The time width WD and the overlapping period width Wov are selected and set.

  Then, from the reference time data Rt and the photographing time of the newly captured face image, the dictionary time width WD and the overlapping period width Wov selected and set are taken into consideration, and the period dictionary data to be newly generated The start point ST and end point ED of the period are calculated, the period is specified, period dictionary data TDi, j for the specified period is generated, and newly registered in the face identification dictionary database 10. At the time of this new registration, as will be described later, the dictionary time width WD and overlapping period width Wov, the start point ST and end point ED of the period, etc. are stored as accompanying information corresponding to the period dictionary data TDi, j.

  Further, in this embodiment, when the user selects the variable length mode, when the period dictionary data is newly registered, the input of the personal age when the user has a newly captured face image is For example, a variable dictionary period table TBLv as shown in FIG. 9B is used to set the dictionary period width WD and the overlapping period width Wov according to the input age.

  Then, in the same manner as in the fixed-length mode, the reference time data Rt and the newly captured face image shooting time are newly taken into consideration in the selected dictionary time width WD and overlap period width Wov. The period start point ST and end point ED for the period dictionary data to be generated are calculated, the period is specified, the period dictionary data TDi, j for the specified period is generated, and the face identification dictionary It is newly registered in the database 10. Then, as in the fixed length mode, at the time of this new registration, as will be described later, the dictionary time width WD and the overlapping period width Wov, the start point ST and the end point ED of the period, etc. are stored in the period dictionary data TDi, j Is stored as accompanying information.

  The fixed dictionary period table TBLx in FIG. 9A and the variable dictionary period table TBLv in FIG. 9B are stored in the face identification dictionary database 10 as dictionary period table data TBL, as shown in FIG. .

  FIG. 10 is a diagram for explaining the format of data stored in the face identification dictionary database 10 according to the first embodiment.

  In the first embodiment, as shown in FIG. 10, the dictionary period table data TBL, the period mode selection designation data, and the selected period mode are fixed length mode for the entire face identification dictionary database 10. Sometimes the table number is included, and in the variable length mode, the user period setting information file US including the reference pointer to the variable dictionary period table is stored.

  As described above, the personal dictionary data Di for each individual Pi is composed of a plurality of period dictionary data TDi, 1, TDi, 2,... Corresponding to each period 1, period 2,. . In this embodiment, each of the period dictionary data TDi, 1, TDi, 2,... Is face identification dictionary data FRDi consisting of an average value of feature data of the face image captured in the corresponding period. , 1, FRDi, 2... And accompanying information ATRi, 1, ATRi, 2.

  Each of the accompanying information ATRi, 1, ATRi, 2,..., ATRi, n,... Is a period before the period corresponding to the period dictionary data TDi, n, as shown in FIG. The previous dictionary pointer that points to the period dictionary data that shares the overlapping period, and the subsequent dictionary pointer that points to the period dictionary data that is after the period corresponding to the period dictionary data TDi, n and that shares the overlapping period A dictionary period parameter, the total number of images in the period, and period information.

  When there is other period dictionary data sharing the corresponding overlapping period, the previous dictionary pointer and the subsequent dictionary pointer are identification data indicating the previous period dictionary data or the subsequent period dictionary data, or information on the storage address position thereof. In addition, when there is no other period dictionary data sharing the corresponding overlapping period, “NULL” data is meant.

  The dictionary period parameter includes a pointer to the dictionary period width corresponding to the period dictionary data TDi, n and a pointer to the overlapping period width. In the fixed-length mode, the pointer to the dictionary period width and the pointer to the overlap period width are the memory addresses of the dictionary period width and the overlap period width selected by the user from the fixed dictionary period table TBLx (face identification dictionary database). In the case of the variable length mode, the memory address (face of the dictionary period width and the overlapping period width selected from the variable dictionary period table TBLv according to the age corresponding to the period of the period dictionary data. Address on the dictionary database for identification).

  The total number of images in the period is the number of face images captured in the past in the period corresponding to the period dictionary data TDi, n. In this embodiment, the number of face images in the first half of the period and the number of face images in the period The number of face images in the latter half is stored separately.

  The period information includes start point data ST indicating the start time of the period corresponding to the period dictionary data TDi, n and end point data ED indicating the end time. As in the description of the reference time, in this embodiment, since the minimum unit of the period is set to the day, the start point data ST and the end point data ED are data representing the year, month, and day, respectively.

  The accompanying information in FIG. 11 is mainly used when searching a dictionary when a personal face image is newly taken in for personal identification.

  As a result of generating the period dictionary data TDi, n as described above, as shown in FIG. 12A in the fixed-length mode, and as shown in FIG. 12B in the variable-length mode. Thus, a plurality of period dictionary data TDi, n are generated for one individual according to the flow of time.

  FIG. 12A showing the case of the fixed length mode shows a plurality of period dictionary data for “Mr. A” as DA1, DA2, DA3,... The width of each dictionary period of the dictionary data DA1, DA2, DA3,... Is set to a predetermined dictionary period WDx selected from the fixed dictionary period table TBLx by the user. And the width of the overlap period between the period dictionary data DA1, DA2, DA3,... Is all set to the overlap period width Wovx selected corresponding to the dictionary period WDx.

  Also, in FIG. 12B showing the case of the variable length mode, a plurality of period dictionary data for “Mr. A” in period 1 according to age are shown as DA11, DA12, DA13,. A plurality of period dictionary data in period 2 according to age is shown as DA21, DA22, DA23,..., And a plurality of period dictionary data in period 3 according to age is shown as DA31, DA32, DA33,. It is shown. For example, in the variable dictionary period table TBLv, period 1 is a period of 0 to 1 year of age, period 2 is a period of 1 to 3 years, and period 3 is a period of 20 to 24 years.

  Accordingly, the dictionary period width and the overlapping period width of each period dictionary data are set to the dictionary period width WD1 and the overlapping period width Wov1 according to the age of the period 1 in the period 1, and to the age of the period 2 in the period 2. The corresponding dictionary period width WD2 and the overlapping period width Wov2 are set. In the period 3, the dictionary period width WD3 and the overlapping period width Wov3 corresponding to the age of the period 3 are set. In this example, as is apparent from the variable dictionary period table TBLv in FIG. 9, WD1 <WD2 <WD3 and Wov1 <Wov2 <Wov3.

  In FIG. 12, a plurality of period dictionary data are generated without a gap between periods according to the flow of time, but in reality, a period dictionary corresponding to a period during which a personal face image is captured for identification. Data is generated and not all period dictionary data is generated for an individual.

  Therefore, at a certain point in time, there is a period dictionary data that exists alone without having an overlapping period with other period dictionary data, and there is a state in which only one period dictionary data exists for a certain age. Periodic dictionary data is increased by acquiring a large number of personal face images for different periods for identification. Then, the period dictionary data sharing the overlapping period with other period dictionary data also increases.

  In the personal identification processing of this embodiment, when personal face image data is captured for identification through the image data input interface 7, the face identification processing unit 12 includes a period including the shooting time of the captured facial image. Is retrieved from a plurality of personal dictionary data stored in the face identification dictionary database 10.

  Then, as a result of the search, the face identification processing unit 12 determines whether one or more period dictionary data corresponding to the period including the captured time of the captured face image has been detected, and outputs the determination result. At the same time, if one or more are detected, the corresponding period dictionary data and the feature data extracted from the captured face image are compared and referenced to determine that the feature data matches the feature data of the captured face image. Look for dictionary data for a period of time.

  When the display information generation unit 13 receives from the output from the face identification processing unit 12 a notification that no period dictionary data corresponding to the period including the shooting time of the captured face image has been detected, As shown in FIG. 7, a period dictionary non-existing screen is displayed to notify the user that the corresponding period dictionary data does not exist.

  As shown in FIG. 7, on this period non-dictionary screen, as shown in FIG. 7, a display image 21 of the captured face image Fin and a message 30 prompting the user to input the name of the face image Fin that has been captured. Is displayed. The user inputs the individual name displayed as the display image 21 in the name input field 31 in the message 30.

  When receiving the input of the name by the user, the display information generation unit 13 supplies the input name information to the dictionary data update processing unit 14. As will be described later, the dictionary data update processing unit 14 performs a new generation process of the period dictionary data of the personal dictionary data Di using the received name as personal identification information.

  When the face identification processing unit 12 finds two period dictionary data including the shooting time of the captured face image in the overlap period as a result of the search, both period dictionary data are used for matching with the feature data. However, in this embodiment, in order to shorten the collation time, the face identification processing unit 12 selects only one period dictionary data and uses it for collation as follows.

  Now, for example, it is assumed that two period dictionary data Dn and Dn + 1 as shown in FIG. 13 are detected as two period dictionary data including the capturing time of the captured face image in the overlapping period. In the lower part of FIG. 13, the mark indicated by a circle on the time axis bar indicates the shooting time of the past face image, and the mark indicated by ☆ indicates the shooting time of the currently captured face image. . In the example of FIG. 13, the dictionary period width is WD, and the overlapping period width is Wov.

  In this embodiment, the number of face images in the first half of the dictionary period width of the two period dictionary data Dn and Dn + 1 is Dn_a and Dn + 1_a, respectively, and the number of face images in the second half is Dn_b and Dn + 1_b, respectively. At this time, the face identification processing unit 12 counts the number of face images Dn_b in the second half of the dictionary period width of the period dictionary data Dn sharing the overlapping period and the number of face images Dn + 1_a in the first half of the dictionary period width of the period dictionary data Dn + 1. The period dictionary data having the larger number of images is used for collation with the feature data of the captured face image.

  In the example of FIG. 13, since Dn_b = 5 and Dn + 1_a = 3, the face identification processing unit 12 uses the period dictionary data Dn for collation with the feature data of the captured face image. decide.

  If the period dictionary data that can be determined to match the feature data of the captured face image is not found as a result of matching between the period dictionary data and the feature data of the captured face image, the CPU 1 In response to the notification from the face identification processing unit 12, a message for allowing the user to input personal identification information such as the personal name of the captured face image on the display 9 through the display information generation unit 13. Is displayed. The CPU 1 instructs the dictionary data update processing unit 14 to newly create the period dictionary data for the period including the photographing time of the face image in response to the personal identification information input according to the message.

  In this case, when the personal identification information input by the user is personal identification information of a new individual Pi that does not yet exist in the face identification dictionary database 10, first, the dictionary data update processing unit 14 In the identification dictionary database 10, an area of the personal dictionary data Di for the new individual Pi is secured, the personal identification information is stored as the personal identification data IDi, and the shooting time of the captured face image is stored in the period dictionary. The reference time data Rt is stored in order to set it as the reference time point for determining the data period.

  Next, the dictionary data update processing unit 14 refers to the period mode of the user period setting information file US preset by the user and stored in the face identification dictionary database 10 together with the dictionary period table data TBL. Determine the width. At this time, if the period mode is the variable length mode, the dictionary data update processing unit 14 generates a message that prompts the user to input the age of the individual Pi at the time of capturing the captured face image. The display information generation unit 13 is instructed to display the image on the display 9. Then, upon receiving the input of the age according to the message, the dictionary data update processing unit 14 determines the dictionary period width from the variable dictionary period table TBLv.

  When the dictionary period width is determined, the dictionary data update processing unit 14 determines a period starting from the reference time stored as the reference time data Rt, and creates period dictionary data using the feature data of the captured face image. Then, it is newly registered in the face identification dictionary database 10.

  When the personal identification information input by the user is the personal identification information of the individual Pi already existing in the face identification dictionary database 10, the personal identification information of the individual Pi is already stored in the face identification dictionary database 10. Dictionary data Di exists and reference time data Rt is stored. Therefore, in this case, the dictionary data update processing unit 14 refers to setting information such as the period mode of the user period setting information file US set in advance by the user in the same manner as described above, and inputs necessary age information. In response, the dictionary period width and the overlap period width are determined, and based on the reference time data Rt, the start and end points of the period for the newly created period dictionary data are determined, and the feature data of the captured face image is obtained. Using this, period dictionary data is created and newly registered in the face identification dictionary database 10.

  At this time, when the photographing time of the captured face image is within the overlap period in the determined dictionary period, the dictionary data update processing unit 14 uses two pieces of period dictionary data sharing the overlap period. Period dictionary data is generated, and the two period dictionary data are newly registered.

  Next, when period dictionary data that can be determined to match the feature data of the captured face image is found, the face identification processing unit 12 instructs the display information generation unit 13 to indicate the period dictionary. The personal identification information of the personal dictionary data to which the data belongs, that is, the personal name in this example is displayed and output as a collation result so that the user can confirm it as described above. As a result of the confirmation, when it is determined that the collation result is correct, the face identification processing unit 12 newly captures the dictionary data during the period determined to match the feature data of the captured face image. The dictionary data update processing unit 14 is instructed to update using the feature data.

  In response to this instruction, the dictionary data update processing unit 14 performs an update process of dictionary data for a period determined to match the feature data of the captured face image.

  Then, the dictionary data update processing unit 14 further includes another period dictionary data that is within the overlapping period of the period dictionary data in which the captured time of the captured face image is updated and that shares the overlapping period. In other words, if one period dictionary data is selected for comparison with the feature data of the captured face image by the method described with reference to FIG. The period dictionary data is also updated using the feature data of the captured face image whose shooting time is within the overlapping period.

  In the first embodiment, the update of the face identification dictionary data FRDi, j is a simple average of the feature data of the face image captured for the period corresponding to the period dictionary data TDi, j.

[Processing in the personal identification device of the first embodiment]
Next, the processing operation in the personal identification device of the first embodiment will be further described.

[Dictionary period parameter setting]
As described above, in the personal identification device of this embodiment, the setting of dictionary period parameters such as the period mode by the user is received in advance. FIG. 14 is a flowchart for explaining the operation in the dictionary period parameter setting process. Processing of each step in the flowchart of FIG. 14 is executed by the CPU 1 using the RAM 4 as a work area according to the program in the ROM 3.

  The CPU 1 receives the user's instruction to start setting the dictionary period parameter and starts the flowchart of FIG. 14. First, the CPU 1 controls the display information generation unit 13 and the display interface 6 to set the dictionary period to the fixed length fixed length mode. A period mode selection screen including a message prompting the selection setting of the period mode to be changed or set to the variable length variable length mode is displayed on the display 9 (step S101).

  Then, the CPU 1 accepts the user's period mode selection input from the operation input unit 8 through the period mode selection screen displayed on the display screen of the display 9 (step S102), and the fixed length mode and the variable length mode are set as the period modes. Are selected (step S103).

  When it is determined in this step S103 that the fixed length mode has been selected, the CPU 1 reads the fixed dictionary period table TBLx from the dictionary period table data TBL of the face identification dictionary database 10, and displays the display information generation unit 13 and the display interface 6. The display 9 displays a period setting screen including a message prompting the user to select the fixed-length dictionary period WD and the overlapping period width Wov according to the table number of the fixed dictionary period table. (Step S104).

  Then, the CPU 1 accepts a selection input of the table number through the period setting screen displayed on the display screen of the display 9 from the operation input unit 8 (step S105), and that the period mode is the fixed length mode and is selected. The obtained table number is stored in the user period setting information file US (step S106). Then, the dictionary period parameter setting processing routine is terminated.

  If it is determined in step S103 that the variable length mode is selected, the CPU 1 stores in the user period setting information file US that the period mode is the variable length mode (step S107). Then, the dictionary period parameter setting processing routine is terminated.

  In this embodiment, the dictionary period width and the overlap period width in the fixed length mode are set as a pair according to the table number, but in the fixed length mode, the dictionary period width and the overlap period width are respectively set. You may enable it to set separately.

  In this embodiment, in the variable length mode, since there is one variable dictionary period table TBLv, it is not necessary to select a table, but a plurality of variable dictionary periods having different dictionary period widths and overlapping period widths for each age are used. A table may be prepared in the dictionary period table data TBL so that the user can select and set from the plurality of variable dictionary period tables.

  Further, in this embodiment, the period mode is set for the entire face identification dictionary database 10, but when the personal dictionary data Di of each individual Pi is first created, it is shown in the flowchart of FIG. It may be configured to allow user setting so that the period mode can be set for each individual Pi.

  In this case, the format of the data stored in the face identification dictionary database 10 is as shown in FIG. 15 instead of FIG. 10, and the user period setting information file US corresponds to the personal identification data ID in accordance with the personal dictionary. It is provided for each data Di.

[Face identification processing and dictionary data generation update processing]
Next, the face identification processing operation and the dictionary data generation update processing operation in the personal identification device having the above-described configuration will be described. FIGS. 16 to 22 are flowcharts for explaining the face identification processing operation and the period dictionary data search processing operation, period dictionary data new registration and update processing operations related thereto. The processing operation described with reference to FIGS. 16 to 22 is performed under the control of the CPU 1. The image data input interface unit 7, the face identification image processing unit 11, the face identification processing unit 12, the display information generation unit 13, the dictionary Each unit of the data update processing unit 14 executes.

  It should be noted that the CPU 1 may be configured to execute the processing of each unit as software processing by using the RAM 4 as a work area based on a software program stored in the ROM 3, and omitting each unit. As described above.

  First, the CPU 1 takes in image data including a face image and information including Exif (Exchange image file format) information accompanying the image data through the image data input interface 7, and transfers the information to the face identifying image processing unit 11 (step S111). . If there is no Exif information attached to the image data of the face image, the user is prompted to input the shooting time, and the shooting time is input to acquire the shooting time information. As described above.

  Based on the control start instruction from the CPU 1, the face identification image processing unit 11 performs face rotation processing based on the face image, eye detection, and detected eye position from the received image data as described above. A feature vector is generated from the plotted data (step S112). At this time, shooting time information is extracted from Exif information attached to the image data and held, or the shooting time input by the user is held.

  Then, when the feature identifying image processing unit 11 succeeds in generating the feature vector, the face identifying image processing unit 11 outputs information indicating that fact and the generated feature vector, and the image data size of the face image is too small to generate the feature vector. When it fails, information indicating that generation of the feature vector has failed is output.

  The CPU 1 determines whether or not the feature vector of the captured face image has been successfully generated based on the information from the face identifying image processing unit 11 (step S113). When it is determined that the generation of the feature vector of the face image has been output from the face identification image processing unit 11 and it is determined that the generation of the feature vector has failed, the information is sent to the display information generation unit 13. Forward.

  When the display information generation unit 13 receives information indicating that the generation of the feature vector of the face image has failed, the display information generation unit 13 generates and outputs the display information of the non-collation notification screen illustrated in FIG. The CPU 1 sends the display information of the non-verification notification screen to the display 9 through the display interface 6, and as shown in FIG. 6, it cannot be collated using the captured face image and is used for the dictionary. A message 29 to the effect is displayed (step S114).

  If the CPU 1 determines in step S113 that information indicating that the facial image feature vector has been successfully generated from the face identifying image processing unit 11 and the generated feature vector have been output, the CPU 1 The face identification processing unit 12 is caused to transfer the information indicating that it has been performed, the generated feature vector, and the shooting time information.

  The face identification processing unit 12 first searches the dictionary database for face identification 10 for period dictionary data corresponding to the period including the received shooting time, and determines whether one or more period dictionary data has been obtained as a result of the search. The determination result is output, and when one or more period dictionary data is obtained, the period dictionary data is loaded and set in the verification memory as the verification period dictionary data (step of FIG. 17). S121).

  The CPU 1 receives the output from the face identification processing unit 12 and determines whether or not one or more period dictionary data for matching is found as a result of the search in the face identification processing unit 12 (step S122). When it is determined that no period dictionary data has been found, a notification to that effect is sent to the display information generation unit 13. Upon receiving this notification, the display information generation unit 13 generates and outputs display information for the period dictionary-less screen as shown in FIG. The CPU 1 sends the display information to the display 9 through the display interface 6, and displays a message 30 that prompts the user to input the name of who the captured face image is (step S123).

  Then, the CPU 1 waits for the input of the name by the user in the name input field 31 in the message 30 (step S135 in FIG. 18), and when it is determined that the input of the name has been accepted, It is determined whether or not there is personal dictionary data Di of the input name (step S136). When it is determined that there is the personal dictionary data Di having the inputted name, the process jumps to step S128 in FIG. 17 to execute the update routine of the face identification dictionary database 10.

  If it is determined in step S136 that the personal dictionary data Di of the input name does not exist in the face identification dictionary database 10, the CPU 1 uses the input name as personal identification information and corresponds to this The dictionary data Di is stored in the face identification dictionary database 10 and newly registered (step S133 in FIG. 18). Then, this processing routine ends.

  The new registration of the personal dictionary data Di in the face identification dictionary database 10 will be described in detail later. The new registration of the personal dictionary data Di to the face identification dictionary database 10 can be executed as a part of the update process of the face identification dictionary database 10 described later.

  In step S122, when it is determined that one or more matching period dictionary data has been found based on the output from the face identification processing unit 12, the CPU 1 sends a collation start instruction to the face identification processing unit 12. In response to this, the face identification processing unit 12 receives the feature vector of the captured face image received from the face identification image processing unit 11 and the period dictionary data retrieved from the face identification dictionary database 10 and loaded into the matching memory. TDi, n is collated to search for period dictionary data TDi, n that can be determined to match. When the face identification processing unit 12 can detect the period dictionary data TDi, n that can be determined to match, the face identification processing unit 12 detects that information and the individual corresponding to the period dictionary data TDi, n that is determined to match. The specific information, in this example, the personal name information is output. When the dictionary data TDi, n cannot be detected for a period that can be determined to match, the information to that effect is output (step S124).

  The CPU 1 receives the output from the face identification processing unit 12, determines whether or not face identification dictionary data FRDi that can be determined to match is detected (step S 125), and a period dictionary that can be determined to match When it is determined that the data TDi, n is detected, the personal name information as the personal identification information corresponding to the period dictionary data TDi, n received from the face identification processing unit 12 and corresponding to The information is transferred to the display information generation unit 13 together with information indicating that the identification has been completed.

  When the display information generating unit 13 receives the information indicating that the face has been identified and the personal name information, the display information generating unit 13 displays a face identification result confirmation screen for allowing the user to confirm the face identification result as illustrated in FIG. Generate and output display information. The CPU 1 sends the display information of the face identification result confirmation screen to the display 9 through the display interface 6, and displays a message 22 for inquiring the user whether the face identification result is correct as shown in FIG. S126).

  Then, the CPU 1 waits for the confirmation operation input through the screen of FIG. 3 by the user, and determines whether or not the confirmation operation input is a correct result of the face identification collation (step S127). When the CPU 1 determines that the confirmation operation input by the user indicates that the collation result is “correct”, the CPU 1 executes an update routine of the face identification dictionary database 10 (step S128). . Then, this processing routine ends. Details of the routine for updating the face identification dictionary database 10 in step S128 will be described later.

  If it is determined in step S127 that the confirmation operation input by the user indicates that the collation result is “incorrect”, information indicating that is sent to the display information generation unit 13. Then, the display information generation unit 13 generates and outputs the display information of the incorrect face identification result screen as shown in FIG. 4, so that the CPU 1 sends the display information to the display 9 through the display interface 6. A message 25 shown in 4 for prompting the user to input his / her name as to which face image the captured face image is is displayed (step S134 in FIG. 18).

  Then, the CPU 1 waits for the input of the name by the user in the name input field 26 in the message 25 (step S135), and when determining that the input of the name has been accepted, the CPU 1 inputs the input to the face identification dictionary database 10. It is determined whether there is personal dictionary data Di of the name (step S136). When it is determined that there is the personal dictionary data Di having the inputted name, the process jumps to step S128 in FIG. 17 to execute the update routine of the face identification dictionary database 10.

  If it is determined in step S136 that the personal dictionary data Di of the input name does not exist in the face identification dictionary database 10, the CPU 1 uses the input name as personal identification information and corresponds to this The dictionary data Di is stored in the face identification dictionary database 10 and newly registered (step S133). Then, this processing routine ends.

  Next, when it is determined in step S125 in FIG. 17 that the dictionary data TDi, n has not been detected for a period that can be determined to match the feature vector of the captured face image, the CPU 1 displays information to that effect as display information. The data is sent to the generation unit 13. When the display information generation unit 13 receives information indicating that the period dictionary data TDi, n that can be determined to match is not detected, the display information generation unit 13 sets “no match” as the face identification result as shown in FIG. Display information of the screen without matching face identification result for informing the user is generated and output. The CPU 1 sends the display information to the display 9 through the display interface 6 and displays a message 27 shown in FIG. 6 urging the user to input the name of who the captured face image is. (Step S131 in FIG. 18).

  Then, the CPU 1 waits for the input of the name by the user in the name input field 28 in the message 27 (step S132), and when it is determined that the input of the name has been accepted, the input name is used as personal identification information. The corresponding personal dictionary data Di is stored in the face identification dictionary database 10 and newly registered (step S133). Then, this processing routine ends.

[Periodic dictionary data search processing]
FIG. 19 shows a flowchart of an example of a period dictionary data search processing routine executed by the face identification processing unit 12 in step S121 in FIG.

  First, the face identification processing unit 12 designates and extracts the first period dictionary data of the first personal dictionary data Di from the face identification dictionary database 10 (step S141). Next, the face identification processing unit 12 determines whether or not all period dictionary data has been extracted from the face identification dictionary database 10 (step S142), and determines that there is no period dictionary data to be extracted this time because all the period dictionary data has been extracted. Sometimes this processing routine is terminated.

  If it is determined in step S142 that all the period dictionary data has not yet been extracted from the face identification dictionary database 10 and it has been determined that the period dictionary data has been extracted from the face identification dictionary database 10, the extracted period dictionary data corresponds to the period. Is detected from the corresponding incidental information, and the correspondence between the detected period and the photographing time of the face image captured this time for personal identification is examined (step S143).

  Then, the face identification processing unit 12 determines whether or not the photographing time of the captured face image is a time within a period corresponding to the period dictionary data (step S144). When it is determined in step S144 that the captured time of the captured face image is not within the period corresponding to the period dictionary data, the face identification processing unit 12 has not finished referring to the face identification dictionary database 10. The next period dictionary data is designated (step S148), the process returns to step S142, and the processes after step S142 are repeated.

  In step S144, when it is determined that the shooting time of the captured face image is within the period corresponding to the period dictionary data, the face identification processing unit 12 sets the shooting time of the captured face image in the period dictionary data. It is determined whether or not it is within the overlapping period of the corresponding period (step S145).

  When it is determined in step S145 that the shooting time is not within the overlapping period, the face identification processing unit 12 sets the period dictionary data for verification (step S146) and stores it in the verification memory (step S147). Then, the face identification processing unit 12 designates the dictionary data for the next period in the face identification dictionary database 10 that has not been referred to (step S148), and then returns to step S142 to perform the processing after step S142. repeat.

  If it is determined in step S145 that the shooting time is within the overlapping period, the face identification processing unit 12 determines whether or not period dictionary data that shares the overlapping period has already been created (FIG. 20). In step S151), if it is determined that it has not been created yet, the period dictionary data is set for verification (step S146) and stored in the verification memory (step S147). Then, the face identification processing unit 12 designates the dictionary data for the next period in the face identification dictionary database 10 that has not been referred to (step S148), and then returns to step S142 to perform the processing after step S142. repeat.

  When it is determined in step S151 that the period dictionary data sharing the overlapping period has already been created and exists, the face identification processing unit 12 uses two period dictionaries as described with reference to FIG. Data Dn and Dn + 1 are compared with the number of images Dn_b and the number of images Dn + 1_a in the respective half-dictionary period widths across the overlap period (step S152).

  When it is determined that Dn_b> Dn + 1_a as a result of the comparison in step S152, the face identification processing unit 12 sets the period dictionary data Dn for collation (step S154), and when it is determined that Dn_b ≦ Dn + 1_a, Period dictionary data Dn + 1 is set for verification (step S155). After step S154 and step S155, the face identification processing unit 12 stores not only the period dictionary data set for collation but also two overlapping period dictionary data in the collation memory (step S156). ). This is because, when updating the period dictionary data later, it is necessary to update not only the period dictionary data set for collation but also two overlapping period dictionary data.

[sign up]
FIG. 21 shows a flowchart of an example of a new registration processing routine executed by the dictionary data update processing unit 14 in step S133 of FIG.

  First, the dictionary data update processing unit 14 determines whether or not the personal identification information input by the user is personal identification information of a new individual Pi that does not yet exist in the face identification dictionary database 10 (Step S1). S161) When it is determined that the information is the personal identification information of the new individual Pi, an area of the personal dictionary data Di for the new individual Pi is secured in the face identification dictionary database 10 (step S162), and the personal identification is performed. Information is stored as personal identification data IDi, and the captured time of the captured face image is stored as reference time data Rt so as to be a reference time for determining the period of the period dictionary data (step S163).

  Next, the dictionary data update processing unit 14 reads the user period setting information file US (step S164), and determines whether the period mode is the fixed length mode or the variable length mode (step S165).

  When it is determined in step S165 that the period mode is the fixed length mode, the dictionary data update processing unit 14 refers to the table number to determine the dictionary period width and the overlapping period width, and the dictionary period width and the overlapping period. Using the width and the reference time data Rt, the dictionary period of the newly created period dictionary data TDi, 1 is set (step S166).

  Subsequently, the dictionary data update processing unit 14 creates period dictionary data TDi, 1 using the feature data extracted from the captured face image, and stores it as face identification dictionary data FRDi, 1 of the personal dictionary data Di. Then, the accompanying information ATRi, 1 is also created and stored (step S167). Then, this processing routine ends.

  When it is determined in step S165 that the period mode is the variable length mode, the dictionary data update processing unit 14 causes the display information generation unit 13 to indicate the age of the individual Pi at the time of capturing the captured face image. Control is performed so that a message prompting the user to input is generated and displayed on the display 9 (step S168).

  Then, upon receiving the input of the age according to the message (step S169), the dictionary data update processing unit 14 determines the dictionary period width and the overlapping period width from the variable dictionary period table TBLv. Then, using the reference time Rt and the determined dictionary period width and overlapping period width, the dictionary period of the newly created period dictionary data TDi, 1 is set (step S170).

  Subsequently, the dictionary data update processing unit 14 creates period dictionary data TDi, 1 using the feature data extracted from the captured face image, and stores it as face identification dictionary data FRDi, 1 of the personal dictionary data Di. Then, the accompanying information ATRi, 1 is also created and stored (step S167). Then, this processing routine ends.

[Registration]
FIG. 22 shows a flowchart of an example of a new registration processing routine executed by the dictionary data update processing unit 14 in step S128 of FIG.

  First, the dictionary data update processing unit 14 refers to the period information and dictionary period parameters of the accompanying information of the period dictionary data TDi, n to be updated, and the dictionary period and overlapping period of the period dictionary data TDi, n to be updated. Is detected (step S181).

  Next, the dictionary data update processing unit 14 determines whether or not the capture time of the captured face image is within the detected overlap period (step S182), and the capture time of the captured face image is overlapped. When it is determined that it is not within the period, one existing period dictionary data to be updated is updated. That is, the feature data of the captured face image is averaged in addition to the previous dictionary data, thereby updating the dictionary data (step S183). Then, this processing routine ends.

  In step S182, when it is determined that the shooting time of the captured face image is within the detected overlap period, the dictionary data update processing unit 14 has already stored two period dictionary data sharing the overlap period. It is determined whether or not it exists (step S184). As described above, in this embodiment, since the existing period dictionary data sharing the overlapping period is stored in the verification memory, the determination is performed by referring to the verification memory. It is possible to search whether there is dictionary data for a period to share

  If it is determined in step S184 that the period dictionary data sharing the overlapping period does not yet exist, the dictionary data update processing unit 14 updates the existing period dictionary data sharing the overlapping period as described above. At the same time, other period dictionary data sharing the overlapping period is newly created and newly registered in the same manner as the above-described new registration (step S185). Then, this processing routine ends.

  When it is determined in step S184 that two period dictionary data sharing the overlapping period already exist, the dictionary data update processing unit 14 stores the existing two period dictionary data sharing the overlapping period. Update processing is performed as described above (step S186). Then, this processing routine ends.

[Modification]
In the above description of the embodiment, the period mode has been described as being set by the user. However, the personal identification device may be set by the manufacturer before shipment from the factory.

  In addition, in the above description, the reference for determining the dictionary period to which the period dictionary data corresponds is the shooting time of the first captured face image for each individual Pi, but is not limited thereto. For example, the birthday of each individual Pi may be input, and the dictionary period may be obtained based on the birthday. Moreover, you may make it ask a user to input on the basis of specific date.

  In addition, although 1st Embodiment described above is a case where this invention is applied to a face identification processing apparatus, it cannot be overemphasized that it can apply not only to face identification but when using a voice as an identification element. .

[Second Embodiment]
In the first embodiment described above, the period dictionary data is obtained by simply averaging feature data extracted from face image data photographed in a corresponding period. However, even during the period corresponding to each period dictionary data, the face and voice of the individual changes with the passage of the identification element time, so the dictionary considering the change of the individual identification element with the passage of the time It is considered that the personal identification performance can be further improved by updating the data.

  In the second embodiment, from such a viewpoint, the feature data extracted from the face image is weighted according to the photographing time, and the weighted feature data is updated in the period dictionary data.

  FIG. 23 is a block diagram illustrating a configuration example of the personal identification device based on face images according to the second embodiment. In FIG. 23, the same parts as those in the block diagram of the configuration example of the personal identification device according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. Also in the second embodiment, a part of what is shown in each block can be a software configuration.

  In the second embodiment, the dictionary data update processing unit 14 includes a weight coefficient calculation unit 15 and a dictionary data generation / update unit 16. Other than that, the hardware configuration is the same as that of the first embodiment.

  In the second embodiment, the data format of the face identification dictionary database 10 is configured as shown in FIG. That is, as each period dictionary data Di, n, in addition to the face identification dictionary data FRDi, n and the accompanying information ATRi, n, in the second embodiment, the dictionary data generation information array ARi, n is used. To be prepared.

  As shown in FIG. 25, the dictionary data generation information array ARi, n is composed of one or more k (k = 1, 2,..., K) dictionary data generation information DPIi, n, k. .

  The dictionary data generation information DPIi, n, k includes k pieces (k = 1 to K, maximum) taken at different points in the period corresponding to the period dictionary data TDi, n of one individual Pi that is a face identification target. In this embodiment, as shown in FIG. 25, time information Ti, n, k and captured face image feature data Vi, n, k The weight coefficient Wi, n, k for the feature data Vi, n, k used for updating the face identification dictionary data using the feature data Vi, n, k of the captured face image is the dictionary. It is included in the data generation information DPIi, n, k. The weighting factors Wi, n, k will be described in detail later.

  In the second embodiment, a maximum of K pieces, for example, 40 pieces of dictionary data generation information can be stored per one period dictionary data TDin. This is in consideration of the storage capacity used as the dictionary database 10. In this embodiment, the data stored as the dictionary data generation information DPIi, n, k always keeps the latest N pieces of data so that the dictionary data generation information DPIi, n, k to be stored is stored. Try to update.

  In the second embodiment, the face identification dictionary data FRDi, n of each period dictionary data TDi, n is associated with the period dictionary data TDi, n in the face identification dictionary database 10 as will be described later. Each feature data (feature vector) of the stored dictionary data generation information DPIi, n, k is weighted by a weighting factor for the feature data, and the weighted data is used as described later. Generate.

  In the second embodiment, as will be described later, a plurality of calculation methods are prepared as the calculation methods (setting methods) of the weighting factors Wi, n, k, and a plurality of users prepared by the user are provided. One of the calculation methods of the weighting factors Wi, n, k can be selected. Then, as shown in FIG. 24, weight coefficient parameter setting information ST is stored in the face identification dictionary database 10 as information for specifying the selected weight coefficient calculation method.

  The weighting factor calculation method may be selected and set for each individual personal dictionary data Di to be identified, but in this embodiment, one weighting factor calculation method is used for the face identification dictionary database 10 as a whole. The selection is set.

  In this embodiment, the dictionary data update processing unit 14 first updates the period dictionary data TDi, n of the personal dictionary data Di corresponding to the specified personal identification information in the face identification dictionary database 10. When the number of dictionary data generation information DPIi, n, k included in the period dictionary data TDi, n is referred to and the number is less than the maximum K, the newly acquired face image feature vector Vin The imaging time information is additionally stored in the period dictionary data TDi, n as the latest dictionary data generation information.

Further, the dictionary data update processing unit 14 has the maximum number of dictionary data generation information DPIi, n, k included in the period dictionary data TDi, n corresponding to the personal identification information to be updated in the face identification dictionary database 10. Is the oldest dictionary data generation information DPIi, 1 among the dictionary data generation information DPIi, n, k at that time, and each dictionary data generation information DPIi, n , K in order,
DPIi, n, k → DPIi, n, k-1
And

  Then, the newly fetched feature vector Vin of the face image and the information on the imaging time are stored as the latest dictionary data generation information DPIi, n, K information, and the period dictionary data TDi, n is updated. At this time, since the weighting factor is not calculated, the stored weighting factor remains unchanged.

  The weighting coefficient is calculated and stored for at least the latest dictionary data generation information DPIi, n, K using information on the imaging time included in the period dictionary data TDi, n updated as described above. . As will be described later, the weighting factors for the dictionary data generation information originally stored as the other period dictionary data TDi, n may or may not be updated according to the selected weighting factor calculation method. It may be used as it is.

  The dictionary data update processing unit 14 passes the weighting factor parameter setting information ST to the weighting factor calculation unit 15 and instructs the weighting factor calculation method executed by the weighting factor calculation unit 15 as will be described later. The calculation unit 15 is passed the imaging time information Ti, n, K of the latest dictionary data generation information DPIi, n, K and necessary past dictionary data generation information DPIi, n, K, and the weighting coefficient is calculated. Instruct.

  In the second embodiment, the weight coefficient calculation unit 15 specifies the period dictionary data TDi, n corresponding to the personal identification information when the personal identification information is identified based on the identification result in the face identification processing unit 13. The weighting coefficient for the feature vector used when updating the face identification dictionary data FRDi, n is calculated using the latest imaging time information and past imaging time information.

  As described above, the personal identification device according to the second embodiment provides a plurality of calculation methods (setting methods) for the weighting coefficient, and in this example, four calculation methods are prepared. Which of the four weight coefficient calculation methods is used is selected and set, and the selection setting information is stored in the face identification dictionary database 10.

  In this embodiment, when the user instructs “selection of weighting factor setting method” from the setting menu, for example, through the operation input unit 8, a “weighting factor setting method selection screen” as shown in FIG. Is displayed.

As shown in FIG. 26, in this embodiment, as a weighting factor calculation method (setting method),
(1) Set weighting factors linearly in the order of imaging,
(2) A weighting factor is set nonlinearly in the order of imaging,
(3) A weighting factor is set according to the imaging interval (section),
(4) A weighting factor is set according to the imaging interval (time difference from the previous image),
Are displayed as a list of weight coefficient calculation method candidates that can be selected and set. Details of each calculation method will be described later.

  As shown in FIG. 26, in this example, a check mark entry field 30 for the user to select and set is provided at the top of each weight coefficient calculation method candidate. The user uses the weighting factor calculation method candidate by clicking on the check column 32 provided at the head of the weighting factor calculation method candidate desired by the user with a pointing device such as a mouse on the list display screen of FIG. It is possible to select and set the weighting coefficient calculation method to be performed. The example of FIG. 26 shows a state where “(1) linearly set weight coefficients in the order of imaging” is selected.

  In this case, depending on the selected weighting factor calculation method, the weighting factor calculating unit 15 calculates only the weighting factor for the feature vector of the newly captured face image, or the newly captured face image. In addition to the weighting coefficient for the feature vector, the weighting coefficient for the feature vector of the face image captured in the past stored corresponding to the personal identification information may be recalculated.

  Next, each of the four types of weighting factor calculation methods used in this embodiment will be further described.

[First example: linearly setting weighting factors in order of imaging (linear calculation method)]
FIG. 27 is a diagram for explaining a method of calculating (setting) weighting factors linearly in the order of imaging. This weighting factor calculation method is hereinafter referred to as a linear calculation method.

  When this linear calculation method is adopted, the weighting factor for the feature data of the personal dictionary data Di stored corresponding to the personal identification information specified based on the identification by the newly captured face image is The latest dictionary data generation information DPIi, n, New (when N pieces of dictionary data generation information are stored, DPIi, n, New = DPIi, n, K) is not only a weight coefficient Wi, New. , All of the weight coefficients Wi, n, k of the dictionary data generation information DPIi, n, k (k = 1, 2,...) Captured and stored in the past are recalculated and updated. .

  For this reason, when this linear calculation method is employed, the dictionary data update processing unit 14 passes the information on the imaging time of all the updated dictionary data generation information to the weighting coefficient calculation unit 15.

  In this linear calculation method, in this embodiment, the weighting factor calculation unit 15 first captures the imaging time Ti, n of the oldest dictionary data generation information in the updated period dictionary data TDi, n. , 1 as a reference, the imaging times Ti, n, k (k = 1, 2,..., New) of the dictionary data generation information of the updated period dictionary data TDi, n A time difference ΔTi (1_k) from the old imaging time Ti, n, 1 is calculated.

  Then, as shown in FIG. 10, the weight coefficient calculation unit 15 linearly increases the weight coefficient Wi, n, k as the calculated time difference ΔTi (1_n) increases as ).

That is, the weight coefficient calculation unit 15
Wi, n, k = p × ΔTi (1_k) + q (Expression 1)
Thus, the weighting coefficient Wi, n, k for each feature vector Vi, n, k is calculated.

  In FIG. 27, the time difference ΔT (1_1) = 0, and the weighting coefficient Wi, n, 1 at this time is the minimum value q of the predetermined weighting coefficient. In addition, when the maximum value of the predetermined weighting factor is r, the weighting factors Wi, n, New at the time difference ΔTi (1_New) with respect to the latest imaging time are set to Wi, n, New = r. .

Further, the slope p of the straight line of the linear function of FIG.
p = (r−q) / (ΔTi (1_New) −ΔTi (1_1))
And according to the difference (Ti, n, New-Ti, n, 1) between the latest imaging time and the oldest imaging time among the plurality of updated dictionary data generation information DPIi, n, k. It is not constant and changes each time new data is fetched.

Here, in the face identification dictionary database 10, K pieces of dictionary data generation information DPIi, n, 1 to DPIi, n, K for each period dictionary data TDi, n stored in correspondence with the personal identification information are stored. If stored, the time difference ΔTi (1_New) is
Time difference ΔTi (1_New) = ΔTi (1_K) = Ti, n, K−Ti, n, 1
It is. However, even if it is newly added in the face identification dictionary database 10, if less than K pieces of dictionary data generation information corresponding to the personal identification information are stored, the less than K pieces of dictionary data generation information are generated. The imaging time of the latest information is the latest imaging time, and the time difference between the imaging time and the oldest imaging time Ti, n, 1 is the time difference ΔTi (1_New) for the latest imaging time.

  In the above description, the imaging times Ti, n, 1 of the dictionary data generation information are based on the imaging times Ti, n, 1 of the oldest dictionary data generation information in the updated period dictionary data TDi, n. The time difference ΔT (1_k) with respect to n, k is calculated, and the weight coefficient Wi, n, k is set so as to increase linearly as the time difference ΔT (1_k) increases. The imaging times Ti, n, and New may be used.

  In this case, when the time difference ΔT (New_New) (= 0), the weight coefficient Wi, n, New becomes the maximum value r, and becomes linearly smaller as the time difference ΔT (New_k) increases, and the oldest imaging time Ti. , N, 1, the weight coefficient Wi, n, 1 becomes the minimum value q.

[Second Example: Setting Weighting Factor Nonlinearly in Order of Imaging (Nonlinear Calculation Method)]
FIG. 28 is a diagram for explaining a method of calculating (setting) weighting coefficients nonlinearly in the order of imaging. Hereinafter, this weighting factor calculation method is referred to as a nonlinear calculation method.

  Even when this non-linear calculation method is adopted, the weight for the feature data of the period dictionary data TDi, n stored corresponding to the personal identification information specified based on the identification by the newly captured face image The coefficient is the weight coefficient Wi, n, of the latest dictionary data generation information DPIi, n, New (when K dictionary data generation information is stored, DPIi, n, New = DPIi, n, K). Not only New, but all the weighting factors Wi, n, k of dictionary data generation information DPIi, n, k (k = 1, 2,...) Captured and stored in the past are recalculated. Updated.

  For this reason, also when this nonlinear calculation method is adopted, the dictionary data update processing unit 14 passes the information on the imaging time of all the updated dictionary data generation information to the weight coefficient calculation unit 15.

  Also in this nonlinear calculation method, in this embodiment, the weighting factor calculation unit 15 first captures the imaging time Ti, n, 1 of the oldest dictionary data generation information among the updated period dictionary data TDi, n. Is based on the imaging time Ti, n, k (k = 1, 2,..., New) of the dictionary data generation information of the updated period dictionary data TDi, n and the oldest A time difference ΔTi (1_n) from the imaging time Ti, n, 1 is calculated.

  Then, as shown in FIG. 28, the weighting factor calculation unit 15 sets the weighting factors Wi, n, and k so as to increase nonlinearly as the calculated time difference ΔTi (1_k) increases. ).

That is,
When the time difference ΔTi (1_k) <ΔTθ,
Wi, n, k = f (ΔTi (1_k)) + q
When the time difference ΔTi (1_k) ≧ ΔTθ,
Wi, n, k = r
... (Formula 2)
Thus, the weighting coefficient Wi, n, k for each feature vector Vi, n, k is calculated.

  F (ΔTi (1_k)) in (Expression 2) is a non-linear function with the time difference ΔTi (1_k) as a variable, and a variety of non-linear change pattern curve functions can be used. In the example of FIG. 28, the weight coefficient of the feature vector of the past face image is reduced to suppress its influence on the face identification dictionary data, and the weight coefficient is increased as the feature vector of the recent face image is increased. A curve function is used to increase its influence on dictionary data.

  As can be seen from FIG. 28 and (Equation 2), in the nonlinear calculation method of this embodiment, the time difference ΔT (i_New) corresponding to the latest imaging time Ti, n, New is smaller by a predetermined threshold time difference ΔTθ. With the time difference interval PT as the priority interval, the weighting factors Wi, n, k for the time difference ΔT (1_k) included in the priority interval PT are set to the maximum value r. In other words, the feature vector of the recent face image is used as the maximum weight coefficient so that the influence on the face identification dictionary data is further increased.

  In this embodiment, the user can adjust and set the threshold time difference ΔTθ. That is, in this embodiment, when the second “non-linear setting in imaging order” is selected by the user on the selection setting screen of the weighting factor setting method shown in FIG. When the CPU 1 determines that the corresponding period dictionary data TDi, n is to be updated, the display information generation unit 13 generates and displays the “priority section PT adjustment” screen shown in FIG. 12 according to the control of the CPU 1. 9 is displayed.

  In this “priority section PT adjustment” screen, as shown in FIG. 29, updated dictionary data generation information DPIi, n, k stored as period dictionary data TDi, n to be updated is displayed. A priority section PT adjustment column 34 is provided so that the arrangement of the imaging times included in each is displayed in the imaging time sequence column 33 and can be adjusted in accordance with the imaging time sequence.

  In the imaging time array column 33, the horizontal direction is the time direction, and the imaging times are indicated by vertical lines 33a. Further, the priority section PT adjustment column 34 displays the time difference ΔT (1_k) from the above-mentioned oldest imaging time Ti, n, 1 as a horizontal bar corresponding to the imaging time array column 33 with the horizontal direction as the time difference direction. It is. In the priority section PT adjustment column 34, the user can set the priority section PT and therefore the threshold time difference ΔTθ by moving the position of the adjustment point mark 35 in the horizontal direction (time difference direction). Has been.

  Of course, the priority section PT may be zero and not necessarily provided.

  In this non-linear calculation method, similarly to the above-described linear calculation method, the imaging time Ti, n, 1 of the oldest dictionary data generation information in the updated period dictionary data TDi, n is used as a reference. Instead, the time difference may be calculated based on the latest imaging time Ti, n, New.

  In this case, the weighting factor setting characteristic curve is obtained by using the center time difference position between the latest time difference and the oldest time difference as the symmetrical axis position, and the characteristic curve shown in FIG. It is the same as that reversed in line symmetry so that the position with respect to the time difference is reversed.

  In the above description, the priority section PT is set before the personal identification information is specified and the weighting coefficient is calculated. In the weighting coefficient setting method selection setting screen shown in FIG. When the second “non-linear setting in imaging order” is selected by the user, the size of the priority section PT may be set by the user. In this case, the user does not need to reset the priority section PT every time the weighting factor is updated.

  In the linear calculation method of the first example described above, when the time difference ΔT (1_k) is equal to or larger than the predetermined time difference ΔTθ, the value of the corresponding weight coefficient Wi, n, k is set to the maximum value. A priority section PT that is r may be provided.

[Third example: Weighting coefficient is set according to imaging interval (section)]
Normally, when the imaging interval is close (the imaging time is close), there is not much change in the individual face captured at those imaging times, and it is considered that their feature vectors are almost unchanged. On the other hand, when the imaging interval is far away, there is a possibility that the individual's face has changed, and it is considered that there is a change in their feature vectors.

  In this third example, in view of this, the weighting factor is set based on the imaging interval. When the imaging interval is short, the influence of one face image on the face identification dictionary data is reduced. Therefore, the weighting coefficient is set to a small value, and when the imaging interval is long, the weighting coefficient is set to a large value in order to increase the influence of one face image on the face identification dictionary data.

  There are various methods for determining whether the imaging interval is short or long, but in the weight coefficient calculation method in the case of the third example, the period including the oldest and latest imaging time is divided into a plurality of sections. Then, the number of times of capturing face image data (number of times of capturing image capturing time) in each divided section is calculated. If the capturing number is small, it is determined that the capturing interval is long, and if the capturing interval is large, it is determined that the capturing interval is short. A weighting factor corresponding to the imaging time included in each divided section is set according to the determination result.

  Even when this third example is adopted, the feature data of the period dictionary data TDi, n stored corresponding to the personal identification information specified based on the identification by the newly captured face image The weighting factor is the weighting factor Wi, n of the latest dictionary data generation information DPIi, n, New (when K dictionary data generation information is stored, DPIi, n, New = DPIi, n, K). , New, and all of the weighting factors Wi, n, k of dictionary data generation information DPIi, n, k (k = 1, 2,...) Captured and stored in the past are recalculated. Has been updated.

  Therefore, also when this third calculation method is adopted, the dictionary data update processing unit 14 passes the information on the imaging time of all the updated dictionary data generation information to the weight coefficient calculation unit 15. The weighting factor calculation unit 15 calculates the corresponding weighting factor using the information on the imaging time.

  FIG. 30 is a diagram for explaining a weighting factor calculation method in the case of the third example. FIG. 30A is a diagram illustrating a distribution of imaging times when time is taken in the horizontal direction, and each of the circles indicates the imaging time. FIG. 30B shows an example of the weighting coefficient for each divided section.

  That is, in the third example, the divided section is set for each predetermined time length t with the oldest imaging time Ti, n, 1 of the updated dictionary data generation information as a reference. In the example shown in FIG. 30B, four divided sections DV1, DV2, DV3, DV4 are set.

  Next, the number of imaging times included in each divided section DV1, DV2, DV3, DV4 is counted. Then, the weighting coefficient for each of the divided sections DV1, DV2, DV3, DV4 is determined so that the smaller the count result value is, the larger the weighting coefficient is.

  In this case, in determining the weighting factor corresponding to the number of imaging times included in each of the divided sections DV1, DV2, DV3, DV4, a method of linearly assigning weighting factors to the number of imaging times, the number of imaging times Various methods such as a method of assigning weight coefficients in a non-linear manner can be used.

  Then, it is determined in which divided section the imaging time Ti, n, k of each dictionary data generation information DPIi, n, k is included, and the weighting factor set in the divided section of the determination result is The weighting factors Wi, n, k corresponding to the imaging times Ti, n, k are calculated.

  Note that the predetermined predetermined time length t may be fixedly set, but may be adjusted and set by the user in the same manner as the adjustment setting of the priority section PT described above.

  Further, as in the above-described example, the time interval from the oldest imaging time to the latest imaging time is not a predetermined interval for each predetermined section length with reference to the oldest imaging time. Dividing sections may be set so as to be divided into numbers. When setting such a divided section, the section from the time before the oldest imaging time by the margin time to the time after the latest imaging time by the margin time is divided into a predetermined number. May be.

[Fourth Example: Setting a weighting factor according to the imaging interval (time difference from the previous image)]
The weighting factor calculation method of the fourth example is a method of calculating a weighting factor according to the imaging interval as in the third example, and is the latest method for determining whether the imaging interval is short or long. The time difference between the imaging time Ti, n, New and the previous imaging time is used.

  That is, in the case where the weighting factor calculation method of the fourth example is adopted, the period dictionary data stored corresponding to the personal identification information specified based on the identification by the newly captured face image When TDi, n is updated as described above, the dictionary data update processing unit 14 uses the latest imaging time for the newly captured face image and the previous imaging time as the weighting coefficient calculation unit 15. To pass.

  The weighting factor calculation unit 15 calculates a time difference ΔT ((New-1) _New) between the latest imaging time Ti, n, New and the previous imaging time from the information of the two received imaging times. . Then, a weight coefficient Wi, n, New corresponding to the calculated time difference ΔT ((New-1) _New) is calculated.

  In the case of the weighting factor calculation method of the fourth example, only the weighting factors Wi, n, New corresponding to the latest imaging times Ti, n, New need be calculated, and other past imaging times are supported. As the weighting factor, the weighting factor stored as the dictionary data generation information can be used as it is, and it is not necessary to update them.

  In the fourth example, as a calculation method of the weighting factors Wi, n, New corresponding to the calculated time difference ΔT ((New-1) _New), either a linear calculation method or a non-linear calculation method is used. be able to.

  FIG. 31 shows an example in which, in the fourth example, as a method for calculating the weighting factors Wi, n, New corresponding to the calculated time difference ΔT ((New-1) _New), a linearly determining method is employed. Indicates.

In other words, in the example shown in FIG.
When the time difference ΔT ((New-1) _New) <MAX_INT,
Wi, n, k = p × ΔTi (1_k) + q
When the time difference ΔT ((New-1) _New) ≧ MAX_INT,
Wi, n, k = r
... (Formula 3)
Thus, the weight coefficients Wi, n, New for the feature vector of the newly captured face image (when K or more pieces of dictionary data generation information are stored as the period dictionary data TDi, n, Wi, n , New = Wi, n, K).

  Here, the threshold value MAX_INT corresponds to a relatively long time interval in which it is considered that there is a change between the previously captured face image and the newly captured face image, and is greater than this threshold value MAX_INT. If the calculated time difference ΔT ((New-1) _New) is large, the weighting coefficient is maximized so as to strongly influence the face identification dictionary data. In this example, the threshold value MAX_INT is, for example, 24 hours.

[Updating face identification dictionary data FRDi, n]
As described above, when the weight coefficient calculation unit 15 finishes updating the weight coefficient Wi, n, k for the updated dictionary data generation information DPIi, n, k, the dictionary data update processing unit 14 The calculated weighting factors Wi, n, k are used as the updated weighting factors Wi, n for each dictionary data generation information DPIi, n, k of the period dictionary data TDi, n corresponding to the personal identification information of the face identification dictionary database 10. The data is stored as n, k and is also transferred to the dictionary data generation / update unit 16 to generate or update face identification dictionary data FRDi, n.

  The dictionary data generating / updating unit 16 updates the face identification dictionary data FRDi, n of the period dictionary data TDi, n corresponding to the specified individual specifying information by the calculation formula shown in (Formula 4) of FIG. To do. That is, the dictionary data generating / updating unit 16 includes the feature vectors Vi, n, k included in each of the maximum K pieces of dictionary data generating information DPIi, 1 to DPIi, K and the corresponding weight coefficients Wi, n, k. As a result of dividing the sum total obtained by multiplying by the sum of K weighting factors Wi, n, k, updated face identification dictionary data FRDi, n is obtained.

  Conventionally, the face identification dictionary data FRDi, n (corresponding to the feature vector) is stored in N pieces of dictionary data generation information DPIi, n, 1 to DPIi, n, as shown in (Equation 5) of FIG. Compared to a value obtained by a simple average obtained by dividing the total sum of feature vectors included in K by the number K, N dictionary data generation information DPIi, n, 1 to DPIi, n, N weighting factors Wi, n and k are used, and the accuracy of the face identification dictionary data FRDi, n is improved.

  In this embodiment, the latest dictionary data generation information DPIi, n, 1 to DPIi, n, K, which is a finite number, always remains in the face identification dictionary database 10. Therefore, the face identification dictionary data FRDi, n is updated with higher accuracy.

  When the face identification dictionary database 10 has no more than K pieces of dictionary data generation information DPIi, n, k as the period dictionary data TDi, n, the dictionary data update processing unit 14 newly captures the face The feature vector, the acquisition time information, and the weighting coefficient Wnew about the image are additionally stored and updated in the face identification dictionary database 10 as the latest dictionary data generation information DPIi, n, k, and the face identification dictionary data FRDi, n Execute the generation or update process.

  In generating the face identification dictionary data FRDi, n in that case, the dictionary data generating / updating unit 16 stores the period dictionary data TDi in the face identification dictionary database 10 instead of K in (Equation 4). , N is used to generate or update face identification dictionary data FRDi, n, using the number of dictionary data generation information DPIi, n, k stored as n.

  Then, when K pieces of dictionary data generation information DPIi, n, k are stored as the period dictionary data TDi, n in the face identification dictionary database 10, the dictionary data update processing unit 14 performs the processing as described above. Discard the oldest dictionary data generation information and store the newly captured data of face image feature data, imaging time information, and weight coefficient information as the latest dictionary data generation information Then, the dictionary data generation information DPIi, n, k constituting the period dictionary data TDi, n is updated, and the respective weight coefficients Wi, n, k of the updated dictionary data generation information DPIi, n, k are set. The used face identification dictionary data FRDi, n is updated.

  In the block configuration of FIG. 23 described above, the dictionary data update includes the face identification image processing unit 11, the face identification processing unit 12, the display information generation unit 13, the weight coefficient calculation unit 15, and the dictionary data generation / update unit 16. Each unit of the processing unit 14 may have a hardware configuration, or the CPU 1 may be configured as a software function unit that executes software processing using the RAM 4 as a work area based on a software program stored in the ROM 3. You may do it.

  In the above configuration, the dictionary data generation information DPIi, n, k includes at least a feature vector, time information, a total score, and a weighting factor. However, the weighting factor is not stored. When the face identification dictionary data FRDi, n is updated, it may be calculated from the latest captured imaging time and the stored past imaging time. However, when the weighting factor calculation method of the fourth example is adopted, since the weighting factor calculated in the past is not updated, it is better to save the weighting factor.

  Further, instead of the feature vector, the captured face image information may be stored, and the feature vector may be extracted again from the stored face image information. In this case, the dictionary data generation information DPIi, n, k may include at least face image information and imaging time information.

[Processing in the personal identification device of the second embodiment]
Next, the update processing operation and the weighting factor calculation operation of the face identification dictionary database in the personal identification device of the second embodiment having the above-described configuration will be described.

[Update processing of face identification dictionary database]
FIG. 33 is a flowchart of an example of a process for updating the period dictionary data TDi, n in the personal dictionary data Di of the face identification dictionary database 10 according to the second embodiment. In this embodiment, each processing step in FIG. 33 is executed by the dictionary data update processing unit 14 including the weight coefficient calculation unit 15 and the dictionary data generation / update unit 16 according to the control of the CPU 1. The CPU 1 may be configured to execute the process of each unit as a software process by using the RAM 4 as a work area based on a software program stored in the ROM 3, and the above units may be omitted. Is as described above.

  In the following description, for the sake of simplicity, the dictionary data generation information DPIi, n, k of the period dictionary data TDi, n is a process after K pieces are stored.

  First, the dictionary data update processing unit 14 reads all the dictionary data generation information DPIi, n, k of the period dictionary data TDi, n corresponding to the specified individual specifying information, and information DPIi of the oldest imaging time. , N, 1 are discarded, and the arrangement of the dictionary data generation information DPIi, n, k is sequentially shifted to the one with the earlier imaging time one by one. Then, the newly acquired feature vector and imaging time information are stored again in the face identification dictionary database 10 as the latest information DPIi, n, K (step S201).

  Next, the dictionary data update processing unit 14 reads the weighting factor parameter setting information ST in the face identification dictionary database 10 and passes it to the weighting factor calculation unit 15 to determine which weighting factor calculation method is employed. The weighting coefficient calculation unit 15 is notified (step S202).

  Upon receiving this weighting factor parameter setting information ST, the weighting factor calculation unit 15 first sets the weighting factor calculation method selected by the user to “weighting factor setting (previous image according to imaging interval)” in the above-described fourth example. It is determined whether or not the time difference between the two (step S203).

  When it is determined in step S203 that the weighting factor calculation method selected by the user is not the above-described fourth example, the weighting factor calculation unit 15 generates the dictionary data for the updated period dictionary data TDi, n. All the imaging time information Ti, n, k (k = 1 to K) of the information DPIi, n, k is acquired, and all the weighting factors Wi, n, k (k = 1 to K) are recalculated. Update (step S204).

  After step S204, the dictionary data generation / update unit 16 acquires the weighting factors Wi, n, k (k = 1 to K) generated in step S204, and the acquired weighting factors Wi, Each of the corresponding feature vectors Vi, n, k is weighted by n, k (k = 1 to K), and the face identification dictionary data FRDi, n is obtained by using (Equation 4) of FIG. 32 described above. Is generated or updated, and the generated or updated face identification dictionary data FRDi, n is written back to the face identification dictionary database 10 as face identification dictionary data FRDi, n corresponding to the personal identification information. (Step S206).

  When it is determined in step S203 that the weighting factor calculation method selected by the user is the fourth example described above, the weighting factor calculation unit 15 uses the dictionary data generation information for the updated personal dictionary data Di. Of the DPIi, n, k, the latest imaging time information Ti, n, K and the previous imaging time information Ti, n, (K-1) are acquired, and the latest feature vector Vi, The weighting factors Wi, n and K for n and K are calculated (step S205).

  Following this step S205, the dictionary data generating / updating unit 16 acquires the weighting factors Wi, n, K generated in step S204, and the weighting factors Wi, n of all past dictionary data generation information. , K (k = 1 to (K−1)) are used to weight each of the corresponding feature vectors Vi, n, k, and using (Equation 4) of FIG. 32 described above. The face identification dictionary data FRDi, n is generated or updated, and the generated or updated face identification dictionary data FRDi, n is stored in the face identification dictionary database 10 in the face identification dictionary corresponding to the personal identification information. Data is written back as FRDi, n (step S206).

  Note that when the period dictionary data TDi, n is newly registered, the weight coefficient is calculated and stored as a part of the dictionary data generation information DPIi, n, k. Since it is the same as that of the embodiment, its detailed description is omitted.

[Recalculation of weighting factors]
Next, the weighting factor calculation process shown as step S204 and step S205 in FIG. 33 will be described with reference to FIG. 34 and FIG. In FIG. 33, the processing procedure is described from the viewpoint of recalculating the weighting factors for all feature vectors or calculating the weighting factors only for the feature vectors captured at the new imaging time. 34 and 35 show an example of the case where the processing is executed by sequentially determining what the weighting coefficient parameter setting information ST is.

  In the following description, for the sake of simplicity, the dictionary data generation information DPIi, n, k of the period dictionary data TDi, n is a process after K pieces are stored.

  That is, as shown in FIG. 34, the weighting factor calculation unit 15 first determines whether or not the weighting factor calculation method selected by the user is a method for calculating the weighting factor according to the imaging order (step S301). .

  When it is determined in step S301 that the weighting coefficient is calculated in accordance with the imaging order, the weighting coefficient calculating unit 15 determines all the dictionary data generation information DPIi, n, k of the updated period dictionary data TDi, n. The imaging time information Ti, n, k (k = 1 to K) is obtained, and the oldest imaging time Ti, n, 1 and the respective imaging times Ti, n, k (k = 1 to K) are obtained. The time difference ΔTi (1_k) (k = 1 to K) is calculated (step S302).

  Then, the weighting factor calculation unit 15 determines whether or not the weighting factor setting method in the imaging order is a method of linear setting (step S303), and when determining that the method of linear setting is linear, As described above as the calculation method, the weight coefficient Wi, n, k for each feature vector Vi, n, k is calculated by the above (formula 1), and the calculated weight coefficient Wi, n, k is generated as dictionary data. The data is passed to the update unit 16 (step S304). Then, this processing routine ends.

  If it is determined in step S303 that the weighting factor setting method in the imaging order is a nonlinear setting method, the weighting factor calculation unit 15 uses the (Equation 2) as described above as the nonlinear calculation method. The weight coefficients Wi, n, k for the feature vectors Vi, n, k are calculated, and the calculated weight coefficients Wi, n, k are passed to the dictionary data generation / update unit 16 (step S305). Then, this processing routine ends.

  When it is determined in step S301 that the weighting factor calculation method selected by the user is not a method for calculating the weighting factor according to the imaging order, the weighting factor calculation unit 15 determines that the weighting factor calculation method selected by the user is Then, it is determined whether or not it is set (difference from the previous image) according to the imaging interval of the fourth example described above (step S311 in FIG. 35).

  In step S311, the weighting factor calculation method selected by the user is not set according to the imaging interval of the fourth example described above (difference from the previous image) but according to the imaging interval of the third example described above. When it is determined that it is a setting (section), the weighting factor calculation unit 15 sets a plurality of divided sections as described with reference to FIG. 30 described above, and for each set divided section, the divided section The number k of imaging times included in is calculated (step S312).

  Next, the weighting factor calculation unit 15 sets a weighting factor for each divided section according to the calculated number of imaging times (step S313). Next, the weighting factor calculation unit 15 determines which division section each of the imaging times Ti, n, k belongs to, and the weighting factor corresponding to each of the imaging times Ti, n, k based on the determination result. Wi, n, k are calculated (step S314). Then, this processing routine ends.

  Each of the calculated weighting factors Wi, n, k is transferred to the dictionary data generation / update unit 16 and stored as information of the dictionary data generation information DPIi, n, k.

  In step S311, when it is determined that the weighting factor calculation method selected by the user is set (difference from the previous image) according to the imaging interval of the fourth example described above, the weighting factor calculation unit 15 Then, a time difference ΔTi (K−1_K) between the imaging time Ti, n, K of the newly captured face image and the previous imaging time Ti, n, (K−1) is calculated (step S315). .

  Then, the weighting factor calculation unit 15 calculates the weighting factors Wi, n, K for the feature vector of the face image captured at the new imaging time Ti, n, K according to the calculation formula of (Formula 3) described above. (Step S316). Thus, this processing routine is finished.

  The calculated weighting factors Wi, n, K are transferred to the dictionary data generation / update unit 16 and stored as dictionary data generation information DPIi, n, K information.

  In the above description, after personal identification is performed, the face identification dictionary data is updated using the new captured face image following the personal identification. Image data or feature vectors, time information, and weighting factors may be temporarily stored, and the face identification dictionary data may be updated at an appropriate time after the personal identification time.

[Effects of Second Embodiment]
As described above, in the above-described embodiment, when updating the face identification dictionary data FRDi, n of the period dictionary data TDi, n, each of the maximum K pieces of dictionary data generation information DPIi, n, k is calculated. Rather than taking the simple average of feature vectors, the sum of the feature vectors multiplied by the weighting factor that increases the most recently calculated based on the imaging time is divided by the sum of the weighting factors. Since each feature vector contributes to generation of the face identification dictionary data FRDi, n according to the imaging time, it is possible to obtain more accurate dictionary data.

[Modification of Second Embodiment]
The above description of the second embodiment is a case where the present invention is applied to a face identification processing device, but the scope of application of the present invention is not limited to the case of a personal identification device using such a face image. Needless to say.

  For example, in the case of individual identification processing, when identifying an individual as an identification element that changes over time, such as a human voice, the weighting factor is calculated based on the capture time when the identification element is incorporated. Thus, the present invention can also be applied when updating personal identification dictionary data.

[Third Embodiment]
Also in the case of the third embodiment, the period dictionary data is not a simple average of the feature data extracted from the face image data photographed in the corresponding period, but improves the individual identification performance. To be able to.

  In the identification of a face image, the size of the face image is, for example, a size of vertical × horizontal = 64 × 64 pixels or more, the face image is facing the front, both eyes are aligned horizontally, etc. It is necessary to satisfy the reference condition as an image to be identified.

  However, actually, the captured face image does not satisfy all the above-mentioned reference conditions.

  In the face image identification processing, when the reference condition is not satisfied, the image data of the face image that has been subjected to image correction processing such as image data interpolation processing or image rotation processing is applied so that the reference condition is satisfied as much as possible. By applying to this, face identification processing is enabled.

  Conventionally, when the captured face image can be identified personally, the feature data of the face image is generated and updated as valid feature data having all equal weights. Yes. That is, conventionally, the feature data of the face image that does not satisfy the reference condition as described above and the feature data of the face image that satisfies the reference condition are treated as data having the same weight, and the dictionary data is generated and updated. is doing.

  In this way, when the captured face image can be identified, the feature data is all used with the same weight, and the dictionary data of each personal face image is created and updated. The accuracy of the updated dictionary data may be impaired.

  The above is not limited to the generation and update of face image dictionary data, but the captured image is processed with the same weight even though there is a predetermined reference condition for the captured image. Present in all cases of doing.

  The same can be said for the voice information when the information as an individual identification element to be captured is not only a face image but also an identification element such as a voiceprint.

  In view of the above points, the third embodiment is used when individual identification is performed using an identification element such as a face image by performing processing with reference to a predetermined standard condition for information to be captured. As dictionary data, more accurate dictionary data can be created and / or updated.

  FIG. 36 is a block diagram showing a configuration example of the personal identification device based on face images of the third embodiment. 36, the same parts as those in the block diagram of the configuration example of the personal identification device according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. Also in the third embodiment, a part of what is shown in each block can be a software configuration.

  In the third embodiment, a score calculation unit 17 for calculating a score, which is information on the degree of reliability (reliability) in performing face identification, is provided connected to the system bus and is used for face identification. Information from the image processing unit 11 is supplied to the score calculation unit 17. In terms of hardware, other configurations are the same as those of the first embodiment shown in FIG.

  Also in the third embodiment, as in the second embodiment described above, each of the period dictionary data TDi, n of the personal dictionary data Di of the face identification dictionary database 10 is a dictionary data generation information array. ARi, n is provided.

  Then, in the third embodiment, each of the maximum K pieces of dictionary data generation information DPIi, n, k constituting the dictionary data generation information array ARi, n is shown in FIG. To update the face identification dictionary data using the information Ti, n, k, the feature data Vi, n, k of the captured face image, and the feature data Vi, n, k of the captured face image The weight coefficient Wi, n, k for the feature data Vi, n, k to be used and the total score SOi, n, k are included.

  In the second embodiment, the weighting factors Wi, n, k are calculated based on the shooting time of the captured face image, but in this third embodiment, as will be described later, the total score SOi, n , K.

  In the third embodiment, except for the point of calculating the total score SOi, n, k and the point that the weighting coefficient Wi, n, k is calculated based on the total score SOi, n, k, The same configuration and processing operation as in the second embodiment are performed.

  Also in the third embodiment, a maximum of K pieces, for example, 40 pieces of dictionary data generation information DPIi, n, k can be stored for each period dictionary data TDi, n. This takes into account the storage capacity used as a dictionary database. In the third embodiment, the data accumulated as the period dictionary data TDi, n is referred to the overall score to be described later so that only the data with a high overall score is left (that is, stored), The accumulated period dictionary data TDi, n is updated.

  In the third embodiment, the period dictionary data TDi, n is not necessarily newly created or updated when a new face image is captured, but the period dictionary data TDi. , N is determined based on the total score, and it is determined that it is better to newly create or update the period dictionary data. A new creation or update process of TDi, n is executed.

  In the third embodiment, the face identifying image processing unit 11 supplies the face calculating result face image data, the eye detecting result data, and the like to the score calculating unit 15. In this example, the image data size of the detected face image is added to the face image data of the face detection result supplied from the face identifying image processing unit 11 to the score calculation unit 15. Here, the image data size of the face image is composed of the number of pixels in the vertical direction (number of pixels) of the face image of the face detection result and the number of pixels in the horizontal direction (number of pixels) of the face image.

  In this example, the score calculation unit 17 receives the face detection result data and the eye detection result data from the face identification image processing unit 11, and extracts the face image extracted from the image captured through the image data input interface 7. A score, which is information on the degree of reliability (reliability) in performing face identification, is calculated. When receiving a notification from the face identification image processing unit 11 that face detection, eye detection, and finally feature vector calculation has failed, the score calculation unit 17 obtains a score for the captured face image. The calculation process is not performed.

  In this embodiment, the score calculation unit 17 uses the score, which is information on the degree of reliability in performing face identification, for the captured face image as a deviation from the reference condition of the face image for face identification. Calculate based on detection.

Here, the reference condition of the face image for face identification is, in this example,
(A) The face image data includes a predetermined number of pixels at least in the horizontal and vertical directions. In this embodiment, for example, the face image data having a data size of horizontal × vertical = 64 pixels × 64 pixels or more (B) the face is facing the front (facing the front)
(C) It can be mentioned that both eyes are aligned horizontally.

  Based on the above reference conditions, in this embodiment, four scores Sa, Sb, Sc, and Sd are calculated for the feature data of the captured face image. In this embodiment, the total score SO is calculated for the feature data of the captured face image from the four calculated scores Sa, Sb, Sc, and Sd. Each value of the scores Sa, Sb, Sc, and Sd is represented by “0” to “1”. When the score value is “0”, it means that there is no reliability, and the score value is “ As the value approaches “1”, the reliability becomes higher. When the score value is “1”, the reliability is sufficiently high.

[Score Sa; Data size of face image]
The score Sa among the four scores relates to the data size of the face image detected from the captured image.

  When the minimum face image data size required for face identification is MIN_SIZE and the face image data size MAX_SIZE is sufficient for face identification processing based on the reference condition (A) for the face image for face identification In this embodiment, the score Sa is calculated as a value having a relationship as shown in FIG. 38 with respect to the data size Xa of the captured face image.

That is,
Sa = 0 when Xa <MIN_SIZE.
When MIN_SIZE ≦ Xa ≦ MAX_SIZE, Sa = k · Xa + q (q <0). However, k is a proportionality constant where k> 0.
When MAX_SIZE <Xa, Sa = 1.

  Here, in this embodiment, MIN_SIZE is selected to be equal to the data size of horizontal × vertical = 64 pixels × 64 pixels, for example. Further, MAX_SIZE is a data size that is larger than the data size of 64 pixels × 64 pixels and that can sufficiently identify the face image.

  Here, the data size of the captured face image is the data size of the face image detected from the original image captured through the image data input interface 7, and includes data interpolation processing by the face identification image processing unit 11. It is not the data size of the face image after applying.

  The face identification image processing unit 11 requires the minimum data size described above as the face image data for face identification processing, in this example, a data size of horizontal × vertical = 64 pixels × 64 pixels or more. When the data size of the detected face image is smaller than this, a data interpolation process is performed to try to detect a feature vector as the data size of the face image larger than the minimum data size. In addition, the detected face image is rotated in order to correct the horizontal position of both eyes described above. In this case, pixel data may be interpolated. Even a face image subjected to such an interpolation process may be able to identify a face if a feature vector can be extracted.

  The score Sa described above is information on the degree of reliability of the data size of the face image before the pixel interpolation processing in the face identification image processing unit 11 as described above.

  In this embodiment, when the face image data size Xa is smaller than MIN_SIZE, it is determined that there is almost no reliability of the feature vector for face identification, and Sa = 0. If the face image data size Xa is larger than MAX_SIZE, it is determined that the reliability of the feature vector for face identification is very high, and Sa = 1. When the face image data size Xa is larger than MIN_SIZE and smaller than MAX_SIZE, it is determined that the reliability of the feature vector for face identification corresponds to (proportional to) the data size Xa. The score Sa is a value proportional to the data size Xa.

  In the example of FIG. 38, when the face image data size Xa is larger than MIN_SIZE and smaller than MAX_SIZE, the score Sa as the reliability of the feature vector for face identification is the size of the data size Xa. However, it is of course possible to change the data size Xa in a non-linear manner (for example, a value corresponding to the square of Xa).

[Score Sb: Threshold of deviation (face orientation) of face image from front facing state]
As described above, the face image for face identification is based on the state of facing the front. In general, when a face image is deviated from the front facing state beyond a predetermined allowable range, it is impossible to calculate a feature vector for the face image. Therefore, in this embodiment, the score Sb is calculated based on the deviation of the face image from the front facing state.

  Here, the score Sb represents the degree of reliability based on whether or not the feature vector for the face image can be calculated.

  As shown in FIG. 39, the direction of change assumed for the orientation (angle) of the face is a roll that is a rotation direction about a rotation axis 41 assumed to penetrate from the front direction of the face to the back direction. The direction, the pitch direction that is the rotation direction around the rotation axis 42 orthogonal to the horizontal direction, and the rotation axis 41 about the roll direction, and the rotation direction 41 perpendicular to the rotation direction 41 about the roll direction. The three directions are the yaw direction, which is the rotation direction around the rotation axis 43 orthogonal to the direction.

  Among the three directions, regarding the roll direction, the face identification image processing unit 11 rotates and corrects the image so that the positions of both eyes are aligned in the horizontal direction, and performs face identification processing. The calculation of the score Sb is not considered.

  In this embodiment, the score Sb is calculated considering only the two directions of the pitch direction and the yaw direction. In this embodiment, the score Sb = 0 when the orientation of the face image is deviated from the frontal biological state to the extent that the feature vector for the face image cannot be calculated for either the pitch direction or the yaw direction. For both the pitch direction and the yaw direction, the score Sb = 1 is set when the amount of deviation from the frontal biological state of the face image is within the range in which the feature vector for the face image can be calculated.

In this embodiment, when the angle of the face image orientation deviation from the front facing state in the pitch direction is Apitch and the angle of the face image orientation deviation from the front facing state in the yaw direction is Ayaw. The range within which the feature vector for the face image can be calculated is, for example,
−15 degrees ≦ Apitch ≦ + 15 degrees
−15 degrees ≦ Ayaw ≦ + 15 degrees.

Therefore, in this embodiment,
When −15 degrees ≦ Apitch ≦ + 15 degrees and −15 degrees ≦ Ayaw ≦ + 15 degrees are satisfied, the score Sb = 1. When the angle condition is not satisfied, the score Sb = 0.

[Score Sc: Deviation amount in roll direction]
The score Sc is a degree of reliability of the face image on the face identification according to the roll direction deviation amount which is not considered in the score Sb. Therefore, the detected face image before the rotation correction in the roll direction is performed is supplied to the score calculation unit 14.

  In this embodiment, as shown in FIG. 40, the amount of deviation in the roll direction is the vertical distance between the left and right eyes detected by the part detection process (both eyes in the horizontal direction) in the face image before rotation correction. It is a deviation from the state of being aligned on a straight line, and is calculated as a distance Xc) between the vertical position of the left eye and the vertical position of the right eye.

  That is, when the detected eyes are aligned in a straight line in the horizontal direction as shown in FIG. 40A, the vertical distance Xc between the eyes is Xc = 0. Then, as shown in FIGS. 40B and 40C, when the detected face image is rotated in the roll direction, the larger the amount of rotation, the higher the vertical direction between the detected eyes. The distance Xc becomes larger.

  The score Sc is largest when both eyes are aligned in the horizontal direction, that is, when the distance Xc = 0 between the eyes in the vertical direction is Sc = 1. As the vertical distance Xc between the eyes increases, the value of the score Sc decreases and approaches zero. In this embodiment, as shown in FIG. 41, the score Sc is configured to take a value according to a Gaussian distribution function with a vertical distance Xc between the eyes as a variable.

  FIG. 42 shows a formula for calculating the score Sc as (Formula 6). In (Expression 6) shown in FIG. 42, b is a value that becomes Sc = 1 when the vertical distance Xc between both eyes is Xc = 0, δ is a standard deviation, μ is an average value, These values are obtained by statistical analysis using a large number of actual face image samples.

[Score Sd: Deviation amount in yaw direction within allowable range]
In the above-described calculation of the score Sb, if the amount of deviation in the yaw direction from the front-facing state of the face image is within the allowable range, the score Sb = 1, the degree of reliability is the same.

  This score Sd is a value that represents the degree of reliability in the face identification of the captured face image in more detail even if the amount of deviation in the yaw direction from the front facing state of the face image is within the allowable range. is there.

  In this embodiment, as shown in FIG. 43, the amount of deviation in the yaw direction from the face-to-face state of the face image is the center line 51 of the face image (the contour 50 Only the line parallel to the rotation axis 43 in the yaw direction (see FIG. 39) and passing through the center position in the horizontal direction of the contour 50 of the face image at least at the eye position). , The distance Xd from the midpoint position 52 of the detected distance between the eyes is calculated.

  Note that the midpoint position 52 between the eyes is the same position regardless of whether the face image has been subjected to rotation correction in the roll direction or is a face image that has not been subjected to rotation correction. In this embodiment, the score calculation unit 17 obtains the distance Xd from the face image before the roll direction rotation correction is performed.

  As shown in FIG. 43A, if the midpoint position 52 of the detected distance between the eyes is on the center line 51 of the face image, Xd = 0 and the face image is in front-facing state. Therefore, it is considered that a more accurate feature vector can be extracted, and the score Sd is Sd = 1.

  Then, as shown in FIGS. 43B and 43C, as the distance Xd between the midpoint position 52 of the detected distance between the eyes and the center line 51 of the face image increases, the value of the score Sd becomes It gets smaller and approaches zero. In this embodiment, as shown in FIG. 44, the score Sd follows a Gaussian distribution function with the distance Xd between the detected midpoint position 52 between the eyes and the center line 51 of the face image as a variable. It is configured to take a value.

  FIG. 45 shows a formula for calculating the score Sd as (Formula 7). In (Equation 7) shown in FIG. 45, c is a value at which Sd = 1 when the distance Xd = 0 between the midpoint position 52 of the detected distance between the eyes and the center line 51 of the face image, and δ is Standard deviation, μ is an average value, and these values are obtained by statistical analysis using a large number of actual face image samples.

[Total score]
In the third embodiment, from the four scores Sa, Sb, Sc, and Sd described above, the score calculation unit 17 calculates the total score SO using (Equation 8) shown below. Here, since what is calculated by the score calculation unit 17 is the total score for the newly captured face image data, the total score calculated by the score calculation unit 17 is described as SOnew.

  SOnew = Sa × Sb × Sc × Sd (Equation 8)

(Equation 8) is an example of a method for calculating the total score SOnew, and various other calculation methods can be used. For example,
SOnew = Sa + Sb + Sc + Sd (Formula 9)
As a result, the total score SOnew may be obtained.

Further, for each of the four scores Sa, Sb, Sc, Sd, weight coefficients ka, kb, kc, kd are determined,
SOnew = ka * Sa * kb * Sb * kc * Sc * kd * Sd (Formula 10)
You may ask for the total score SOnew as
SOnew = ka · Sa + kb · Sb + kc · Sc + kd · Sd (Formula 11)
As a result, the total score SOnew may be obtained.

[Weight coefficient W of face image feature vector]
The total score SO may be used as it is for updating the face image identification dictionary data FRDi, n of the period dictionary data TDi, n. In this embodiment, the weight for the feature vector of the face image is calculated from the total score SO. A coefficient W is obtained, and update processing of the face image identification dictionary data FRDi, n using the weight coefficient W is performed.

  Therefore, in this embodiment, the score calculation unit 17 calculates the weight coefficient Wnew for the feature vector of the newly captured face image from the total score SOnew.

In this embodiment, the weighting factor Wnew is generated as being linearly proportional to the total score SOnew. That is, when the proportionality constant is Cs,
Wnew = Cs · SOnew (Equation 12)
The weight coefficient Wnew is calculated as follows. When Cs = 1, the total score SOnew is used as it is as the weighting coefficient Wnew.

Note that the method of calculating the weighting factor Wnew is not limited to the method according to the above (Equation 12), and the weighting factor Wnew corresponds nonlinearly to the value of the total score SOnew. For example, Wnew = Cs · (SOnew) ² · .. (Formula 13)
As an alternative, the weighting coefficient Wnew may be calculated.

  Further, when the total score SOnew is equal to or less than the first threshold value set in advance, the weighting factor Wnew = 0, and when the total score SOnew is greater than the first threshold value, the above (formula 12) and (formula 13) are set. ).

  Furthermore, when the total score SOnew is equal to or lower than the first threshold value set in advance, the weighting factor Wnew = 0 and the preset second threshold value (the second threshold value is larger than the first threshold value). When it is larger, Wnew = 1, and when the total score SOnew is larger than the first threshold value and smaller than the second threshold value, it is calculated by the above (Formula 12) and (Formula 13). Also good.

  As described above, the score calculation unit 17 calculates the total score SOnew for each captured face image, and calculates the weighting coefficient Wnew based on the total score SOnew. Then, the calculated total score SOnew and the weight coefficient Wnew are passed to the dictionary data update processing unit 14.

[Updating face identification dictionary data FRDi]
In the third embodiment, the dictionary data update processing unit 14 determines the period of the personal dictionary data Di corresponding to the personal identification information when the personal identification information is identified based on the identification result in the face identification processing unit 13. The dictionary data generation information DPIi, n, k of the dictionary data TDi, n is updated, and using the updated dictionary data generation information DPIi, n, k, the period dictionary data TDi, n corresponding to the personal identification information The face identification dictionary data FRDi, n is updated.

  That is, the dictionary data generation information DPIi, n, k of the period dictionary data TDi, n already stored in the face identification dictionary database 10 as the personal dictionary data Di of the individual Pi having the newly captured face image. Each of the scores SOi, n, k is compared with the total score SOnew for the newly captured face image, and the dictionary data generation information DPIi, n, k stored in the face identification dictionary database 10 is compared. Whether or not the total score SOnew, n, k is smaller than the total score SOnew for the newly captured face image, that is, the total score SOnew for the newly captured face image is the face identification dictionary. The total score SOi, n, k of the dictionary data generation information DPIi, n, k stored in the database 10 To determine whether or not greater than or Re.

  Then, the dictionary data update processing unit 14 obtains the total score SOi of the dictionary data generation information DPIi, n, k stored in the face identification dictionary database 10 for the total score SOnew for the newly captured face image. When it is determined that it is larger than either n or k, the face identification dictionary data FRDi, n of the period dictionary data TDi, n is updated.

  At the time of updating the face identification dictionary data FRDi, n, the dictionary data update processing unit 14 uses the dictionary data generation information DPIi, n, stored in the period dictionary data TDi, n in the face identification dictionary database 10. Data with the smallest total score SOi, n, k of k is deleted. The dictionary data update processing unit 14 then replaces the deleted dictionary data generation information DPIi, n, k with the feature vector and time data, the weighting coefficient Wnew, the total score SOnew, and the like of the newly captured face image. As shown in FIG.

  Then, the update of the face identification dictionary data FRDi, n is executed by the calculation formula shown in (Formula 4) of FIG. 32 used in the description of the second embodiment. That is, the dictionary data update processing unit 14 includes the feature vectors Vi, n, k included in each of the K pieces of dictionary data generation information DPIi, n, 1 to DPIi, n, K and the corresponding weight coefficients Wi, n. , K is divided by the sum of N weighting coefficients, and updated face identification dictionary data FRDi, n is obtained.

  As described above, this face identification dictionary data FRDi, n (corresponding to the feature vector) has conventionally been K pieces of dictionary data generation information DPIi, n, 1 to DPIi, n, and K, the dictionary data generation information DPIi, n, 1 to DPIi, n, compared to that obtained by a simple average obtained by dividing the sum of feature vectors by the number K. It becomes a value using a weighting coefficient corresponding to the total score of K, and the accuracy of the face identification dictionary data FRDi, n is improved.

  Since K dictionary data generation information DPIi, n, 1 to DPIi, n, K always has a high total score, it remains in the face identification dictionary database 10. The data FRDi, n is updated with higher accuracy.

  When it is determined that the total score SOnew for the newly captured face image is smaller than any of the dictionary data generation information DPIi, n, k stored in the face identification dictionary database 10, the personal dictionary data Di The face identification dictionary data FRDi, n is not updated. This is because if the update using the total score SOnew of the feature vector for the newly captured face image is performed, the accuracy of the face identification dictionary data FRDi deteriorates.

  When the dictionary data generation information DPIi, n, k is less than or equal to the maximum number in the period dictionary data TDi, n in the face identification dictionary database 10, the dictionary data update processing unit 14 The total score SOnew for the face image data captured in step S3 is not compared with the dictionary data generation information DPIi, n, k stored in the face identification dictionary database 10, and the newly captured face image is compared. Are stored in the face identification dictionary database 10 as the latest dictionary data generation information DPIi, n, k, and the face identification dictionary data FRDi. , N generation / update processing is executed. In generating the face identification dictionary data FRDi, n in that case, the dictionary data update processing unit 14 stores the period dictionary data TDi in the face identification dictionary database 10 in place of K in (Expression 4). , N is used to generate / update face identification dictionary data FRDi, n using the number of dictionary data generation information DPIi, n, k stored as part of n.

  When the maximum number K of dictionary data generation information DPIi, n, k is stored in the face identification dictionary database 10 as part of the period dictionary data TDi, n, the dictionary data update processing unit 14 The above-described total score SOnew for the newly captured face image is compared with the total score SOi, n, k of the stored dictionary data generation information DPIi, n, k, and the total score The face identification dictionary data FRDi, n using a higher one is updated.

  In the block configuration shown in FIG. 1 described above, each of the face identification image processing unit 11, the face identification processing unit 12, the display information generation unit 13, the dictionary data update processing unit 14, and the score calculation unit 17 includes hardware components. The CPU 1 may be configured as a software function unit that executes software processing using the RAM 4 as a work area based on a software program stored in the ROM 3.

  In the above configuration, the dictionary data generation information DPIi, n, k includes at least a feature vector, time information, a total score, and a weighting factor. However, the weighting factor is not stored. Later, it may be calculated from the stored total score.

  Further, instead of the feature vector, the captured face image information may be stored, and the feature vector may be extracted again from the stored face image information. The total score and the weighting factor may be recalculated from the stored face image information. In that case, the dictionary data generation information DPIi, n, k may include at least face image information.

[Processing in the personal identification device of the third embodiment]
Next, the score calculation operation for the captured face image feature data and the update processing operation for the face identification dictionary data in the personal identification device of the third embodiment having the above-described configuration will be described.

[Update processing of face identification dictionary database]
The period dictionary data update processing of the face identification dictionary database 10 in the third embodiment will be described. FIG. 46 is a flowchart of an example of an update process of the face identification dictionary data FRDi, n of the period dictionary data TDi, n of the face identification dictionary database 10. In this embodiment, each processing step in FIG. 46 is executed by the score calculation unit 17 and the dictionary data update processing unit 14 according to the control of the CPU 1. The CPU 1 may be configured to execute the process of each unit as a software process by using the RAM 4 as a work area based on a software program stored in the ROM 3, and the above units may be omitted. Is as described above.

  First, the CPU 1 instructs the score calculation unit 17 to start calculation of the total score SOnew. Upon receiving this, the score calculation unit 17 calculates the total score SOnew and the weighting coefficient Wnew as described above, and outputs the calculation result (step S401). A detailed flow of the calculation process of the total score SOnew and the weighting coefficient Wnew will be described in detail later. The CPU 1 transfers the total score SOnew and the weighting coefficient Wnew to the dictionary data update processing unit 14.

  In the case where the score calculation unit 17 is independently provided as hardware as in this embodiment, when the face identification image processing unit 11 successfully generates the feature vector, the detected face image data, An eye detection output or the like may be received by the score calculation unit 17 and the calculation process of the total score SOnew and the weighting coefficient Wnew may be performed in parallel with the face identification process from that point. In this case, at the update timing of the face identification dictionary database 10, the score calculation unit 17 has already calculated the total score SOnew and the weighting coefficient Wnew and stored them in the work area RAM 4. In S401, the CPU 1 reads out the total score SOnew and the weighting coefficient Wnew stored in the work area RAM 4 and transfers them to the dictionary data update processing unit 14.

  The dictionary data update processing unit 14 determines whether or not the value of the total score SOnew for the feature vector of the captured face image calculated (obtained) in step S401 is SOnew = 0 (step S402). If SOnew = 0, this is notified to the CPU 1. Upon receiving this notification, the CPU 1 determines that the feature vector of the captured face image is an unreliable feature vector that cannot be used for updating the dictionary data, and ends this processing routine as it is.

  If it is determined in step S402 that SOnew = 0 is not satisfied, the dictionary data update processing unit 14 determines the name of the person to be collated as a collation result or the user on the screen of FIG. 4 or FIG. As the input dictionary data generation information DPIi, n, k of the person to be collated, it is determined whether or not K, which is the maximum storable number, has already been accumulated (step S403).

  When it is determined that the K pieces of dictionary data generation information DPIi, n, k have not been accumulated, the dictionary data update processing unit 14 determines the feature vector, time information, total score SOnew, and weight coefficient Wnew for the captured face image. And the like, including new dictionary data generation information DPIi, n, k for a collation target person (including both a collation target person who is correct as a collation result and a collation target person whose name is entered by the user; the same applies hereinafter). Is added to the face identification dictionary database 10 (step S404).

  Then, the dictionary data update processing unit 14 generates and updates the face identification dictionary data FRDi, n of the person to be collated according to (Equation 4) described above (step S408). At this time, the number of dictionary data generation information DPIi, n, k is K or less, so in (Equation 4), the dictionary data generation information DPIi, n, k is stored instead of K. A number is used to perform the operation.

  When it is determined in step S203 that the maximum number K of dictionary data generation information DPIi, n, k has already been accumulated, the dictionary data update processing unit 14 determines the K dictionary data of the person to be verified. The total score SOi, n, k of the generation information DPIi, n, k is sequentially read and compared with the total score SOnew calculated for the newly captured face image (step S405).

  Then, the dictionary data update processing unit 14 has a value smaller than the total score SOnew for the captured face image among the total scores SOi, n, k of the N pieces of dictionary data generation information DPIi, n, k. It is determined whether or not there is any data (step S406). If there is no such data, it is determined that there is no need to update the dictionary data, and this processing routine is terminated.

  In step S406, the total score SOi, n, k of each of the K dictionary data generation information DPIi, n, k has a value smaller than the total score SOnew for the captured face image. Is determined, the dictionary data update processing unit 14 detects the one having the minimum total score SOi, n, k among the K pieces of dictionary data generation information DPIi, n, k, and the detected total Dictionary data consisting of the accumulated contents of dictionary data generation information DPIi, n, k having the smallest score SOi, n, k, and feature vectors, time information, total score, weighting coefficient, etc. for the face image that is the total score SOnew The information for generation is replaced with the same content (step S407).

  Since the contents of the K dictionary data generation information DPIi, n, k are changed by the replacement in step S407, the weight coefficients of the K dictionary data generation information DPIi, n, k, Using the feature vector, the calculation based on the above-described (Expression 4) is performed, and the update processing of the face identification dictionary data FRDi, n for the person to be collated is executed (step S408).

  This is the end of the dictionary database update processing routine.

  In addition, when newly registering the period dictionary data TDi, n, a total score is calculated, a weighting coefficient is calculated, and these are stored as a part of the dictionary data generation information DPIi, n, k. Since only the differences are the same as in the case of the first embodiment, a detailed description thereof will be omitted.

[Score calculation processing]
Next, the calculation process of the total score and the weighting coefficient in the score calculation unit 17 will be described. FIG. 47 and FIG. 48, which is a continuation thereof, are flowcharts illustrating an example of the flow of the total score and weight coefficient calculation processing in the score calculation unit 17.

  That is, the score calculation unit 17 detects the face detection data size Xa received from the face identification image processing unit 11 (step S501), and the detected face image data size Xa is the minimum of the preset face image. It is determined whether or not the data size is smaller than MIN_SIZE (step S502).

  When the score calculation unit 17 determines that the data size Xa of the face image is smaller than the minimum data size MIN_SIZE, the score Sa value based on the data size Xa of the face image is Sa = It is set to 0 (step S503).

  When it is determined that the face image data size Xa is not smaller than the minimum data size MIN_SIZE, the score calculation unit 17 determines that the face image data size Xa is a predetermined enough face image data size MAX_SIZE. It is determined whether or not the value is greater than (step S504).

  When the score calculation unit 17 determines in step S504 that the face image data size Xa is larger than the predetermined sufficient data size MAX_SIZE, the value of the score Sa is as shown in FIG. Sa = 1 is set (step S505).

  When it is determined in step S504 that the face image data size Xa is smaller than the predetermined sufficient data size MAX_SIZE, the score calculation unit 17 sets the score Sa as shown in FIG. A value corresponding to the data size Xa of the detected face image is set (step S506).

Next, the score calculation unit 17 detects the face direction (face angle) of the captured face image (step S507). Then, the score calculation unit 17 detects the angle Ayaw of the face image orientation deviation from the front facing state in the yaw direction and the angle Apitch of the face image orientation deviation from the front facing state in the pitch direction (step) S507), whether the angles Ayaw and Apitch are both within the allowable range described above, that is,
−15 degrees ≦ Ayaw ≦ + 15 degrees and −15 degrees ≦ Apitch ≦ + 15 degrees
It is determined whether or not (step S508).

  Then, when the score calculation unit 17 determines in step S508 that both the angle Ayaw and the angle Apitch are within the allowable range, the score calculation unit 17 determines the value of the score Sb regarding whether the face angle is within the allowable range. , Sb = 1 (step S509).

  If the score calculation unit 17 determines in step S508 that both or one of the angle Ayaw and the angle Apitch is not within the allowable range, the score Sb is set to Sb = 0 (step S510). ).

  Then, after step S509 or step S510, the process proceeds to step S511 in FIG. 48, and the score calculation unit 17 deviates from the horizontal of the left and right eyes detected by the part detection process, as described with reference to FIG. That is, the vertical distance Xc between the left and right eyes described above is detected.

  Then, the score calculation unit 17 determines whether or not the detected distance Xc is Xc = 0 and both eyes are aligned horizontally (step S512), and when determining that the eyes are aligned horizontally, the score Sc The value is Sc = 1 (step S513). When it is determined in step S512 that the lines are not horizontally arranged, the score calculation unit 17 calculates the score Sc according to the Gaussian distribution function described above (Expression 6) (see FIG. 42). The value of the score Sc is calculated (step S514).

  Then, after step S513 or step S514, the process proceeds to step S515, and the score calculation unit 17 determines that the center line 51 of the face image and the midpoint position 52 between the eyes as described with reference to FIG. The distance Xd is detected (step S515).

  Then, the score calculation unit 17 determines whether or not the detected distance Xd is Xd = 0 and the face image is in the front facing state (step S516), and the face image is in the front facing state. When the determination is made, the value of the score Sd is set to Sd = 1 (step S517). If it is determined in step S516 that the face image is not in the front-facing state, the score calculation unit 14 is a calculation formula for the score Sd according to the Gaussian distribution function described above (Formula 7) (see FIG. 45). Based on the above, the value of the score Sd is calculated (step S518).

  Then, after step S513 or step S514, the process proceeds to step S515, and the score calculation unit 17 calculates the total score SOnew using the calculated scores Sa, Sb, Sc, Sd as described above. The weight coefficient Wnew is calculated from the calculated total score SOnew (step S519). Above, the score calculation process in the score calculation part 17 is complete | finished.

  In the above description, after personal identification is performed, the face identification dictionary data is updated using the new captured face image following the personal identification. Image data or feature vectors, time information, an overall score, and weighting factors may be temporarily stored, and the face identification dictionary data may be updated at a time after the personal identification time.

  Moreover, you may make it memorize | store either an integrated score and a weighting coefficient instead of memorize | storing both in a temporary memory part or a personal dictionary data memory | storage part. In the case where the weighting factor is stored, the determination as to whether or not to update the dictionary data may be made using the weighting factor instead of the total score.

[Effect of the third embodiment]
As described above, in the above-described embodiment, each of the maximum K pieces of dictionary data generation information DPIi, n, k in the period dictionary data TDi, n of each individual Pi has a feature vector of a face image. Since the total score SOi, n, k representing the reliability of the face image is stored, the total score SOnew representing the reliability of the feature vector of the newly captured face image and the stored N faces By comparing the total score SOi, n, k (k = 1 to K) of the feature vector of the image, the feature vector of the newly captured face image is effective for updating the face identification dictionary data. It can be determined whether or not.

  As a result of comparison, the total score SOnew for the feature vector of the newly captured face image is obtained from any one of the K dictionary data generation information DPIi, n, k in the period dictionary data TDi, n of the individual Pi. Is also deleted by deleting the one with the lowest total score SOi, n, k, and instead storing data having the same content as the dictionary data generation information, such as a new face image feature vector. Only data with a high total score value and high reliability can be stored as K dictionary data generation information for the period dictionary data TDi, n of each individual Pi.

  The face identification dictionary data FRDi, n is also updated by updating the face identification dictionary data FRDi, n using only the K dictionary data generation information DPIi, n, k with high reliability. Therefore, the dictionary data can be always highly accurate.

  Further, in the above embodiment, in the update calculation of the face identification dictionary data FRDi, n, instead of taking a simple average of the respective feature vectors of the k pieces of dictionary data generation information DPIi, n, k, Since the sum of the product vectors multiplied by the weighting factor based on the corresponding total score is divided by the sum of the weighting factors, each feature vector is stored in the face identification dictionary data FRDi according to its reliability. , N contributes to the generation of dictionary data with higher accuracy.

[Modification of Third Embodiment]
The third embodiment described above is a case where the present invention is applied to a face identification processing device. Needless to say, the present invention is not limited to face identification.

  For example, in the case of personal identification processing, when identifying an individual using a person's voice or the like as an identification element, one or more based on the deviation from one or more reference conditions when each of the identification elements is captured. Calculate the score. If there are a plurality of calculated scores, the total score is calculated using the plurality of scores.

  And like the above-mentioned embodiment, based on a score or a total score, it is possible to select a plurality of data to be stored as dictionary data generation information or to update identification dictionary data. it can.

  In addition, when capturing a plurality of photographic images in a personal computer and saving them as an album, for example, when storing the captured photographic images in a memory, the captured image is, for example, an image showing a human face. A reference condition such as an image having a lot of high frequency components (having a clear outline) is defined, a score or an overall score is calculated from a deviation from the reference condition, and the score or the overall score is converted into a photographic image It can also be applied to the case of storing in correspondence with the.

  In this way, it is possible to preferentially display images close to the reference condition, such as arranging photographic images in descending order of score or overall score.

It is a block diagram which shows the hardware structural example of 1st Embodiment of the personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure used in order to explain an example of face identification processing in an embodiment of a personal identification device by this invention. It is a figure for demonstrating the example of the storage content of the dictionary database for face identification in 1st Embodiment of this invention. It is a figure for demonstrating the example of the storage content of the dictionary database for face identification in 1st Embodiment of this invention. It is a figure for demonstrating the example of a format of the storage content of the dictionary database for face identification in 1st Embodiment of this invention. It is a figure for demonstrating the example of a format of the storage content of the dictionary database for face identification in 1st Embodiment of this invention. It is a figure for demonstrating the period which the period dictionary data in embodiment of this invention respond | correspond. It is a figure for demonstrating the period which the period dictionary data in embodiment of this invention respond | correspond. It is a figure which shows the flowchart for demonstrating the example of the dictionary period parameter setting in embodiment of the personal identification device by this invention. It is a figure for demonstrating the other example of the format of the storage content of the dictionary database for face identification in embodiment of this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the face identification process in embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the face identification process in embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the face identification process in embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the search process of the period dictionary data in embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the search process of the period dictionary data in embodiment of the personal identification device by this invention. It is a figure which shows the flowchart for demonstrating the flow of the new registration process of the period dictionary data in embodiment of the personal identification device by this invention. It is a figure for demonstrating the flow for the update process of the dictionary data for face identification in embodiment of the personal identification device by this invention. It is a block diagram which shows the hardware structural example of 2nd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the example of a format of the storage content of the dictionary database for face identification in 2nd Embodiment of this invention. It is a figure for demonstrating the example of a format of the storage content of the dictionary database for face identification in 2nd Embodiment of this invention. It is a figure for demonstrating selection setting of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the 1st example of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the 2nd example of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure used for description of the 2nd example of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the 3rd example of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the 4th example of the weighting coefficient calculation method in 2nd Embodiment of the personal identification device by this invention. It is a figure used in order to demonstrate an example of the calculation method of the dictionary data for face identification in 2nd Embodiment of the personal identification device by this invention. It is a figure which shows the flowchart for demonstrating the flow of the update process of the dictionary data for face identification in 2nd Embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the weighting coefficient calculation process in 2nd Embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the weighting coefficient calculation process in embodiment of the personal identification device by this invention. It is a block diagram which shows the hardware structural example of 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating the example of a format of the storage content of the dictionary database for face identification in 3rd Embodiment of this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure for demonstrating one of the scores calculated about the face image in 3rd Embodiment of the personal identification device by this invention. It is a figure which shows the flowchart for demonstrating the flow of the update process of the dictionary data for face identification in 3rd Embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the score calculation process in 3rd Embodiment of the personal identification device by this invention. It is a figure which shows a part of flowchart for demonstrating the flow of the score calculation process in 3rd Embodiment of the personal identification device by this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Face identification dictionary database, 11 ... Face identification image processing part, 12 ... Face identification processing part, 14 ... Dictionary data update processing part, 15 ... Weight coefficient calculation part, 16 ... Dictionary data generation / update part, 17 ... Score calculator

Claims (10)

As dictionary data for identifying an individual, a plurality of period dictionary data generated from feature data extracted from an identification element for identifying the individual in each of a plurality of different periods,
The personal identification device characterized in that identification of the individual in each of the different periods is performed using the period dictionary data of a corresponding period. The personal identification device according to claim 1,
The length of the period to which each of the plurality of period dictionary data corresponds is the same. The personal identification device according to claim 1,
The length of the period to which each of the plurality of period dictionary data corresponds is variable. The personal identification device according to claim 1,
The period of the period dictionary data corresponding to two adjacent periods among the periods of the plurality of period dictionary data includes a period partially overlapping each other. The personal identification device according to claim 1,
The length of the period to which each of the plurality of period dictionary data corresponds is variable according to the age of the individual in the period. The personal identification device according to claim 4,
When the time when the identification element is captured from the individual is within the overlapping period of the period dictionary data corresponding to the two adjacent periods, personal identification is performed using one of the period dictionary data. A personal identification device characterized by performing. The personal identification device according to claim 6,
One of the period dictionary data is dictionary data having a larger number of identification elements captured in a corresponding period. The personal identification device according to claim 1,
The period dictionary data is updated each time the new identification element of the individual in a corresponding period is acquired,
A personal identification device, wherein the period dictionary data is updated by weighting the feature data extracted from the identification element with a weighting coefficient according to the time of capture. The personal identification device according to claim 1,
The period dictionary data is updated each time the new identification element of the individual in a corresponding period is acquired,
The feature data extracted from the identification element is weighted based on reliability information determined according to a deviation from a predetermined reference condition for the acquired identification element, and A personal identification device characterized by updating period dictionary data. A plurality of period dictionary data generated from feature data extracted from identification elements for identifying the individual in each of a plurality of different periods as dictionary data for identifying an individual,
The personal identification method characterized in that identification of the individual in each of the different periods is performed using the period dictionary data of a corresponding period.

Images