JPH05143651A - Face picture retrieving device - Google Patents

Abstract

(57) [Abstract] [Purpose] Neural network information processing mechanism measures the similarity between the input face image and the teacher's face image, and makes the phone a personal specification, entrance control, suspect check at bank, answering machine T
It is applied to the ID check of V intercoms. [Structure] A camera or an image input device for inputting a face image, a device for converting into a digital image, a mechanism for trimming this digital image, a function for searching by a neural network, and a function for embedding a face image in the neural network. And a mechanism for constantly sending the image from the camera to the neural network. [Effect] It is possible to perform an image search in a short time, and the nonlinear similarity of the search result is output to an analog digital value, which enables quantitative evaluation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for retrieving a face image using a neural network information processing mechanism, which is applied to ID check, safety monitoring, optical desk image filing system and the like.

[0002]

2. Description of the Related Art Conventionally, in facilities and devices used by a large number of people who are public or unspecified, a person who has a magnetic card, an IC card, or the like can carry an identification check, an ID check, etc. It was done in card format.

FIG. 4 is an example of registration of a destination by a conventional telephone. In this example, the registered person can enumerate the contents of the button 21 by displaying the telephone number or name of the person on the screen 20 and call the intended person. There is. Shown here is a telephone that each individual has at hand. However, when the user moves on the premises, if the user wants to set his / her own telephone at the place where the user moves, normally, at the place where he / she goes, set the telephone or TV phone to his own specifications. Therefore, the magnetic or IC card ID card 2
I replaced 2 and inserted it into the phone and set it.

FIG. 7A shows an example of a conventional time card machine for managing working hours. Conventionally, a card 22 such as a magnetic card or an IC card owned by each person is inserted into a card machine 28 to perform mechanical or electrical marking.

FIG. 8 (a) shows an example of a conventional ID check when a specific person enters or leaves the confidential data management room. Each person has a magnetic or IC card ID card 22,
This was inserted into the inspection machine 29, and an electrical check was performed to make a determination and permission to enter the room was given.

FIG. 9 (a) shows an example of an intercom installed in a gate fence 30, which is conventionally a remote TV monitoring device 2 for crime prevention.
It was only 1. Further, in the conventional surveillance work in a financial institution such as a bank, in general, video output from surveillance cameras arranged in various places such as a security room is intensively displayed on a monitor screen, and , Is constantly looking at fraud and its potential risks.

On the other hand, to identify a face image, the face itself is
Since it is a three-dimensional element including the head, it is very difficult to identify it. For this reason, the conventional face image retrieval device is a corner "analysis of face image by parallel top-down processing method" The Institute of Electronics, Information and Communication Engineers Computer Vision 61-3, Nagashima "Study on automatic generation method of three-dimensional model of face" Electronic information and communication Society OS 90-31, IE 90-46, Numazaki “Person identification from face image based on model” IEICE IE 87-136 results in processing of three-dimensional data.

FIG. 11 (a) shows a conventional method for identifying a person from a face image based on a three-dimensional model, in which a three-dimensional basic model of the head is created in advance as numerical data and placed, and the input face image is input. On the other hand, the model is deformed so as to match, and further the direction is deformed in the frontal direction of the face. Further, the deformed model and the already registered two-dimensional image are compared, Identify by the matching result.

FIG. 11 (b) is a diagram of facial feature points at the stage of studying automatic generation of a three-dimensional model of a face for performing conventional live-action conference communication. 3D model data was obtained by laser scanning of plaster, and the input image of the side face was curved, and the contour of the front face was deformed so as to match each of the derived characteristic points with the characteristic points of the model. Using the three-dimensional data obtained in this way, it is transferred to the virtual conference space.

FIG. 11 (c) shows an example of a face image analysis method by the conventional parallel predicate top-down processing. The contour line extracted by the image signal processing is applied to the input face image by the predicate logical means. Then, the knowledge rule base created in advance is searched by the production type execution and the pattern matching is performed. This processing is performed in parallel processing and parallel execution format for each face knowledge database registered in advance. It is completely useless for each execution in which no goal is obtained, but this method does not require complicated control of algorithm switching. Therefore, a fast search is achieved.

[0011]

As described above, the conventional telephones and various ID checking devices use the IC card or the like in order to have their own setting specifications. However, it is necessary to carry an IC card with you every time you move, and there is a risk of loss. If you lose it, the reissuing process cannot be performed immediately, and the setting specifications cannot be reproduced. There were various restrictions such as possible. These dangers are not only in the telephones but also in the entrance ID cards and the like, and there are various problems.

Further, since the face image search process is a three-dimensional data process, there is a problem that it is complicated and difficult to identify. The present invention uses a neural network type information processing function to search for a face image, and an ID check function based on matching of face images enables entry control, personalization of a telephone, suspect check, and visitor identification. The purpose is to do.

[0013]

A face image retrieval apparatus according to the present invention is a mechanism for converting an image input by a camera and a contact type image input device into a digital image, a mechanism for extracting a face image portion, and a mechanism for extracting the face image portion. A mechanism for inputting a face image into the input layer of the neural network mechanism, means for embedding a teacher face image signal for face determination in the neural network, and a neural network information processing mechanism for processing by the error backpropagation system algorithm It is a thing.

A face image retrieval apparatus according to another invention is
2. The retrieval apparatus according to claim 1, wherein the neural network mechanism inputting means for the extracted face image has a coordinate axis transformation device for axially transforming the digital image of the image, and the axially transformed digital image is directly compressed and subjected to conversion processing. And a compression / conversion processing means for inputting to the neural network.

[0015]

The neural network mechanism input means of the image retrieval apparatus according to the present invention registers and embeds a face image signal taken in from a camera or an image scanner into a neural network. Further, the BP type neural network mechanism uses the information comparison function to utilize the immediacy and non-linearity of the neural network processing, and slightly deforms the input signal by the non-linear similarity measure output as an analog quantitative digital value. Even if it includes, the identification operation of the party becomes possible.

In another aspect of the invention, the coordinate axis conversion means of the image retrieval apparatus performs the axis conversion of the image signal of the image digital image with the coordinate axes of the entire enclosing angle, and the compression and conversion processing means converts the information of the axially converted digital image. It is directly compressed and converted and input to the neural network mechanism.

[0017]

【Example】

Example 1 and FIGS. 1 to 3 show examples of the present invention. FIG. 1 shows the basic concept of the system used in the present invention, and is a conceptual diagram of the processing form of a neural network. In FIG. 1, 1 corresponds to a neuron functional element of the input layer, which also has a data storage function, to which a digital value corresponding to the analog amount of the input image is input. The signal from the 1 input layer is sent to the input side synapse transmission system 4 including a variable weighting coefficient called a synapse. The input-side synapse transmission system 4 is connected from each input-layer neuron element 1 to the intermediate-layer neuron element 2 in a brute force manner, and the intermediate-layer neuron element 2 is further connected to the next intermediate layer. It is connected to the neuron element or the final output layer neuron element 3. This intermediate layer neuron 2 has no limitation on the number of layers,
It is appropriately determined depending on the required input category and the required teacher element.

In the entire transmission systems 4 and 5, signals are transmitted from the input layer neuron element 1 to the output layer neuron element 3 in full multiplex parallel fashion. Then, the final result appears as a result after the embedding. On the other hand, this signal transmission system has the model teacher signal 1 in the education mode.
The error between 8 and the output value from the input category 14 is indirectly reflected, and feedback 15 is performed to change the weighting coefficient.
Therefore, the whole system goes asymptotically to an arbitrary stable state, and is held in the final stable state.

FIG. 2A is a conceptual diagram for the image processing of the present invention. Reference numeral 6 formally represents the input image signal system as transparency. The signal that has passed through the transparency 6 is processed by the neural network processing system 9.

FIG. 2B shows the above relationship as a side view, in which coherent illumination light 8 is applied from the front of the transparency 6 and the image signal as it is is applied to the neural network 9.

Similarly, FIG. 2C is a side view. The arrangement is similar to the previous one, but the difference is that
A coordinate axis conversion function device shown by 10, which is
For convenience, it is represented by a convex lens of the optical system. The arrangement is such that the coherent light 8 and the transparency 6
The image signal input transmitted through is incident on the second scattering type transparency 7 at the focal position of the lens function 10.
The output through the transparency 7 is a signal obtained by Fourier-transforming the first image signal with the coordinate axes of the entire envelopment angle, which is the input layer neuron element 1
Entered in. In such a conversion process, since the target image element in the input image has incompatibility related to its position,
This is performed to prevent this, but this should not be limited to the Fourier transform, and other axis transform methods may be used.

FIG. 3 is a block diagram of functional blocks when constructing an actual system based on the above basic principle.
11 is a TV camera, 12 is an image reader device, and 13 is A for converting an image analog signal into a digital image.
A D / D converter, and 14 is an image clipping device that extracts only the face portion from the image data. The face image data extracted by the image clipping device 14 is processed by the neural network 9. The signal terminal 18 is for a teacher signal, and when the operation mode switching SW 19 is connected to the education mode side, an error between the teacher signal 18 and the neural network output result is transmitted in the reverse direction of the image cropping device 14. State feedback is performed by the operation of changing the weighting factor of 4. 17 is a monitoring error signal for checking the degree of achievement of learning, 15 is an error back propagation part,
Reference numeral 16 is an output terminal of the result in the search mode.

Six application examples will be shown below. Application example 1 and FIG. 5 (a) are application examples of the present invention that can be set without using an IC card. First, the destination registered in advance obtains face images as much as possible. By presenting this on the screen, it is possible to respond visually. This makes T more intimate
V telephone can be realized.

FIG. 5 (b) is a diagram showing a process of setting an image from the TV camera of the present invention in a telephone set for one's own use. The attached TV camera 21 puts a face image of the person 25 to be set, the face image is converted into a digital image, and the image 23 is sent to the exchange 22. On the exchange 22 side, a search is made for a similar image to the image sent from the telephone, and the candidate 24
To the telephone side. If there is only one candidate, the series of related face image data corresponding to the candidate is continuously sent back without being returned. It is set by this, and becomes a setting for oneself. With this, bring your IC card as before,
You can freely set it for yourself on the go without inserting it.

Application example 2, FIG. 6 is an example of application to a monitoring device in a financial institution such as a bank. First, the camera 21
Then, when the image of the window of the bank is displayed on the monitor CRT 26, it is monitored in the central monitoring room. This video signal is branched and taken out, and is A / D converted by the A / D converter 13 to be a digital image. From this digital image, a face image portion required by the image clipping device 14 is digitally extracted. The extracted face image portion is input to the neural network information processing mechanism 9. 27 outputs the output
It is displayed by an LED, and in actuality, a warning may be displayed on the CRT. Reference numerals 11 and 12 denote face image input devices for embedding the suspect's face in the neural network. By comparing the image data in the image clipping device 14 and the image data in the neural network information processing mechanism 9, if there is a similar face, a warning is issued. In this example,
It has the function of acting as a proxy for part of the monitoring work of conventional financial institutions such as banks.

First, facial photographs of a plurality of suspects such as bank robbers are input from the input device 11 or 12, and the neural network information processing mechanism 9 learns and embeds them. afterwards,
Return to check mode and introduce it to the monitoring work. If left in this state, when a suspect suddenly visits a bank, he will immediately issue an alarm. This would allow the guards to immediately put themselves in a vigilant position and, if the suspect had the sign, could immediately arrest them. Whether or not this early warning and monitoring is possible is very important, and it is possible to prevent a large number of crimes before they occur or without causing serious damage.

Application example 3, FIG. 9A is an application example in a general household. At the gate 30, a gate fence intercom with a TV camera 21 is installed. Here, this TV
The feature is that the intercom with a camera has an answering machine function. By having this feature, even
Even if a person visits while he / she is away, he / she can respond to the person. For example, even if an ordinary liquor store picks up an order, if you know it, by sending a pre-recorded message, your daily chores will be smooth and you will be able to respond to strangers even if you do not know them. It becomes possible to have a function such as recording an answering machine's message like an answering machine.

FIG. 9B shows the camera 21 and the monitor 26.
Is a component of the TV intercom, and a visitor identification is performed by attaching a digital image forming device of the image clipping device 14, the neural network information processing mechanism 9, a neural network type, and a face image search function element to the component. .. According to this, a recording device that can freely control the recording / reproducing location by the micro CPU is installed, and the recording / reproducing can be handled individually for each visitor. Incidentally, 31 is a tape recorder controllable by a CPU, and 32 is a speaker of a gate fence TV intercom.

Application example 4, FIG. 7 (b) shows a camera 21.
And an example of a time recorder 28 for managing working hours using a neural network. The face image displayed on the camera 21 is confirmed by a neural network and stamped.

Application example 5, FIG. 8 (b) is an application to entrance control using the present invention, in which only the ID check function is performed by using the camera 21 and the neural network.
It gives permission to enter the room. Also in this case, it is not necessary to carry around an ID card or the like.

Application example 6, FIG. 10 is an example of application in an optical disk image filing system. For example, in a company, personal information of an employee is stored together with a face image, and the personnel information of the employee is processed such as reissuing an ID card, creating a new member, and searching. More specifically, the CRT 26 is a display for displaying a face image and various information, and causes the optical disc drive 34 to record and reproduce the face image via the camera 11 and the image reader 12. Reference numeral 33 is its control unit. Reference numeral 35 is a LAN line connection. Then, these image data can be accessed through a LAN line, and even if an employee goes to a place related to the company on a business trip or the like, the main image can be obtained from the remote place through the company computer network. The system is accessed, and by this, the remote TV-phone type computer terminal 3
From 6, it is possible to identify an individual without using an ordinary ID card, and a data contact class is set according to the individual, and it is possible to prevent leakage of confidential data along with access records. .. A second application example of this system is an example in which a suspect is used to identify a suspect in police or the like. By inputting a face image using a scene photograph of a certain crime or a video image for surveillance, it is possible to immediately extract a suspicious suspect candidate with high similarity.

[0032]

As described above, according to the present invention, the facial image can be determined in a short time by the neural network information processing mechanism, and the presented image is the same as the image used in the education mode. Even if they are not exactly the same, the similarity comparison is performed non-linearly, and the analog quantitative digital value is reported. In addition, since the above-described nonlinear similarity comparison is performed, there is flexibility that it is possible to make a determination even for an image in which a part of the face is masked.

Further, in another invention, the image digital image is axially converted, and the axially converted digital image is directly compressed and converted to be inputted to the neural network. Even if the target image element has incompatibility related to its position, the identification operation of the parties becomes possible.

[0034]

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of a neural network.

FIG. 2A is a three-dimensional representation of an input form of an image for image processing into a neural network. (B) is an input form diagram in which pixels of an image are directly input. (C) is an input form diagram in which the coordinate axes of the image input are converted.

FIG. 3 is a functional block diagram for searching a face image using a neural network.

FIG. 4 is a diagram of a conventional method of setting a function of a conventional telephone set to a personalized specification.

FIG. 5A is a TV by face image control using the present invention.
Diagram of how to make your own phone. FIG. 6B is a diagram of a method for personalizing a TV phone by controlling a face image using the present invention.

FIG. 6 is a functional circuit block diagram when applied to crime prevention monitoring of a bank.

FIG. 7A is a perspective view of a conventional time recorder for work management. (B) is a perspective view of a time recorder of the present invention for work management.

FIG. 8A is a perspective view of a conventional entry / exit management device for a confidential data room. (B) is a perspective view of the access control apparatus of this invention to a confidential data room.

FIG. 9A is a perspective view applied to a home gate intercom. (B) is an internal block diagram of an example applied to a home entrance intercom

FIG. 10 is a block diagram of an employee ID card creation and personnel data processing system using an optical disk image filing system.

FIG. 11A is a diagram showing a conventional method for identifying a person from a face image based on a model. (B) is a figure which shows the feature point part of the face by the conventional automatic generation method of the three-dimensional model of the face. (C)
Is a diagram showing a conventional facial image analysis method by parallel top-down processing.

[Explanation of symbols]

1 Neuron element of input layer of neural network 2 Neuron element of intermediate layer of neural network 3 Neuron element of output layer of neural network 4 Synapse transmission system of input side of neural network 5 Synapse transmission system of output side of neural network 6 Direct input image Transparency 7 Transparency of axis conversion image input 8 Coherent illumination light 9 Three-dimensional display of neural network 10 Convex lens or Fourier transformer 11 TV camera 12 Image reader device 13 A / D converter of video signal 14 Image clipping device 15 Error back propagation part 16 Result sending part 17 Error output for learning degree evaluation 18 Teacher signal input 19 Neural net processing result 20 Image display of TV phone 21 Camera part attached to TV phone 22 IC card for TV phone 2 Image sent from the camera attached to the TV phone for personalization of the switch 24 Image of settings sent from the switch according to the intended specifications 25 New user 26 Monitor TVCRT group for crime prevention monitoring 27 Neural Alarm output from the net 28 Time recorder for work management 29 ID check machine to the secret room 30 Home entrance 31 Recording / playback tape recorder 32 controllable by CPU 32 Entrance TV intercom speaker 33 Optical disk image filing system controller 34 Optical disk Image filing drive 35 Computer LAN line 36 TV telephone type computer terminal at remote location

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Internal reference number FI technical display location H04L 9/00 9/10 9/12

Claims (2)

[Claims] 1. A means for converting an image input by a camera or a contact image input device into a digital image, a means for extracting a face image portion, and a means for inputting the extracted face image to an input layer of a neural network mechanism. And a neural network information processing means for performing processing by an error back-propagation type algorithm, and a face image retrieval apparatus. 2. A neural network mechanism inputting means for the extracted face image, a coordinate axis converting device for axially converting the digital image of the image, and the digitally converted image is compressed and converted as it is and input to the neural network. 2. The face image search device according to claim 1, further comprising compression and conversion processing means.