JP2006011591A - Individual authentication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an individual authentication system allowing further improvement of a security level, and difficult to break through security until a living person oneself directly performs operation. <P>SOLUTION: Four kinds of contact type biological characteristic information detection parts 342, 343, a face photographing camera 341, a bone conduction sound detection part 340 and an air conduction sound detection part 304 are set as a mother group of an authenticating characteristic information acquisition part, and at least two kinds are selected from them and are provided in a cellphone 1. Acquisition of authenticating characteristic information by at least two designated authenticating characteristic information acquisition parts is simultaneously executed in a telephone use grasp holding state. When the acquisition of the authenticating characteristic information is not simultaneously performed, acceptance authentication (e.g. authentication that one is a regular user) of an authentication processing target person is not performed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a personal authentication system using a mobile phone.

JP 2000-259828 A Japanese Patent Laid-Open No. 2004-80080

  As a personal authentication method, a so-called speaker recognition technique using personality information included in a voice wave of a person to be authenticated is widely used. For example, recently, as disclosed in Patent Documents 1 and 2, various personal authentication methods including speaker recognition have been proposed in order to increase the security level of mobile phones. In recent years, the proliferation of mobile phones has been increasing rapidly, and competition for new model development has intensified, so the model replacement cycle has also been shortened. Since mobile phones serve as a storage medium for personal data such as a telephone directory and mail address list, a problem has been pointed out that waste telephones with data remaining are sold as junk and cause personal information leakage. In addition, mobile phones equipped with information communication terminal functions such as the Internet connection are becoming standardized, and are widely used for payment for information provision, payments for shopping, mobile banking, etc. Since diversion to lock operation terminals such as buildings and automobiles is also considered, a higher security level is required. Patent Documents 1 and 2 disclose a technique for raising a security level by combining not only voice authentication but also an authentication method using another means such as face image matching or fingerprint matching.

  In recent years, as security systems become more sophisticated, criminal techniques that illegally break them have become more sophisticated or bold. For example, as in Patent Document 1 and Patent Document 2, when a method that combines authentication using images such as fingerprints and faces and authentication using voice is adopted, it seems that it is very difficult to break through security. However, if the following method is adopted, it is not impossible to pass through all the security steps spread over a plurality of stages. In other words, the presence or absence of a legitimate user using a photo or video for the face, a recording tape for the voice, a photo-engraved stamp for the fingerprint, or an arm or finger cut from the person to be authenticated. Are virtually reproduced, and acceptance authentication is sequentially obtained (hereinafter, such fraud is referred to as “substitute false authentication”). This method can break through security even if the living person is not on the spot, and does not necessarily require a risky method such as kidnapping and abduction. In addition, even if a violent crime involving kidnapping is involved, once the information necessary for authentication is obtained from the person, it is sufficient to use a copy or an acquired product (finger etc.), so it is used There is a possibility that the trap will not work even if the person is killed for the purpose of sealing.

  It is an object of the present invention to provide a personal authentication system that can further improve the security level and is difficult to break through unless a living person directly operates.

Means and actions / effects for solving the problems

The present invention relates to a personal authentication system for authenticating a person to be authenticated using a mobile phone, and in order to solve the above problems,
An authentication feature information acquisition unit provided in the mobile phone,
Provided at the position where the hand of the person subject to authentication processing touches when the person subject to authentication processing holds the mobile phone in the same state as when the telephone function other than the authentication process is used. A contact-type biometric feature information detecting unit for detecting biometric feature information of the hand,
A camera for photographing a face provided at a position where the face of the person subject to authentication processing can be photographed when the person subject to authentication processing holds the mobile phone in a state of grasping the use of the telephone;
A bone conduction sound detection unit for detecting the voice information of the person to be authenticated by bone conduction sound;
An air conduction sound detection unit for detecting voice information of the person to be authenticated by air conduction sound;
For authentication including two or more selected from the group consisting of (including two or more types of contact-type biometric feature information detection units when two or more types of contact-type biometric feature information detection units are provided) A feature information acquisition unit;
Authentication feature information acquisition control means provided in a mobile phone, wherein the authentication feature information acquisition unit simultaneously executes acquisition of authentication feature information by two or more authentication feature information acquisition units in a telephone use grasp holding state; ,
An authentication processing unit that is provided inside or outside the mobile phone, and that performs authentication processing of the person to be authenticated based on the individual authentication feature information acquired by each of the two or more authentication feature information acquisition units,
If the acquisition of the authentication feature information is not performed for at least two designated authentication feature information acquisition units at the same time, it is characterized in that the acceptance authentication of the person to be authenticated is not performed.

  "Telephone functions other than authentication processing" refers to general mobile phone functions that require a call function. One or more functions such as e-mail creation / transmission function, still image or video shooting function, and videophone function Can be added.

  In the mobile phone authentication technologies such as Patent Document 1 and Patent Document 2, fraud such as the above-described false authentication is possible only by gathering a plurality of authentication forms, and in a living state. This is because no special consideration is given to the identification of whether or not the authorized user is directly performing the authentication operation.

  According to the present invention, at least two types are selected from the four types of contact biometric feature information detection unit, face photographing camera, bone conduction sound detection unit, and air conduction sound detection unit as the mother group of the authentication feature information acquisition unit. To be installed on the mobile phone. And in the telephone use grasp holding state, acquisition of the authentication feature information by the specified at least two authentication feature information acquisition unit is executed at the same time, and if the acquisition of the authentication feature information is not made at the same time, Since acceptance authentication (for example, authentication that the user is an authorized user) is not performed, it is difficult to break through the security unless the living person directly operates. As a result, it is possible to effectively eliminate fraud in which feature information obtained by substitutes or the like is sequentially given to individual authentication feature information acquisition units, and the security level can be further improved.

  For the meaning of “simultaneously executing acquisition of authentication feature information by two or more authentication feature information acquisition units”, for example, separate control of information acquisition processing by each authentication feature information acquisition unit Of course, it includes a mode of performing "simultaneously" in a literal sense, such as in parallel on a computer. However, in the present invention, in the case where individual information acquisition processing by different authentication feature information acquisition units is sequentially performed within the total period from the start of the information acquisition processing for authentication to completion. Even if there is a redundant period that does not belong to any information acquisition processing period (for example, the information acquisition process by the second authentication feature information acquisition unit after the information acquisition process by the first authentication feature information acquisition unit is completed) Information acquisition processing by each authentication feature information acquisition unit is “simultaneously” when the ratio of the waiting time until the start of the authentication is limited to 50% or less (preferably 10% or less) Define what has been done. In other words, it is only necessary to give the criminal who is going to perform substitute false authentication a time allowance for exchanging the “substitute”. Also, when individual information acquisition processing by two or more authentication feature information acquisition units is performed by time-division parallel processing, the information acquisition processing is assumed to be performed at the same time.

  In addition, for the function of “not accepting and authenticating the person to be authenticated when authentication feature information acquisition is not performed for at least two specified authentication feature information acquisition units at the same time”, the authentication It is possible to determine the synchronization of feature information acquisition using a dedicated processing routine, and to actively control the acceptance / rejection of authentication based on the determination result. It is also possible to determine the sequence of information acquisition processing so that the information itself acquired in the case of non-simultaneous is formed, and as a result, it is possible to realize that the acceptance authentication cannot be obtained. For example, if the time and the number of steps allocated to the acquisition processing of a plurality of pieces of authentication feature information are limited to a level that cannot be handled by sequential information input, when the information input is performed sequentially, time series Therefore, at least a part of information inputted later inevitably protrudes from the time and the number of steps. In this case, the corresponding information acquisition unit forcibly advances the information acquisition within the above time or number of steps even if the information source is absent, so the acquired information is blank or some information Even if it is acquired, it will not be meaningful information that will allow you to obtain acceptance authentication, but will be wrecked. In this case, since the information formed in this way is supplied to subsequent authentication processing without being particularly distinguished from regular information, for example, it is inevitably rejected authentication, so that determination of simultaneity is unnecessary. it is obvious.

  The contact-type biometric feature information detection unit can be configured by, for example, a known fingerprint detection unit. The contact-type biometric feature information detection unit can be configured by a contact detection sensor that detects a contact state between the mobile phone and the hand in the telephone use grasp holding state. Of course, it can be combined with a fingerprint detection unit.

  The contact detection sensor may be a device that binaryly detects the presence or absence of contact, such as a proximity switch, but in this case, it is difficult to give the contact detection information itself remarkable personal identification, The usage form on the authentication system is auxiliary, such as only detecting a continuous grasping and holding state (of course, it is easy to avoid fraud). On the other hand, in order to obtain hand biometric information with higher personal identifiability, a surface-type contact sensor that detects information on a mobile phone contact distribution and grasping pressure distribution can be used as the hand biometric information. In this case, the authentication processing means may perform authentication based on information on contact distribution or grasped pressure distribution detected by the surface contact sensor. There are individual differences in the form of grasping by the hand of the mobile phone depending on the size of the user's hand, solid shape, grasping force, and grip of the grip, etc., which are reflected in the contact distribution on the surface of the mobile phone and the distribution state of the grasping pressure Is done. Therefore, if this information is detected by a surface contact sensor, it can be utilized as feature information for authentication with high personal identification. Moreover, since the above distribution information (especially pressure distribution) is information peculiar to the living person, the substitution false authentication by the cut arm or the replica is very difficult.

  Note that the surface contact sensor can be provided so as to cover a surface to be grasped of the casing of the mobile phone. As a surface-type contact sensor suitable for pressure distribution detection, a sheet-like pressure sensor module in which a plurality of pressure-sensitive contacts whose contact resistance is changed by pressing force is dispersedly arranged in a sheet can be used (for example, Fujikura Technical Report). 104 (April 2003), pp. 32-36). Use of such a sheet-like pressure sensor module has an advantage that pressure distribution information with high individual identification can be obtained directly.

  Next, as a telephone use holding state that can be adopted in the present invention, a holding state used during an ordinary call, that is, a face contact type holding state that is a holding state in which a mobile phone handset is placed on the face is used. (See FIG. 29). The advantage of using the face-contact type holding state as the holding state for authentication is that the user can hold and authenticate the phone with the same natural feeling as when using the mobile phone as the original phone. Therefore, the acquisition state of the authentication feature information is less likely to vary, and it is easy to form a highly accurate authentication record. However, since the handset is placed in contact with the face, it is clearly unsuitable for photographing a face with a camera provided in the mobile phone. Therefore, as the feature information acquisition unit for acquired information authentication, two or more members selected from the group consisting of a contact-type biological feature information detection unit, a bone conduction sound detection unit, and an air conduction sound detection unit are used in combination. Even if a face photographing camera is not provided, the face photographing camera may not be used in the mode in which the authentication feature information is acquired in the face contact type holding state.

  In the face-contact type holding state, it is most natural to hold and talk with a mobile phone by hand. Therefore, as a feature information acquisition unit for authentication, a contact-type biological feature information detection unit, a bone conduction sound detection unit, Using in combination with at least one of the sound guide detection units makes it difficult to create a unique sense of tension or discomfort when faced with authentication, makes it possible to smoothly input authentication information, and to obtain accurate authentication results. There are advantages. In this case, the authentication feature information acquisition control means can simultaneously acquire at least one of the air conduction sound and the bone conduction sound as the authentication feature information and the biological information of the hand.

  In the face contact type holding state, since the telephone is brought into contact with the face, there is an advantage that the bone conduction sound can be utilized as the authentication feature information. In this case, both the bone conduction sound detection unit and the air conduction sound detection unit are used as the authentication feature information acquisition unit, and the authentication feature information acquisition control unit generates the sound generated by the bone authentication processing target person as the bone conduction sound. By detecting simultaneously by a detection part and an air conduction sound detection part, it can be comprised as what acquires both bone conduction audio | voice information and air conduction audio | voice information as authentication characteristic information.

  Conventionally, in the authentication method based on speaker recognition, regarding the voice detection step, only the detection accuracy due to noise or the like is considered, and the air conduction sound emitted from the vocal cords through the airway into the air (in the present invention, this is referred to as “ Whether the air conduction sound is detected by a normal microphone or the bone conduction sound is detected by a dedicated bone conduction microphone can be appropriately selected according to the sound environment in which the system is used. It was considered good and there was no idea of using both together.

  However, air conduction sound is air as a medium through which sound waves are transmitted, whereas bone conduction sound medium is a human tissue that is interposed between a bone conduction sound detection unit (specifically, a bone conduction microphone) and a vocal cord. It is a skeleton and the acoustic impedance structure is completely different. As a result, the detected speech waveform is also affected, and there are considerable differences between the air conduction sound, the conduction sound, and the detection waveform even though the sound sound is emitted from a common vocal cord. The propagation path of bone conduction sound is complicated and inhomogeneous compared to air of air conduction sound medium because human tissue and skeleton are involved, and parameters that affect sound propagation such as propagation speed, amplitude, and acoustic resonance frequency Therefore, the original sound waveform from the vocal cords undergoes much greater alteration than the air conduction sound in the process of propagation as a bone conduction sound. Naturally, there are individual differences in the human body tissue and skeleton serving as a propagation path, and accordingly, there is a unique difference in the waveforms of the air conduction sound and the bone conduction sound.

Therefore, the present inventor has paid attention to such a difference between the bone conduction voice information and the air conduction voice information, and has found that combining the both produces various epoch-making effects in personal authentication technology. . Specifically, the following specific effects that cannot be achieved by the bone conduction voice information and the air conduction voice information alone are produced.
(1) It becomes possible to newly grasp characteristic information derived from the difference between the two waveforms, which could not be known solely by the bone conduction sound and the air conduction sound. As a result, the security level of personal authentication can be significantly increased.
(2) Since both the bone conduction voice information and the air conduction voice information are the same kind of voice information as the information type, it is easy to share the processing of hardware and software, resulting from the difference in waveform It is also easy to extract feature information by calculation.
(3) The bone conduction sound is relatively difficult to accurately reproduce by recording or the like because of the human body contact at the time of detection. If it is configured to sample this and the air conduction sound at the same time, the living person directly Unless it is operated, it becomes very difficult to break through security.
(4) By simultaneously detecting the sound generated by the person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit, the sound sources of the bone conduction sound and the air conduction sound become the same, and are uttered separately. Compared to the case where the detected voice is detected as bone conduction sound or air conduction sound, the correlation between the bone conduction sound and the air conduction sound as a speech waveform is strengthened. Components, that is, feature information that can be used for authentication can be grasped more clearly, and authentication accuracy can be improved.

  The authentication processing means in the case of performing authentication with the air conduction sound and the bone conduction sound includes the standard speech feature information as the collation destination of the collation source voice feature information based on both the bone conduction speech information and the air conduction speech information. The stored standard voice feature information storage unit and collation means for collating the source voice feature information with the standard voice feature information can be configured. Standard voice feature information is created in advance based on the air conduction sound information and the bone conduction sound information of the authentication specific target person (the person who should be accepted and authenticated (that is, the person who should be authenticated as “correct”)), By using this as a verification destination of verification source voice feature information acquired from the authentication processing target person at the time of authentication, it is possible to simplify the authentication processing and improve accuracy. In the case where authentication is performed using a phase difference as described later as standard voice feature information, the standard voice of the person to be authenticated is used as a bone conduction sound detection unit and an air conduction sound detection unit provided outside the system. It is also possible to detect and create by. However, from the viewpoint of reducing the influence of differences in characteristics between hardware, standard voice feature information is detected and created by the bone conduction sound detection unit (provided in the system itself) and the air conduction sound detection unit. Is more effective, and the process of creating standard audio feature information is naturally simplified.

  The audio feature information can include a frequency spectrum of bone conduction sound and a frequency spectrum of air conduction sound. In this case, the collation means collates the frequency spectrum with each standard frequency spectrum of the bone conduction sound and the air conduction sound included in the standard voice feature information, and accepts the collation match result in both of them. It can be authenticated. Even if the voice of the same person, the frequency spectrum of bone conduction sound and the frequency spectrum of air conduction sound are different from each other, so the frequency spectrum of bone conduction sound and air conduction sound is compared with the corresponding standard frequency spectrum. This increases the accuracy of personal authentication. This effect is obtained by simultaneously detecting the sound generated by the person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit for both the frequency spectrum and the standard frequency spectrum to be authenticated. Increased especially when used. Since verification is performed using the frequency spectrum of both bone conduction sound and air conduction sound, the authentication method includes characteristic information derived from the difference between the two waveforms, which cannot be specified by each waveform alone. Become.

  On the other hand, the personal authentication system of the present invention is a composite that can be calculated only when both the bone conduction sound waveform detected by the bone conduction sound detection unit and the air conduction sound waveform detected by the air conduction sound detection unit are used. It can also be configured as having a composite voice feature information calculation means for calculating voice feature information. In this case, the authentication processing means can perform authentication processing based on the composite voice feature information. This method is nothing but a method of extracting and grasping the feature information derived from the difference between the two waveforms that cannot be specified by each waveform of the bone conduction sound and the air conduction sound as composite voice feature information. The improvement effect of the authentication accuracy and the security level by the combination of voice information can be further enhanced.

  The composite voice feature information calculation means can calculate the phase difference between the air conduction sound waveform and the bone conduction sound waveform as the composite sound feature information. As described above, in the human body tissue and skeleton that are the propagation paths of the bone conduction sound, the biological characteristics of the individual are directly reflected in the distribution state of the acoustic impedance. Specifically, because the living wave (that is, the individual to be authenticated) differs in the formation of reflected waves and the phase delay at impedance discontinuities (eg, tissue boundaries), the bone conduction sound waveform is Each individual to be authenticated with respect to the air conduction sound waveform has a different phase difference, and has personal identification. Therefore, if the phase difference between the air conduction sound waveform and the bone conduction sound waveform is obtained by calculation, this can be used as effective and important information for multi-person authentication. In this case, in order to accurately calculate the phase difference, it is necessary to use the bone conduction sound and the air conduction sound that are simultaneously detected for the same sound.

  In this case, the phase difference between the air conduction sound waveform and the bone conduction sound waveform peculiar to the authentication identification subject specified in advance is obtained as a standard phase difference, and the authentication processing means determines that the calculated phase difference is the standard level. Authentication processing can be performed based on whether or not it matches the phase difference. The waveform phase difference itself is a relatively simple waveform calculation (for example, a method of calculating a difference or addition waveform by setting the phase difference between two waveforms in various ways to obtain a phase difference that minimizes or maximizes the integrated amplitude). There is an advantage that the calculation load can be reduced as compared with spectrum matching or the like.

  Since the air conduction sound and the bone conduction sound also have a difference in frequency spectrum, more accurate calculation of the phase difference is possible by obtaining the phase difference by extracting the frequency components that are included in both waveforms in common. . In this case, the extraction of the frequency component can be performed using a known digital filter technique.

  The composite voice feature information is not limited to the phase difference between the two waveforms as described above. For example, the difference spectrum of each frequency spectrum between the air conduction sound and the bone conduction sound can be used. Bone conduction sound has individual differences in acoustic characteristics such as attenuation or resonance of the human body intervening in the propagation path. As a result, frequency components that are insufficient or emphasized with respect to air conduction sound also vary depending on the individual. Therefore, the difference spectrum between the air conduction sound and the bone conduction sound has personal identification. There are also spectra obtained equivalently by mathematical operations of individual frequency spectra and the above difference spectra, such as the common spectrum of air conduction sound and bone conduction sound (subtracting the above difference spectrum from each frequency spectrum). Naturally, it can be used as composite voice feature information.

  The cause of the phase difference and difference spectrum as described above is mainly due to the skeleton and the mechanical structure of the human tissue that form the propagation path of the bone conduction sound. There is an advantage that misidentification is less likely to occur even if some alteration occurs.

  Further, the authentication processing means performs the authentication process by performing a first authentication process for comparing at least one of the frequency spectrum of the bone conduction sound and the frequency spectrum of the air conduction sound with the standard frequency spectrum, and the first based on the composite voice feature information. It can also be implemented in combination with the second authentication process. The conventional voice authentication method based on either the frequency spectrum of the bone conduction sound or the frequency spectrum of the air conduction sound has high personal identification by the method of spectrum matching, but it is also used for misrepresentation using recording etc. There are also security holes. However, it is possible to effectively prevent the occurrence of the security hole as described above by combining the authentication processing based on the composite voice feature information (particularly, a phase difference that is easy to calculate) as described above.

  In the above, several authentication modes in the face contact type holding state have been exemplified and described. However, in recent mobile phones, in addition to simple call functions, e-mail creation / transmission functions, camera / video shooting functions, videophone functions, etc. have been added one after another. Has also been used frequently. The most common usage holding mode next to the face contact type holding state is a face-to-face holding state in which the user's face is held facing the main display screen by a liquid crystal panel or the like (see FIG. 30). . This type of retention is used when creating e-mails, using the Internet, and when taking still images and videos with the camera provided on the back of the phone, opposite to the main display screen. is there. In this case, since the telephone does not contact the user's face, it is clearly unsuitable for obtaining bone conduction sound. Accordingly, the cellular phone is not provided with the bone conduction sound detection unit as the authentication feature information acquisition unit, or even if provided, in the mode in which the authentication feature information is acquired in the face-to-face holding state. The bone conduction sound detector is not used.

  In this case, as the authentication feature information acquisition unit, any two or more types can be selected from the group consisting of an airway sound detection unit, a contact-type biometric feature information detection unit, and a face photographing camera. In the holding state, the user's face can be displayed on the display unit and authentication photography can be performed, so the following can be exemplified as specific authentication usage forms. First, the mobile phone is provided with a display unit that displays a face image of the person to be authenticated that is photographed by the face photographing camera. The face photographing camera is attached to a position where a face can be photographed when the face of the person to be authenticated and the display unit face the mobile phone. And, as the telephone use holding state, using the face-to-face holding state in which the display unit of the mobile phone and the camera for photographing the face directly face the face of the person to be authenticated, the authentication feature information acquisition unit, A face photographing camera is essential, and this is used in combination with at least one of a contact-type biological feature information detection unit and an air conduction sound detection unit. As a result, it is possible to make a face image indispensable and combine it with either airway sound or biological information of the hand at the same time, and authentication with a high security level is possible.

  Various authentication forms can be adopted as the face-to-face holding state. For example, as a feature information acquisition unit for authentication, a face photographing camera and an air conduction sound detection unit are used in combination. A configuration is also possible in which the information acquisition control means acquires the face-captured image and the air conduction sound information simultaneously as the authentication feature information. In this case, during voice input, the mouth moves in the face image, and it is difficult to satisfy the exact synchronization of information acquisition, but the redundant waiting time is shortened as described above, and the face photographing process and voice input are performed. If the accompanying air conduction sound detection is carried out continuously without putting in a gap, the simultaneity defined in the present invention can be sufficiently satisfied. In this case, in order to more effectively prevent alternative false authentication and the like, the authentication feature information acquisition control means includes a confirmation process of the detection state of the face image by the face photographing camera, a detection process of the air conduction sound information, It is good to execute by alternately repeating.

  Further, when using a face image, considering the simultaneous acquisition of the authentication feature information acquisition, the face photographing camera and the contact-type biometric feature information detection unit are used in combination as the authentication feature information acquisition unit. It is also a desirable aspect that the authentication feature information acquisition control means is configured to simultaneously acquire the face photographed image and the hand biometric information as the authentication feature information.

  In the personal authentication system of the present invention, the authentication feature information acquisition unit is configured by a combination of a contact-type biometric feature information detection unit and at least one of a face photographing camera, an air conduction sound detection unit, and a bone conduction sound detection unit. In this case, the authentication feature information acquisition control means performs the biometric information of the hand by the contact-type biometric feature information detection unit before and after the acquisition process of at least one of the face image, the air conduction sound information, and the bone conduction sound information. It is possible to perform a contact change confirmation process for examining the detected state change. That is, as the contact change confirmation process, for example, before and after the acquisition process described above, by detecting the biological information of the hand twice and checking whether the detected biological information has changed, including during the acquisition process. It can be confirmed whether or not the holding state of the mobile phone (this can be applied to either the face contact type or the face-to-face type) is maintained. This is advantageous in preventing a security breakthrough due to sequential alternative false authentication.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In this embodiment, a case where the function of the personal authentication system of the present invention is incorporated in a mobile phone will be described as an example. FIG. 1 is an external perspective view showing an example of a mobile phone 1. The mobile phone 1 is provided with a receiver 303 on the upper side of the main body and a transmitter 304 on the lower side, and a liquid crystal display device (for example, a color liquid crystal display device) between the two. An on-hook / off-hook switch 306 that switches the liquid crystal monitor 308, the input unit 305, and the mobile phone 1 between an on-hook state and an off-hook state is provided. In the present embodiment, the mobile phone 1 can access not only a line telephone communication network but also an information communication network such as the Internet. The input unit includes a call dial key 305a that is also used as an information input keyboard, a cursor movement key 305b, and a mode switching key 305c for switching use modes such as a call mode and an information search mode.

  The transmitter 304 is composed of a microphone that also serves as an air conduction sound detection unit. On the other hand, the handset 303 is constituted by a bone conduction speaker in the present embodiment, and a bone conduction microphone 340 as a bone conduction sound detecting unit is disposed in the vicinity thereof. The basic configuration of the bone conduction speaker is, for example, Japanese Patent No. 2967777 or Japanese Patent Laid-Open No. 2003-340370, and the basic configuration of the bone conduction microphone is, for example, Japanese Utility Model Laid-Open No. 55-146785, Japanese Patent Laid-Open No. 58-18297. Since it is well known in Japanese Patent Publication No. 63-173991 or Japanese Patent No. 34888749, detailed description is omitted. Both are used for the ear or the jawbone under the ear. All of these constitute an authentication feature information acquisition unit.

  In addition, the mobile phone 1 includes a face photographing camera 341, a surface contact sensor 343 that forms a contact-type biometric feature information detection unit, and a fingerprint detection unit 342 as other authentication feature information acquisition units. As shown in FIG. 1, the form of gripping the mobile phone 1 is almost the same as the basic form although there are variations among users. That is, four phones that are bent to the first side of the phone (left-handed for right-handed people, reverse for left-handed) so that the display unit 308 faces the inside of the hand MH While applying finger 14F, place the side part from the base of thumb MS to the lower half of the second side of the telephone (the left side for right-handed people, the opposite for left-handed people), and the thumb at the top half Hit the tip of. In order to avoid inadvertently touching the input unit 305 and to reduce the uncomfortable feeling that the fingertip hits the face, it is unconsciously held in this way. In the present embodiment, utilizing this, the fingerprint detection unit 342 is provided at a position where the tip of the thumb hits, and the surface contact sensors 343 are provided on both side surfaces.

  In the present embodiment, as shown in FIG. 3, as the surface contact sensor 343, a sheet in which a plurality of pressure-sensitive contacts SP that have already been described and whose contact resistance (or contact capacity) changes due to the pressing force is dispersedly arranged in the sheet. A pressure sensor module is used. The resistance value (pressure detection value) of each pressure-sensitive contact SP is digitally converted by a multi-bit signal, and pressure distribution information is obtained from the signal value of each pressure-sensitive contact SP. As a result, the pressure sensor distribution area PDP corresponding to the grasping pressing area by the four fingers other than the thumb is detected by the surface contact sensor 343 on the first side (here, left), and is detected on the second side (here, right). The surface contact sensor 343 detects a pressure-sensitive distribution area PDP corresponding to the grasping pressing area of the thumb (and the base of the thumb of the palm) in the pressure distribution. Since the shape (and pressure distribution state) of the pressure-sensitive distribution region PDP varies depending on the individual, it can be used as feature information. It should be noted that an integrated surface contact sensor that extends over the three surfaces of the mobile phone 1 on both sides and the back surface may be provided to detect an integrated grasping and pressing area between the finger and palm. It is necessary to change the design, for example, by removing the cover of the battery housing portion that is normally provided on the back surface, so that the battery can be inserted and removed, for example, from the bottom side of the telephone.

  As a surface-type contact sensor that can be used other than the above, an analog capacitive coupling type surface-type touch sensor having a mechanism similar to that of a known touch panel can be used. In this type of touch sensor, fine lines in which vertical lines and horizontal lines are arranged in a grid pattern so as to be in non-contact with each other are formed on the detection surface, and alternating voltages are alternately applied to the vertical lines and horizontal lines at regular intervals. In addition, the impedance change of each wiring is monitored by current detection, and the contact point coordinates on the detection surface are specified from the positions of the vertical line and the horizontal line where the impedance change is detected. In this method, it is difficult to detect the pressure applied to the contact point, which is suitable for specifying the contact distribution state. However, when the same person grasps the mobile phone with different forces, the contact area of the finger or the like changes depending on the degree of the force, so that information on the grasping force can be obtained indirectly.

  Further, as shown in FIG. 2, an input unit pressure sensor 323 that detects contact with the input unit 305 may be provided as a contact-type biometric feature information detection unit.

  Returning to FIG. 1, the face photographing camera 341 is composed of, for example, a CCD camera, and the mobile phone 1 is positioned so that the face can be photographed when the face of the person to be authenticated and the display unit 308 are facing each other. The display unit 308 is provided in the vicinity. This is because the face image for authentication must have a necessary part of the face within the field of view of the camera 341. Therefore, a finder image for photographing that is captured by the camera 341 is displayed on the display unit 308 and has a posture suitable for authentication. This is so that an image can be taken while confirming whether or not an image can be obtained (for example, as shown in FIG. 13, the face fits within a prescribed frame F in the display unit 308 and the line of sight is aligned with the reference line SL). is there. It is also possible to provide a retina photographing camera instead of the face photographing camera 341 and use the retina image as the authentication feature information. In addition to the retina image, an iris image can be taken and used as authentication feature information. When an iris image is used, collation / authentication is performed using the personality of the pattern and color. In particular, the iris pattern is an acquired component and has a low genetic influence, so there is a significant difference even in identical twins, and there is an advantage that it can be reliably identified. An authentication method using an iris pattern has features that it can quickly recognize and collate and has a low misidentification rate. An iris can be photographed using a normal camera. In this case, a dedicated camera may be provided instead of the face photographing camera 341, or an attachment for iris close-up photography is attached to the face photographing camera 341. It is also possible to attach and take a picture.

  FIG. 2 is a block diagram showing an example of the electrical configuration of the mobile phone 1. The main parts of the circuit are an I / O port 311 and a CPU 312 (which constitutes an authentication feature information acquisition control unit, an authentication processing unit, a verification unit, and a composite voice feature information calculation unit), a ROM 313 and a RAM 314 ( A control unit 310 including a bone conduction voice information storage unit and an air conduction voice information storage unit). The input unit 305 and the on-hook / off-hook switch 306 described above are connected to the I / O port 311. The receiver 303 is connected via an amplifier 315 and a D / A converter 316, the transmitter 304 is connected via an amplifier 317 and an A / D converter 318, and the bone conduction microphone 340 is connected to an amplifier 320 and an A / D converter. Each of them is connected to the I / O port 311 via the H.321. A telephone connection circuit 323 is connected to the I / O port 311. The connection circuit 323 includes a connection interface 331 for connecting to the control unit 310, a modulator 332, a transmitter 333, a frequency synthesizer 334, a receiver 335, a demodulator 336, a duplexer 337, and the like connected thereto. ing. The data signal from the controller 310 is modulated by the modulator 332 and further transmitted from the antenna 339 via the duplexer 337 by the transmitter 333. On the other hand, the received radio wave is received by the receiver 335 via the antenna 339 and the duplexer 337, demodulated by the demodulator 336, and then input to the I / O port 311 of the control unit 310. When making a call, for example, an audio signal input from the transmitter 304 is amplified by the amplifier 317, further digitally converted by the A / D converter 318, and input to the control unit 310. The signal is processed by the control unit 310 as necessary, and then output from the receiver 303 via the D / A converter 316 and the amplifier 315.

  On the other hand, a control radio wave transmitter 338 that transmits a control radio wave P is connected to the connection interface 331. The control radio wave P is transmitted from the antenna 339 via the duplexer 337. When the mobile phone 1 moves to another communication zone 102, the network-side radio network controller 104 performs a well-known handover process based on the reception status of the control radio wave P.

  To the I / O port 311, a face photographing camera 341, a fingerprint detection unit 342, and a surface contact sensor 343 are connected. The resistance change of each contact SP (FIG. 3) of the surface contact sensor 343 is input to the digitizer 344 as an analog voltage signal, converted into digital data of pressure for each contact SP, and input to the I / O port 311.

  Next, in the ROM 314, a communication program, which is a basic control program for wireless telephone communication, and a display program for controlling the screen display of the liquid crystal monitor 308 are installed. Further, as shown in FIG. 4, in ROM 314, an authentication processing means is realized by being executed by CPU 312 for authenticating whether or not the user of mobile phone 1 is a regular user. Is also installed. In the present embodiment, the authentication processing is specifically performed by speaker recognition / collation processing using both the speech waveform of the air conduction sound and the speech waveform of the bone conduction sound. The authentication program includes a main program 201 and a group of submodules used by the main program 201. Specifically, the air conduction sound sampling module 202, the bone conduction sound sampling module 203, the air conduction sound / bone conduction sound phase difference. Calculation / collation determination module 204, air conduction sound / bone conduction sound spectrum calculation / collation determination module 205, face image sampling module 207, face image collation / determination module 208, fingerprint sampling module 209, fingerprint collation / determination module 210, It consists of a pressure-sensitive distribution measurement module 211 for detecting a grasping pressing area, a pressure-sensitive distribution collation / determination module 212, and the like. These programs are all executed by the CPU 312 using the RAM 313 in FIG. 2 as a work area.

  Further, as authentication master data 322, audio spectrum master data, specifically air-conducted sound spectrum master data used when authentication by voice is performed in spectrum matching processing (modules involved are reference numerals 205 and 206). 321, bone conduction sound spectrum master data 222 and master data 223 of their difference spectrum are prepared. Also, face image master data 224, fingerprint master data 224, and pressure-sensitive distribution master data 226 are prepared. In the case of the air conduction sound and the bone conduction sound, these data are obtained by a regular user (authentication specific target person) with a sound (“ON”) predetermined for verification, A word or sentence is pronounced, and this is generated by detecting the waveform with the handset 303 (air conduction sound) and the bone conduction microphone 340 (bone conduction sound) and performing a known Fourier transform operation to obtain a spectrum. is there. Further, the face image master data 241 (FIG. 10), the fingerprint master data 243, and the pressure-sensitive distribution master data 226 (FIG. 12) are also received from the authorized user by the face photographing camera 341, the fingerprint detection unit 342, and the surface-type contact sensor 343, respectively. What was acquired in advance is prepared. These data are different for each user, and the rewritable ROM, specifically, the EEPROM shown in FIG. (Electrically Erasable Programmable Read Only Memory) 322 is stored so as to be rewritable, and is loaded into the authentication data memory of the RAM 313 and used as necessary.

  In the following description, a plurality of specific voice authentication methods will be described. However, there are modules and data that are not particularly used depending on the method, and therefore, necessary modules and data are selected and used ( Of course, it is possible to omit modules and data not used in the authentication method).

  Since the method for using the mobile phone 1 is well known for the telephone portion, detailed description thereof is omitted, and authentication processing prior to use will be described in detail below. FIG. 10 is a flow of authentication main processing by the main program 201 (FIG. 4). In order to perform the authentication process, it is necessary to perform an initialization process including registration of data for verification (S1). This initialization process is skipped once it is performed, except when the verification master data is updated. S3 and S4 are authentication processes that form the center of the process. Based on the authentication result, an authentication flag indicating whether or not the function use of the mobile phone 1 is permitted is set in, for example, the RAM 313 (FIG. 2). In S5, the authentication flag is read, and when the prescribed condition is satisfied, the lock is released (S7: that is, usage is permitted), and when not satisfied, the lock is released (S8: that is, that the usage is not permitted). .

  In this embodiment, the authentication process includes a first authentication process (S3) that is a process in the face contact type holding state as shown in FIG. 29 and a second authentication process that is a process in the face-to-face holding state (S3). This is performed in two stages with S4). The first authentication process and the second authentication process can be switched in order, and only the first authentication process is performed (that is, S4 is omitted in FIG. 15), or only the second authentication process is performed ( That is, S3 is omitted in FIG.

  The function of the cellular phone 1 that is unlocked by authentication is not limited to a well-known telephone function (including connection to a telephone communication network or the Internet, a mail function, etc.), for example, lock / unlock of a car. Or, it can be a wireless remote control unit function for automobile functions, such as starting the engine, turning on / off the headlights and interior lights.

  Before entering into a specific description of the authentication process, the flow of each process of the initialization process and the voice recognition process will be described with reference to FIGS. In either case, the main part of the process consists of voice data processing responsible for acquisition and processing of voice data (S301 in the initial process and S402 in the voice authentication process). The audio data processing will be described in detail first with reference to FIG. In the speaker authentication technology, various methods have been devised for causing the authentication target person to pronounce the voice for authentication for the purpose of improving security and the initial data acquisition method differs depending on the method, but both methods are well known. Therefore, only an outline will be described.

(1) Method of uttering only one character (or sound (for example, vowel)) A character to be uttered is designated and generated by display or the like, and sampling is performed.
(2) Method of sequentially uttering by combining multiple characters Basically the same as (1). The order of utterance is guided by display or the like, and waveform sampling is performed sequentially. During actual verification, the utterance order may be fixed, or the utterance order may be changed every time using a random number (in the latter case, the utterance order of characters designated at the time of authentication is changed randomly) There is an advantage that even if you record what you say in a fixed order, you can use it.
(3) Method of uttering a word There may be only one type of word to be used (in this case, it is the same as (2)), or there is a method of selecting from a plurality of types. In the latter case (refer to FIG. 1 below), a selection list of words to be collated is displayed on the screen 108 and selected by the input unit 305, and then the selected word is uttered and sampled. In addition, there is a method in which the number of characters (or recording time) is designated, a user's favorite word is arbitrarily input by the input unit 305, and utterance / sampling is performed. In this case, it is clear that the word is substituted for the password. Further, as a more elaborate method, there is a method in which a question that only a legitimate user can understand is voice-output and a registered answer corresponding to this is voice-inputted. In this case, in the initialization process, it is necessary to input or select the question contents to be output and the answer contents to the question contents.
(4) Method of inputting a sentence This is basically the same as (3), and when a question / answer format is adopted, there may be a method of inputting a plurality of questions and answers in an interactive format.

  When comparing the bone conduction sound and the air conduction sound, the sound component derived from the vocal cord vibration such as a vowel tends to be emphasized from the air conduction sound because the bone conduction sound is closer to the vocal cord. In addition, since the frictional sound and the plosive sound are associated with sound-generating elements other than the vocal cords such as the tongue and lips, the air conduction sound appears more emphasized. Therefore, when performing authentication based on the difference in the waveform or spectrum between bone conduction sound and air conduction sound (especially the difference spectrum), as speech waveform data (bone conduction sound and air conduction sound) to be authenticated, Sounds that include vowels, friction sounds, and plosives (preferably, sound strings in which the most abundant sounds are one of these sound types: for example, “Sashisuseso”, “Sushishinchu no Mushi”, “Aiueo”, etc .: Of course, it may be desirable to specify “sa line”, “ta line” or “a line”. In addition, even in the same vowel, sounds such as “i, e” that use the front part of the tongue for articulation are air conduction sounds, and conversely, sounds such as “u, o” that use the back part of the tongue are bone conduction sounds. Therefore, it is also effective to specify a sound string mainly including either the former or the latter, such as “No (house)” and “Koubo (yeast)”.

  Returning to the description of FIG. 17, in S <b> 501, input of the designated voice is input using both the transmitter 304 and the bone conduction microphone 340. In S502, the sampling is performed (implemented by executing the air conduction sound sampling module 202 and the bone conduction sound sampling module 203 of FIG. 4). Since the user emits the requested sound sequence only once, the sampling must be performed simultaneously in time series (therefore, in the first authentication process described later performed using this, two authentications are used) It is clear that the acquisition of the bone conduction sound and the airway sound that make up the characteristic information is performed simultaneously). In this case, when a single CPU is used, it is executed as parallel processing by time division as shown in FIG. Specifically, the sampling counter is reset in S101, and thereafter, while the sampling counter is incremented, reading of the microphone input port for air conduction sound (S102) and writing of the read value to the memory (RAM 313) (S103), Read of the input port of the bone conduction microphone (S104) and writing of the read value to the memory (S105) are repeated alternately. Depending on the length of the audio data to be sampled, the total sampling time (the value of the sampling counter can be used instead, but other timer means may be used), and the sampling is aborted when the time is up (S107) Unless both the bone conduction sound waveform and the air conduction sound waveform are sampled simultaneously, it is impossible to normally obtain the data of both sounds, for example, by sequential sound input using a tape recorder or the like. It is possible to effectively prevent deception and the like.

  In addition, when inputting speech data using words or sentences, it is determined whether or not the input of speech with a predetermined content (meaning) is completed by using a well-known speech recognition technology, and if completed, sampling is terminated. It can also be configured as follows. In this case, the timer means is not always necessary. Although the hardware is somewhat complicated, the sampling of the air conduction sound and the bone conduction sound can be performed independently by separate (ie, two) CPUs. Both audio waveforms can be sampled in parallel without processing.

  Returning to FIG. 17, when the sampling of the respective sound waveforms of the air conduction sound and the bone conduction sound is completed as described above, it is checked in step S <b> 503 whether or not each sound has been sampled simultaneously. There are various possible check methods. For example, if the air conduction sound and the bone conduction sound are input at a deliberately shifted timing, one of them will protrude outside the sampling time, and the acquired data will have a large blank period. There is a way to take advantage of this because it should happen. In this case, it is checked whether at least one of the acquired air conduction sound waveform and bone conduction sound waveform has a period during which the audio amplitude is equal to or lower than a predetermined lower limit value continuing for a certain period. If is present, it is determined that there is no simultaneity. If it is determined in S503 that there is no simultaneity, the process proceeds to S511, where the processing is terminated and an error or warning is output.

  If simultaneity is satisfied, the process proceeds to S505 and S506, and the detected air conduction sound waveform data and bone conduction sound waveform data are stored and registered in the memory. The following is a calculation process of the composite voice feature information used for authentication (the function of the composite voice feature information calculation means is realized). In S507, the phase difference between the air conduction sound waveform and the bone conduction sound speech waveform is calculated as the composite sound feature information (implemented by executing the air conduction sound / bone conduction sound phase difference calculation / collation determination module 204). As shown in FIG. 8, the air conduction sound waveform and the bone conduction sound waveform are obtained by simultaneously sampling the same sound by individual microphones, and both waveforms when the waveforms are superimposed on the basis of the sampling start timing. Is the reference superposition phase. Since the two waveforms include many common frequency components based on the same sound, as shown in FIG. 9, the overlapping phase of both waveform data is the phase difference (inherently present in the reference overlapping phase) ( That is, if the differential waveform is calculated by relatively shifting so as to eliminate the phase difference (φ to be obtained), the integrated amplitude (average amplitude) of the differential waveform is minimized by the superposition phase ( (See the bottom of FIG. 9). Therefore, if the superposition phase of both waveform data is changed variously while calculating the integral amplitude of the difference waveform and the superposition phase where the integral amplitude is minimized is found, this is obtained as the phase difference φ between both waveforms to be obtained. Obtainable.

In consideration of use as personal feature information used for authentication processing, it is only necessary to obtain a parameter that uniquely corresponds to the phase difference φ to be obtained. The phase difference is not limited to the following, and the following can be substituted.
(1) Phase difference that maximizes the integrated amplitude of the differential waveform
(2) Phase difference that minimizes the integrated amplitude of the sum waveform
(3) Phase difference that maximizes the integrated amplitude of the added waveform

  Hereinafter, the process of obtaining the phase difference φ that minimizes the integral amplitude of the differential waveform will be described with reference to the flowchart of FIG. In S201, the superposition phase difference Σt (the waveform is a superposition of various sinusoidal waveforms, so the calculation unit of the phase difference is not an angle but time) is reset. Next, one of the air conduction sound waveform and the bone conduction sound waveform is set as the first waveform, and the other is set as the second waveform, and the phase of the second waveform is shifted by a predetermined minute time Δt in S202 to obtain the first waveform. Is fixed, and the differential waveform is calculated in S203. In S204, the integral amplitude A of the difference waveform is calculated. The method of calculating the integral amplitude is well known, but can be calculated as follows, for example. First, assuming that the waveform is f (t), all the values of f (t) corresponding to each sampling timing t are added and divided by the number of samplings N to obtain the waveform center line f0. Then, | f (t) −f0 | is calculated for each value of t, and this is added for all t and divided by N to obtain an integrated amplitude. In S205, the value of Σt at that time is set as the phase difference φ and stored in association with the value of the integral amplitude A.

  Next, in S206, Σt is incremented by Δt, and the processes in S202 to S206 are repeated until Σt reaches a predetermined maximum value Σtmax. Considering that the user can speak naturally as the voice designated for authentication, it is desirable to secure the length of the voice sample, for example, for 1 second or longer. The amount of waveform shift required to find the phase difference is sufficient for 0.5 to 2 wavelengths. Therefore, considering that the human voice frequency is on average 1 to 2 kHz, Σt is 0. It should be set to about 5 to 2 ms. The sampling period Δt is preferably about 1/1000 to 1/10 of Σt. Note that the second waveform shift interval may be calculated by setting the interval only in one direction, positive or negative, with the reference overlap phase difference as the origin, or by setting the interval in each of positive and negative. It may be.

  When the above calculation is completed, the process proceeds to S208, the stored minimum value A0 of the integrated amplitude A is found, and the phase difference φ0 corresponding to A0 is determined as the phase difference φ0 to be obtained in S209. As shown in FIG. 6, there is a considerable difference in spectrum between the bone conduction sound and the air conduction sound, and there are frequency components that are not common to each other (for example, in the case of bone conduction sound, the frequency is high). The spectral intensity of the range tends to be missing). Therefore, when calculating the phase difference, it may be desirable to perform waveform calculation after extracting a frequency region with many common components by filtering. This is the end of the description of the phase difference calculation.

  Returning to FIG. 17, in S508 and S509, the frequency spectrum of each waveform of the air conduction sound and the bone conduction sound is calculated, and the result is stored. As described above, this calculation can be performed by performing a well-known Fourier transform process on the original waveform. However, in speaker recognition, the spectral outline shown below (mainly information that reflects voice quality) is used rather than the spectral waveform containing the fine structure shown in FIG. However, it is known that it has excellent measurement reproducibility, is sufficiently effective as personal identification information, and can be easily verified. This spectral outline is also referred to as spectral envelope, and various well-known speech analysis algorithms (for example, short-time accident correlation analysis method, short-time spectrum analysis method, cepstrum analysis method, band filter bank analysis in the case of non-parametric analysis method) In the case of using a parametric analysis method such as a method or a zero-crossing product method, extraction and calculation can be performed by a linear prediction analysis method, a maximum likelihood spectrum estimation method, a covariance method, a PARCOR analysis method, an LSP analysis method, or the like.

  Returning to FIG. 15, in S510, as shown in FIG. 6, the difference in frequency spectrum between the air conduction sound and the bone conduction sound obtained as described above is calculated and stored as difference spectrum data. The above processing is performed by executing the air conduction sound / bone conduction sound difference spectrum calculation / collation determination module 205 and the waveform spectrum comparison / determination module 206 of FIG. This is the end of the description of the audio data processing.

Returning to FIG. 16, the flow of the initialization process will be described.
In the voice data processing of S301, voice input is performed by the voice of the authorized user (authentication specific target person), and the phase difference, the air conduction sound or the bone conduction sound frequency spectrum or the difference spectrum data is obtained by the method described above. In step S302, these are registered in the EEPROM 322 (FIG. 4) as master data (standard voice feature information: standard phase difference, standard frequency spectrum, or standard difference spectrum) used in the subsequent voice authentication process. In S303 to S305, the face image master data 241 (FIG. 10), the fingerprint master data 243 (FIG. 11), and the pressure-sensitive distribution master data 226 (by the face photographing camera 341, the fingerprint detection unit 342, and the surface contact sensor 343). FIG. 12) is acquired and registered.

  Hereinafter, the case where the bone conduction sound and the airway sound are simultaneously acquired as the authentication feature information will be described as an example for the first authentication processing. FIG. 20 shows an example. In S401, the user inputs a designated voice for authentication. In S402, the audio data processing described above is executed, and the phase difference φ is calculated. In S403, the phase difference φ is compared with the standard phase difference φ0 stored as master data. Here, the difference φ−φ0 is calculated. In S406, it is checked whether or not the deviation between the phase difference φ and the standard phase difference φ0 is within the allowable range. If the deviation is within the allowable range, the authentication flag is set to be permitted (S407). (S408). Instead of registering the standard phase difference φ0 as the master, an allowable phase difference range (given by the maximum values φmax and φmin) including the standard phase difference φ0 is registered, and φ belongs to the range. Authentication can also be performed depending on whether or not there is.

  FIG. 21 is an example of a voice authentication process using a difference spectrum instead of a phase difference (the same step numbers are assigned to the steps common with FIG. 20 and the description is omitted). In S402, voice data processing is executed. In S410, as shown in FIG. 6, the calculation result of the difference spectrum between the air conduction sound and the bone conduction sound is read, and in S411, the master data of the difference spectrum (FIG. 4: reference numeral 223). Compare with If it is determined in S412 that both match, the authentication flag is set to permit (S413), and if it is out of range, it is set to non-permitted (S414).

  As shown in FIG. 6, the air conduction sound spectrum and the bone conduction sound spectrum have the same main part, but a significant difference is observed in the spectrum intensity in a specific frequency band (for example, a high frequency component). The air conduction sound spectrum appears stronger than the bone conduction sound spectrum). Therefore, matching / mismatching matching can be performed by comparing the shape of the difference spectrum in the frequency band with that of the master. In particular, it is a spectrum envelope peak (as indicated by “x” in FIG. 6) that exists in one of the air conduction sound spectrum and the bone conduction sound spectrum and does not exist in the other, and the peak position is authenticated. When it varies depending on an individual to be detected, the peak can be detected in the difference spectrum, and verification of the peak position (frequency) can be easily performed with high accuracy authentication verification.

  FIG. 22 is an example of a voice authentication process in which each spectrum of bone conduction sound and air conduction sound is individually verified with the master (the same step numbers are assigned to the steps common with FIG. 20 and description thereof is omitted). Audio data processing is executed in S402, and calculation results of each frequency spectrum of the air conduction sound and the bone conduction sound are read out (S420, S423). These are individually compared with master data (FIG. 4: reference numerals 221, 222). Only in cases where it is determined in S422 and S425 that both the bone conduction sound and the air conduction sound match, the authentication flag is set to permit (S426), and if it is out of range, it is set to non-permitted (S427). .

  As shown in FIG. 6, each frequency spectrum of the air conduction sound and the bone conduction sound generates a unique peak position in accordance with the sound in the spectrum envelope. Therefore, the input sound is determined according to the number and position of the peaks. It is possible to identify whether (for example, a word or a character) is the same as the voice indicated by the master (that is, voice recognition). If the content of the voice is the same, the peak position and intensity (or intensity ratio between peaks) can be compared with the master, and it can be authenticated whether it is a legitimate user or not according to the match / mismatch (that is, Speaker recognition).

  Returning to FIG. 15, when the first authentication process as described above ends in S <b> 3, the process proceeds to the second authentication process (if the rejection condition is satisfied in the first authentication process, the process is performed here). May be discontinued). In the second authentication process, as shown in FIG. 30, the mobile phone 1 is changed to the face-to-face holding state (the phone 1 that has been placed on the ear as shown in FIG. If you move it down 4 turns, you can complete the change in 1 motion).

  FIG. 23 shows a first example of the second authentication process. Here, the airway sound and the face image are processed simultaneously as authentication feature information. In step S601, the face image (I) is photographed. Next, as shown in FIG. 14, a predetermined question 250 is displayed on the display unit 308, and the answer is inputted by voice. As shown in FIG. 23, face image shooting, question display, and answer voice input are repeated twice in this order, and finally a face image is shot again (S601 to S607). In S608, it is checked whether or not the face images (I) to (III) photographed three times coincide with the master (see FIG. 10). In S610 and S612, the response voices (I) and (II) (airway sound spectrum) acquired twice are collated with the master. In S609, S611, and S613, it is determined whether or not there is a match between the respective collations, and if all match, the process proceeds to S614 to set the authentication flag to “permitted” (acceptance authentication), and if there is any mismatch, the process proceeds to S615. Set the authentication flag to non-permitted (rejection authentication).

  Here, it is desirable to authenticate the first or last of the face images taken three times by collating with the master, but the remaining two need only be determined to prevent false authentication, etc. It can be replaced with a simple pattern matching process for confirming whether or not the face is out of view. An example is shown in FIG. If the two patterns have color or gradation, binarization is performed in S701. In S703 to S706, corresponding pixels between pattern frames are sequentially read, and an exclusive OR of pixel setting values (0 or 1) is calculated. If the pattern does not move, the setting value of the corresponding pixel is equal and the value of the exclusive OR is 0, and if the pattern moves, the pixel does not match and the value of the exclusive OR is 1. This exclusive OR value is repeated for each pixel and added to the counter K (S707). If there is an abnormal movement in the pattern, the number of changed pixels increases, and the value of the exclusive OR counter K also increases. In S709 to S714, the final value of K is divided by the total number M of pixels in the frame, and if this value is less than or equal to the allowable value, match determination is performed, and if it exceeds the allowable value, mismatch determination is performed.

  The process of FIG. 26 is not limited to a face image, but can be similarly applied to a fingerprint image and a pressure-sensitive distribution pattern described later (contact change confirmation process). As shown in FIG. 13, if the holding by the hand MH is canceled during the photographing of the face image 240, a change appears in the detected fingerprint image or pressure-sensitive distribution pattern, and rejection authentication can be performed as an abnormality. Further, as shown in the lower part of FIG. 13, when the face image is absent during the holding by the hand MH (that is, during detection of the fingerprint image or the pressure-sensitive distribution pattern), a change appears in the face image pattern. This can also be rejected as abnormal.

  As is clear from the flow of FIG. 23, a plurality of authentication processes using a plurality of types of authentication feature information are performed, but the acquisition of the authentication feature information is performed in a single stage before the processing and is used. The authentication process is performed in a batch at a later stage. This is because the acquisition process of the authentication feature information is continuously and quickly performed to eliminate redundant time and make it difficult to perform sequential alternative false authentication. Face image, fingerprint, pressure-sensitive distribution, and voice input are sequential in processing, but acquisition of face image, fingerprint, and pressure-sensitive distribution information is only one frame pattern acquisition process. The required time is about 1 to 10 ms, and at most 1 second is sufficient for acquiring these three pieces of information. On the other hand, for voice input, since the voice phrase input time is about 3 to 20 seconds, the ratio of the processing redundant period can be sufficiently limited to 50% or less, and the criminal who intends to perform substitute false authentication. There is no time to replace the “substitute”.

For example, prior to S601, a viewfinder image captured by the camera 341 is displayed on the display unit 308 to prompt the user to align the face with the camera 341, and a confirmation button (assigned to any key of the input unit 305 or separately) If an activation signal is given by pressing (providing an authentication button) or the like, the processing from S601 to S607 may be executed at once in a state where a break by a user operation is impossible. Also, since the response interval for the question in S602 to S605 should be a regular user, it should be possible to answer immediately, so that the process immediately proceeds to the voice sampling process, and automatically proceeds to the next step when the time required for the answer has passed. To do. On the other hand, since the image capturing in S601, S604, and S607 is all completed in an instant of about several ms, unless the notification is made especially by the shutter sound output or the display of the photographing message, the processing is on the screen from the viewpoint of the user. Every time a question is displayed, it is a flow of just inputting the answer without putting in a short time, and during that time, it is completely unaware that the face has been shot many times. As a result, the user simply exchanges voice as if having a conversation with the mobile phone 1, and does not actually feel that complicated processing including image matching is performed internally, and the authentication process is performed in a simple mood. Can finish. Questions and answers are more effective if they are related to each other with a series of stories. An example is shown below. Needless to say, the answer is set so that only authorized users can answer.
(Example 1)
(Question (I)) “Who do you like?”
(Answer (I)) "Kaoru-chan"
(Question (II)) “How much do you like?”
(Answer (II)) “I like bones”
(Example 2)
(Question (I)) “Dad, when is Sumire ’s birthday?”
(Answer (I)) "December 21"
(Question (II)) "What will you buy?"
(Answer (II)) "Short cake"

  If the redundancy time is shortened as described above, for example, if a face image is captured first and a fingerprint or pressure-sensitive distribution is input after a considerable redundancy period has passed, The detectable time of the fingerprint and the pressure-sensitive distribution elapses when the phone is not held by hand. In other words, the information acquisition process is systematically performed in the absence of information sources such as fingerprints and palms, and only the blank fingerprints or pressure-sensitive distribution information that remains in the form remains. If this is subjected to authentication processing, rejection authentication will inevitably result in rejection, so the purpose can be achieved.

  FIG. 24 shows a second example of the second authentication process. Here, in S650 to S656, four types of facial image, fingerprint, pressure-sensitive distribution related to cellular phone holding, and air conduction sound are simultaneously acquired as authentication feature information. Specifically, voice input is performed in the center S653, and the detection of the face image, the fingerprint, and the pressure-sensitive distribution is performed once before and after the voice input to confirm the simultaneity. In S657 to S659, first, according to the flow shown in FIG. 26, it is confirmed whether or not each pattern of the face image, the fingerprint, and the pressure-sensitive distribution is moving before and after voice input, and is determined to be moving. Sets the authentication flag to non-permitted (S669). If not moving, the process proceeds to S660 to S667, and the voice, pressure-sensitive distribution, fingerprint and face image are respectively compared with the master, and if all match, the process proceeds to S668 and the authentication flag is set to permit (acceptance authentication), 1 If there is any mismatch, the process proceeds to S669, and the authentication flag is set to non-permitted (rejection authentication).

  By the way, there are fuzzy fluctuating factors in the authentication processing by voice and pattern matching, and it is well known that more complicated processing is required to improve the accuracy of authentication. When a plurality of types of authentication processes are performed in combination as in the present invention, the burden on the CPU that performs the processing becomes heavy, and a long processing waiting time may occur until the authentication is completed. Therefore, there is a concept that the accuracy of verification in each authentication process is somewhat lowered so that the processing load itself can be reduced, and the accuracy of authentication is covered by a combination of a plurality of methods. In this case, the process of FIG. 24 can be modified and executed as shown in FIG. The processing from S650 to S659 is exactly the same as that in FIG. In S660 to S673, the collation result with the master of the voice, pressure-sensitive distribution, fingerprint and face image is not determined alternatively or alternatively, but the collation coincidence is determined by numerical parameters such as points. The acceptance / rejection determination is comprehensively performed based on the result of the determination operation using the numerical parameter. In this case, even if the collation result with certain authentication feature information is somewhat unclear, if the collation result with other authentication feature information is clear, it is possible to make a useful authentication judgment with a small error overall. Become. In this embodiment, if the score is determined and the matching degree is low, points are deducted each time (S670, S671, S672, S674). If set (acceptance authentication) and less than the passing score, the process proceeds to S669 and the authentication flag is set to non-permitted (rejection authentication).

  Finally, FIG. 27 shows an example in which the first authentication process (S3) and the second authentication process (S4) of the authentication main process in FIG. 15 are replaced with a composite authentication process (S3) using only the face contact holding state. It is. As shown in FIG. 29, since the face image data cannot be acquired in the face contact holding state, the composite authentication process performs authentication feature information other than the face image, specifically, as shown in S651 to S655 in FIG. Specifically, sound (here, air conduction sound and bone conduction sound) is combined with hand biometric information (here, fingerprint and pressure sensitive narration), and these are acquired simultaneously. Again, voice input is performed in the center S652, and the fingerprint and the pressure-sensitive distribution are detected once before and after the voice input to confirm the simultaneity. In S658 and S659, the flow shown in FIG. 26 is used to check whether each pattern of the fingerprint and the pressure-sensitive distribution is moving before and after the voice input. If it is determined that the pattern is moving, the authentication flag is not set. The permission is set (S669). If not, the process proceeds to S662-S665 and S403-S422, and the voice, pressure-sensitive distribution, fingerprint, and face image are respectively compared with the master, and if all match, the process proceeds to S668 and the authentication flag is set to permit (acceptance). (Authentication) If there is even one mismatch, the process proceeds to S669 and the authentication flag is set to non-permitted (rejection authentication).

  In addition, as for the voice authentication processing after S403, the authentication processing based on the phase difference in FIG. 19 (second authentication processing: S401 to S406) and the authentication processing based on spectrum matching in FIG. 22 (first authentication processing: S420 to S422). ) And the authentication flag is set to “permitted” only when it is determined that both are the same (S426). If it is out of the range, it is set to “not permitted” (S427). In spectral matching, only air conduction sound is used, but bone conduction sound may be used, or both may be used. However, the calculation of the phase difference is simpler than the spectrum calculation, and spectrum verification is performed only for one of the air conduction sound and the bone conduction sound (for the other, the spectrum calculation itself is omitted), and authentication by the phase difference is assisted. If it is used, it is possible to simultaneously reduce the processing weight and improve the authentication accuracy.

  In the above embodiment, the data acquisition necessary for authentication and the authentication process using the data are all completed inside the mobile phone (the upper concept is an authentication terminal). Can also be carried by a device outside the mobile phone. For example, in a mobile phone, only acquisition of authentication feature information is performed, and the data is transferred directly or after appropriate processing, and then transferred to an authentication data processing apparatus configured by another computer by communication (in this case, for verification) Master data must be transferred to the authentication data processing device in advance). The authentication data processing device receives the transferred data, performs authentication processing by collation in the same manner as described above, and sends the result (data content in the same format as the authentication flag) to the mobile phone. return. The mobile phone performs the unlocking (use permission) or unlocking release (use disapproval) process already described in accordance with the contents of the received result.

  In FIG. 2, the authentication data processing apparatus is an authentication host computer 352 connected to a communication network 351 such as the Internet, and the mobile phone 1 is authenticated via a radio base station 350 by radio wave communication by a communication connection circuit 323. A host computer 352 is connected. The authentication host computer 352 may be connected via a short-range wireless communication network such as a wireless LAN or Blue Tooth, or may be connected via a connector or a cable.

1 is an external perspective view showing an example of a mobile phone used in the personal authentication system of the present invention. The block diagram which shows an example of the electrical constitution of the mobile telephone used for the personal authentication system of FIG. The schematic diagram which shows the example of detection of the pressure-sensitive distribution by a surface-type contact sensor. FIG. 3 is a schematic diagram showing storage contents of a ROM and an EEPROM in FIG. 2. The graph which shows the example of an audio | voice spectrum and a spectrum envelope. The conceptual diagram of the individual frequency spectrum of an air conduction sound and a bone conduction sound, and those difference spectra. The schematic waveform diagram which shows the concept used after filtering an audio | voice waveform. The schematic waveform diagram explaining the phase difference of an air conduction sound and a bone conduction sound. Explanatory drawing of the method of calculating | requiring the phase difference of an air conduction sound and a bone conduction sound by a waveform difference. The conceptual diagram of the authentication by a face image. The conceptual diagram of the authentication by a fingerprint. The conceptual diagram of the pressure-sensitive distribution by a fingerprint. The figure explaining a mode that a face image and the biometric information of a hand are input illegally sequentially. The schematic diagram which illustrates the induction | guidance | derivation format of voice authentication input. The flowchart which shows the flow of an authentication main process. The flowchart which shows the flow of an initialization process. The flowchart which shows the flow of an audio | voice data process. The flowchart which shows the flow of an air conduction sound / bone conduction sound wave form sampling process. The flowchart which shows the flow of an air conduction sound / bone conduction sound phase difference calculation process. The flowchart which shows the flow of the 1st example of a 1st authentication process. The flowchart which similarly shows the flow of a 2nd example. The flowchart which similarly shows the flow of a 3rd example. The flowchart which shows the flow of the 1st example of a 2nd authentication process. The flowchart which similarly shows the flow of a 2nd example. The flowchart which similarly shows the flow of a 3rd example. The flowchart which shows the flow of the pattern collation process which detects the motion of the biometric information of a face image or a hand. The flowchart which shows the example of the authentication main process using the composite authentication process in a face contact type | mold holding state. The flowchart which shows the flow of a composite authentication process. Explanatory drawing of a face contact type | mold holding state. Explanatory drawing of a face-to-face holding state.

Explanation of symbols

1 Mobile phone (personal authentication system)
304 Mic (microphone: air conduction sound detector)
340 Bone conduction microphone (bone conduction sound detector)
341 Face image camera 342 Fingerprint detection unit (contact-type biometric feature information detector)
343 Surface contact sensor (contact type biometric feature information detection unit)
312 CPU (authentication processing means, verification means, composite voice feature information calculation means)
313 RAM (bone conduction voice information storage unit, air conduction voice information storage unit)
322 EEPROM (standard audio feature information storage unit)

Claims (12)

A personal authentication system for authenticating a person to be authenticated using a mobile phone,
An authentication feature information acquisition unit provided in the mobile phone,
When the authentication processing person holds the mobile phone in a telephone use grasp holding state in which the mobile phone is grasped and held in the same state as when a telephone function other than the authentication process is used, the person who is subject to the authentication processing person touches the position. A contact-type biometric feature information detection unit that detects biometric feature information of the hand provided;
A camera for photographing a face provided at a position where the face of the person subject to authentication processing can be photographed when the person subject to authentication processing holds the mobile phone in the telephone use grasp holding state;
A bone conduction sound detector for detecting the voice information of the person to be authenticated by bone conduction sound;
An air conduction sound detecting unit for detecting voice information of the person to be authenticated by air conduction sound;
A feature information acquisition unit for authentication including two or more selected from the group consisting of:
Authentication feature information acquisition that is provided in the mobile phone and that simultaneously executes acquisition of authentication feature information by the two or more authentication feature information acquisition units in the telephone use grasp holding state for at least two specified ones Control means;
Authentication processing means that is provided inside or outside the mobile phone and performs authentication processing of the person to be authenticated based on the individual authentication feature information acquired by each of the two or more authentication feature information acquisition units; Prepared,
The acquisition of the authentication feature information is not performed when the at least two specified authentication feature information acquisition units are not simultaneously performed, and the authentication processing target person is not authenticated. Personal authentication system. The personal authentication system according to claim 1, wherein the contact-type biometric feature information detection unit includes a fingerprint detection unit. The personal authentication system according to claim 1, wherein the contact-type biometric feature information detection unit includes a contact detection sensor that detects a contact state between the mobile phone and the hand in the telephone use grasp holding state. The contact detection sensor is a surface contact sensor that detects contact distribution or grasping pressure distribution information on the surface of the mobile phone as the biological information of the hand, and the authentication processing unit is configured to detect the contact detected by the surface contact sensor. The personal authentication system according to claim 3, wherein the authentication is performed based on information on distribution or grasped pressure distribution. As the telephone use holding state, a face contact type holding state, which is a holding state in which a handset of the mobile phone is placed on the face, is used, and the contact-type biometric feature information detection unit is used as the authentication feature information acquisition unit And two or more selected from the group consisting of the bone conduction sound detection unit and the air conduction sound detection unit are used in combination, and the mobile phone is not provided with or provided with the face photographing camera 5. The personal authentication system according to claim 1, wherein the face photographing camera is not used in a mode in which the authentication feature information is acquired in the face contact type holding state. . In the face contact holding state, as the authentication feature information acquisition unit, the contact biometric feature information detection unit and at least one of the bone conduction sound detection unit and the air conduction sound detection unit are used in combination. 6. The authentication feature information acquisition control means simultaneously acquires voice information of at least one of the air conduction sound and bone conduction sound and the biological information of the hand as the authentication feature information. Personal authentication system. In the face contact holding state, both the bone conduction sound detection unit and the air conduction sound detection unit are used as the authentication feature information acquisition unit, and the authentication feature information acquisition control means The bone conduction sound information and the air conduction sound information are both acquired as the feature information for authentication by simultaneously detecting the sound emitted by the person to be processed by the bone conduction sound detection unit and the air conduction sound detection unit. The personal authentication system according to claim 5 or 6. Composite sound for calculating composite sound feature information that can be calculated only when both the bone conduction sound waveform detected by the bone conduction sound detection unit and the air conduction sound waveform detected by the air conduction sound detection unit are used. 8. The personal authentication system according to claim 7, further comprising characteristic information calculation means, wherein the authentication processing means performs the authentication processing based on the composite voice characteristic information. 9. The personal authentication system according to claim 8, wherein the composite voice feature information calculation means calculates a phase difference between the air conduction sound waveform and the bone conduction sound waveform as the composite sound feature information. The authentication processing means includes a first authentication process for comparing at least one of a frequency spectrum of the bone conduction sound and a frequency spectrum of the air conduction sound with a standard frequency spectrum, and the composite voice feature information. The personal authentication system according to claim 8 or 9, wherein the personal authentication system is implemented in combination with a second authentication process based on the above. The mobile phone is provided with a display unit that displays a face image of the person to be authenticated to be photographed by the face photographing camera. The face photographing camera is connected to the face of the person to be authenticated by the mobile phone. When the display unit is directly facing, it is attached to a position where the face can be photographed,
As the telephone use holding state, a face-to-face holding state in which the display unit of the mobile phone and the face photographing camera are in a holding state facing the face of the person to be authenticated is used, and the authentication feature information acquisition As the unit, the face photographing camera is indispensable, and at least one of the contact-type biometric feature information detection unit and the air conduction sound detection unit is used in combination, and the mobile phone detects the bone conduction sound detection 11. The bone conduction sound detection unit is not used in a mode in which the authentication feature information is acquired in the face-to-face holding state even though the unit is not provided or is provided. The personal authentication system according to any one of the above. The authentication feature information acquisition unit includes a combination of the contact-type biometric feature information detection unit and at least one of the face photographing camera, the air conduction sound detection unit, and the bone conduction sound detection unit. The feature information acquisition control unit is configured to detect the biometric information of the hand by the contact-type biometric feature information detection unit before and after the acquisition process of at least one of the face image, the air conduction sound information, and the bone conduction sound information. The personal authentication system according to any one of claims 1 to 11, wherein a contact change confirmation process for examining a change in the detection state is performed.

Images