CN115273859B - Security testing method and device for voice verification device - Google Patents

Abstract

The present disclosure relates to a security testing method and apparatus for a voice verification apparatus for verifying an identity of a registered person by voice, the method comprising: the method comprises the steps of obtaining a voice sample set comprising a plurality of first voice information of a target person, respectively fusing countermeasure audio information with the plurality of first voice information, respectively inputting the obtained plurality of fused voice information into a voice verification device to obtain first voice characteristics of the target person, optimizing the countermeasure audio information according to the first voice characteristics of the target person and second voice characteristics of registered persons stored in the voice verification device, effectively detecting safety reliability of the voice verification device by utilizing the optimized countermeasure audio information, reminding a user of paying attention to use risks of the voice verification device, and improving the voice verification device by a developer of the voice verification device.

Description

Security testing method and device for voice verification device

Technical Field

The present disclosure relates to the field of information processing technology, and in particular to a security testing method and device for a voice verification device.

Background technique

Voice verification is a relatively reliable identity authentication technology. This technology can extract voice features from input voice and calculate the similarity with the registered pre-stored voice features. If the similarity is greater than the preset threshold, the identity is judged to be consistent; otherwise, the identity is inconsistent. Voice verification technology is widely used in the fields of device authority control, financial activities, and criminal investigation and evidence collection. Compared with face and fingerprint verification, voice verification technology has the advantages of convenience, contactlessness, low cost, and difficulty in forgery. As voice verification devices that apply voice verification technology are increasingly deployed in people's daily lives, the security research of voice verification devices has received more and more attention and importance, and has high practical significance and application value.

Summary of the invention

In view of this, the present disclosure proposes a security testing method and device for a voice verification device.

According to one aspect of the present disclosure, a security testing method for a voice verification device is provided, the method comprising: obtaining a voice sample set, the voice sample set including multiple first voice information of a target person; fusing adversarial audio information with the multiple first voice information respectively to obtain multiple fused voice information; inputting the multiple fused voice information into the voice verification device respectively to obtain a first voice feature of the target person; optimizing the adversarial audio information according to the first voice feature of the target person and a second voice feature of a registered person stored in the voice verification device to obtain optimized adversarial audio information, the optimized adversarial audio information being used to perform a security test on the voice verification device, wherein the voice verification device is used to verify the identity of a registered person through voice.

In a possible implementation, the method also includes: controlling an audio playback device to play the optimized adversarial audio information; obtaining a verification result of the voice verification device for the verification voice information, the verification voice information including the optimized adversarial audio information and a real verification voice emitted by the target person during the period when the audio playback device plays the optimized adversarial audio information; and determining a security test result of the voice verification device based on the verification result.

In a possible implementation, the adversarial audio information is optimized according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device to obtain optimized adversarial audio information, including: optimizing the adversarial audio information based on a first loss function to obtain adversarial audio information in a first state; optimizing the adversarial audio information in the first state based on the first loss function and the second loss function to obtain optimized adversarial audio information; wherein the first loss function is used to indicate the recognition error between the first voice feature and the second voice feature, and the second loss function is used to indicate the influence of the adversarial audio information on the voice recognition content.

In a possible implementation, the adversarial audio information is fused with the multiple first voice information respectively to obtain multiple fused voice information, including: pre-processing the adversarial audio information according to the duration of the multiple first voice information to obtain first audio information corresponding to the duration of each first voice information; transforming the first voice information and the corresponding first audio information respectively to obtain second voice information and corresponding second audio information; wherein the transformation processing includes room impulse response; fusing the second voice information and the corresponding second audio information to obtain the fused voice information.

In one possible implementation, when the target person is an unregistered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a successful verification; when the target person is a registered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a failed verification.

In one possible implementation, the voice verification device includes a voice recognition module, a speaker verification module, and a replay detection module; wherein the voice recognition module is used to identify content information of the verification voice information, the speaker verification module is used to identify whether the verification voice information features belong to the voice information features of a registered person, and the replay detection module is used to detect whether the verification voice information is voice information that is played back after being recorded.

According to another aspect of the present disclosure, a security testing device for a voice verification device is provided, the device comprising: a voice sample set acquisition module, used to acquire a voice sample set, the voice sample set comprising a plurality of first voice information of a target person; a fusion module, used to fuse adversarial audio information with the plurality of first voice information respectively, to obtain a plurality of fused voice information; a feature acquisition module, used to input the plurality of fused voice information into the voice verification device respectively, to obtain a first voice feature of the target person; an optimization module, used to optimize the adversarial audio information according to the first voice feature of the target person and the second voice feature of a registered person stored in the voice verification device, to obtain optimized adversarial audio information, the optimized adversarial audio information being used to perform a security test on the voice verification device, wherein the voice verification device is used to verify the identity of a registered person through voice.

In a possible implementation, the device also includes: an audio playback module, used to control the audio playback device to play the optimized adversarial audio information; a verification result acquisition module, used to obtain the verification result of the voice verification device for the verification voice information, the verification voice information including the optimized adversarial audio information and the real verification voice emitted by the target person during the period when the audio playback device plays the optimized adversarial audio information; a security test result determination module, used to determine the security test result of the voice verification device based on the verification result.

In one possible implementation, the feature acquisition module is used to: optimize the adversarial audio information based on a first loss function to obtain adversarial audio information in a first state; optimize the adversarial audio information in the first state based on the first loss function and the second loss function to obtain optimized adversarial audio information; wherein the first loss function is used to indicate the recognition error between the first speech feature and the second speech feature, and the second loss function is used to indicate the impact of the adversarial audio information on the speech recognition content.

In one possible implementation, the fusion module is used to: pre-process the adversarial audio information according to the duration of the multiple first voice information to obtain first audio information corresponding to the duration of each first voice information; transform the first voice information and the corresponding first audio information respectively to obtain second voice information and corresponding second audio information; wherein the transformation processing includes room impulse response; and fuse the second voice information and the corresponding second audio information to obtain the fused voice information.

In one possible implementation, when the target person is an unregistered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a successful verification; when the target person is a registered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a failed verification.

According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

In the disclosed embodiment, the fused voice information of the adversarial audio information and the first voice information can be input into the voice verification device to obtain the first voice feature of the target person, and the adversarial audio information can be optimized based on the first voice feature of the target person and the second voice feature of the registered person to obtain the optimized adversarial audio information. The optimized adversarial audio information can be used to effectively detect the security and reliability of the voice verification device, reminding users to pay attention to the risks of using the voice verification device, which is conducive to the developers of the voice verification device to improve the voice verification device.

It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG1 is a schematic diagram of a voice verification device according to an embodiment of the present disclosure;

FIG2 is a flow chart showing a method for testing the security of a voice verification device according to an embodiment of the present disclosure;

FIG3 is a schematic diagram showing a method of performing security testing on a voice verification device using adversarial audio information in the related art;

FIG4 is a schematic diagram showing a security test of a voice verification device using optimized adversarial audio information according to an embodiment of the present disclosure;

FIG5 is a schematic diagram showing a method for testing the security of a voice verification device according to an embodiment of the present disclosure;

FIG6 is a schematic diagram showing a security test of optimized adversarial audio information according to an embodiment of the present disclosure;

FIG7 shows a block diagram of a security testing device for a voice verification device according to an embodiment of the present disclosure;

FIG8 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 9 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

FIG1 shows a schematic diagram of a voice verification device according to an embodiment of the present disclosure. As shown in FIG1 , the voice verification device includes a voice recognition module, a speaker verification module, and a playback detection module.

Among them, the voice recognition module is used to identify the content information of the verification voice information, the speaker verification module is used to identify whether the verification voice information features belong to the voice information features of a registered person, and the playback detection module is used to detect whether the verification voice information is voice information played back after recording.

In a possible implementation, the speaker verification module may be a white-box module, and the speech recognition module and the replay detection module may be black-box modules.

In a possible implementation, the voice verification device may be a Practical Speaker Verification (PSV) device, that is, an Automatic Speaker Verification (ASV) device based on a dynamic password.

In a possible implementation, the voice recognition module of the voice verification device is used to identify whether the content of the verification voice information input by the user is consistent with the dynamic password;

The speaker verification module of the voice verification device is used to determine whether the input verification voice information and the voice information registered by the user are from the same person. For example, the similarity between the voiceprint of the verification voice information and the voiceprint of the user's registration voice can be determined to determine whether the input verification voice information and the voice information registered by the user are from the same person.

The playback detection module of the voice verification device is used to detect whether the input language information is the audio that has been recorded and played back, to ensure that the user is a real person.

In a possible implementation, as shown in FIG1 , the voice verification device is used to verify the identity of a registered person through voice.

For example, when a user performs identity authentication, he or she will receive a dynamic password from the voice verification device, such as random characters or text. The user needs to speak the dynamic password within a specified time, which is transmitted through the air channel. The audio acquisition device (such as a microphone) of the voice verification device can record the user's voice information and send the voice information to the voice recognition module, speaker verification module and playback detection module of the voice verification device respectively.

When the voice password spoken by the user is correct, the voice recognition module of the voice verification device can give a detection pass signal; when the user is a registered person, that is, the voice spoken by the user and the registered voice come from the same person, the speaker verification module of the voice verification device can give a detection pass signal; when the voice spoken by the user is spoken by a real person (not a recorded voice played by an audio playback device), the playback detection module of the voice verification device can give a detection pass signal.

Only when the above three modules of the voice verification device give a detection pass signal, the voice verification device can give a reception signal, can receive the user's voice information, and the user's identity verification is successful;

Otherwise, if the voice password spoken by the user is incorrect, or the user is not a registered person of the voice verification device, or the voice verification device receives a recorded voice, the voice verification device may give a rejection signal, refuse to receive the user's voice information, and the user's identity verification fails.

As the application of deep learning models brings performance improvements to voice verification devices, voice verification devices are increasingly used in areas such as device authority control, financial activities, and criminal investigation and evidence collection. Research on security testing methods for voice verification devices is becoming increasingly important.

In the related art, some carefully designed tiny disturbances, namely adversarial audio information, can be added to the input voice of the voice verification device to increase the similarity between the input voice features of the unregistered target person and the voice features of the registered person. Thus, the adversarial audio information can be used to implement the security test of the voice verification device in a target scenario. A voice verification device with high security performance will not identify the target person as the registered person, and a voice verification device with low security performance will identify the target person as the registered person.

Alternatively, the similarity between the input voice features of the registered target person and the registered voice features of the person himself is reduced, so that adversarial audio information can be used to implement security testing of the voice verification device in a non-target scenario. A voice verification device with high security performance can identify the identity of the registered person, while a voice verification device with low security performance cannot identify the identity of the registered person.

However, in the related art, there are the following problems regarding the security testing method of the voice verification device:

(1) The adversarial audio information is generated for a specific sentence, while the voice verification device can use a dynamic password. Every time the dynamic password is changed, a new adversarial audio information needs to be optimized. The adversarial audio information generated for a specific sentence does not meet the real-time requirements and is difficult to be applied to the dynamic password in real time.

(2) The voice verification device includes a voice recognition module to ensure the consistency of the voice content and the dynamic password. The adversarial audio information generated by optimizing the speaker verification module alone may affect the recognition of the voice content.

(3) During the security test of the voice verification device, the adversarial audio information and the target person's voice are played as a signal. In this case, the target person's voice will go through two playback recording processes and will be detected as a playback signal by the playback detection module of the voice verification device.

In view of the above problems, the present disclosure proposes a security testing method for a voice verification device. The voice sample set obtained by the method may include multiple first voice information of the target person, and the content of each first voice information is different. By learning the adversarial audio information on the voice sample set, the optimized adversarial audio information can be made independent of the content. Even when the dynamic password changes, the adversarial audio information can be directly applied to the new voice; and the method can optimize the adversarial audio information in two stages according to the first voice feature of the target person and the second voice feature of the registered person, so that the optimized adversarial audio information can reduce the impact on voice content recognition while meeting the adversarial nature; finally, the adversarial audio information optimized by the method can be played as a separate sound source by an audio playback device, and can pass the playback detection module of the voice verification device in conjunction with the real voice of the target person. The disclosed method can use the adversarial nature of the optimized adversarial audio information to effectively detect the security and reliability of the voice verification device, remind users to pay attention to the risks of using the voice verification device, and help the developers of the voice verification device to improve the voice verification device.

FIG2 is a flow chart of a method for testing the security of a voice verification device according to an embodiment of the present disclosure. As shown in FIG2 , the method includes:

In step S1, a voice sample set is obtained, wherein the voice sample set includes a plurality of first voice information of a target person;

In step S2, the adversarial audio information is fused with the plurality of first voice information respectively to obtain a plurality of fused voice information;

In step S3, the plurality of fused voice information are respectively input into the voice verification device to obtain the first voice feature of the target person;

In step S4, the adversarial audio information is optimized according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device to obtain optimized adversarial audio information, and the optimized adversarial audio information is used to perform a security test on the voice verification device, wherein the voice verification device is used to verify the identity of the registered person through voice.

In a possible implementation, in step S1, the voice of the target person may be collected by an audio collection device (such as a microphone) to obtain a voice sample set, which includes a plurality of first voice information of the target person.

In the voice sample set, each first voice information contains different change factors such as content, duration, emotion, and tone. The more first voice information included in the voice sample set, the more change factors of the target person's voice covered by the voice sample set, which is more conducive to the learning of the adversarial audio information on the voice sample set, making it easier for the adversarial audio information to adapt to changes in these factors.

Therefore, by obtaining a speech sample set, it is helpful to learn adversarial audio information on the speech sample set, which can reduce the impact of adversarial audio information on speech content, and make the optimized adversarial audio information obtained in subsequent steps independent of the content. Even when the dynamic password changes, the adversarial audio information can be directly applied to the new speech, which can improve the real-time and versatility of the security test of this method.

In a possible implementation, in step S2, the adversarial audio information may be set as an audio of a fixed length, and the adversarial audio information may be fused with each first voice information in the voice sample set to obtain a plurality of fused voice information;

It should be understood that the present disclosure does not limit the manner of fusing the adversarial audio information and the first voice information. For example, the adversarial audio information and the first voice information may be fused by directly adding the sound amplitudes at corresponding moments.

In a possible implementation, in step S3, multiple fused voice information are respectively input into the voice verification device. For example, each fused voice information can be input into the voice verification device separately, or multiple fused voice information can be input into the voice verification device in batch data. The present disclosure does not limit the input method of inputting multiple fused voice information into the voice verification device separately. The voice verification device extracts the features of the fused voice information to obtain the first voice feature of the target person.

Among them, the voice verification device can extract features of the input voice, and the process is as follows: first, the voice verification device can extract the frequency domain features of the input audio, then use a deep neural network to extract frame-level features, and then fuse the frame-level features into sentence-level features. Finally, the sentence-level features are mapped into a vector of fixed dimension through a fully connected neural network. This vector is called the voice feature of the input audio.

In one possible implementation, in step S4, based on the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device, the similarity between the first voice feature of the target person and the second voice feature of the registered person can be calculated, and then the loss function can be calculated based on the similarity between the two. The back propagation algorithm is then used to find the gradient of the loss function with respect to the adversarial audio information, and the adversarial audio information is updated based on the obtained gradient to obtain the optimized adversarial audio information.

Among them, the adversarial audio information can be updated according to the obtained gradient, which can be combined with the Fast Gradient Sign Method (FGSM) or the Project Gradient Descent (PGD). The present disclosure does not limit the specific combined algorithm.

The optimized adversarial audio information can be used to perform security tests on the voice verification device. For example, if the voice verification device passes the security test using the optimized adversarial audio information, it means that the security of the voice device is relatively high; if the voice verification device fails the security test using the optimized adversarial audio information, it means that the security of the voice device is relatively low.

In this way, the fused voice information of the adversarial audio information and the first voice information can be input into the voice verification device to obtain the first voice feature of the target person, and the adversarial audio information can be optimized based on the first voice feature of the target person and the second voice feature of the registered person to obtain the optimized adversarial audio information. The optimized adversarial audio information can be used to effectively detect the security and reliability of the voice verification device, reminding users to pay attention to the risks of using the voice verification device, which is beneficial for the developers of the voice verification device to improve the voice verification device and is of great significance to further improving the security of the voice verification device.

Through the above steps S1 to S4, optimized adversarial audio information can be obtained for performing security tests on the voice verification device. In order to better illustrate how the embodiments of the present disclosure use optimized adversarial audio information to perform security tests on the voice verification device, the following first introduces the process of using adversarial audio information in the related art to perform security tests on the voice verification device.

Figure 3 shows a schematic diagram of using adversarial audio information to perform security testing on a voice verification device in the related art. As shown in Figure 3, in the related art, the voice information of a person (personnel A) who performs security testing on the voice verification device and the adversarial audio information can be added to obtain a verification voice information, and the verification voice information can be played through an audio playback device (such as a notebook).

The audio acquisition device (e.g., microphone) of the voice verification device can record the played verification voice information, and each verification module of the voice verification device (e.g., including voice recognition module, speaker verification module, playback detection module) can verify the collected verification voice information to obtain a verification result. By obtaining the verification result of the voice verification device for the verification voice information, the security of the voice verification device can be judged and the security test result of the voice verification device can be obtained.

In the process shown in FIG3 , in the process of obtaining the voice information of person A, it undergoes one transmission record, and in the process of transmitting the verification voice information to the voice verification device, it undergoes another transmission record. Therefore, person A's voice will undergo two transmission records, and it is easy to be detected by the playback detection module of the voice verification device, resulting in that the verification voice information cannot effectively judge the security of the voice verification device, and the determined security test result of the voice verification device is not accurate enough.

Among them, propagation recording refers to the process in which speech is transmitted through the air channel and recorded by the audio collection device into an audio file. If speech is recorded through propagation, the air channel propagation and the speech collection device may cause distortion to the speech. There is loss in the recording of speech after one propagation, and the loss of speech after multiple propagation recordings is cumulative.

Figure 4 shows a schematic diagram of security testing of a voice verification device using optimized adversarial audio information according to an embodiment of the present disclosure. As shown in Figure 4, the method also includes: controlling an audio playback device to play the optimized adversarial audio information; obtaining a verification result of the voice verification device for the verification voice information, the verification voice information including the optimized adversarial audio information and a real verification voice emitted by the target person during the period when the audio playback device plays the optimized adversarial audio information; and determining a security test result of the voice verification device based on the verification result.

For example, the real voice emitted by the target person (person A) and the optimized adversarial audio information played by the audio playback device are transmitted through the air channel and can be recorded together by the audio acquisition device (such as a microphone) of the voice verification device.

Among them, this method can control the audio playback device to play the optimized confrontation audio information, so that the audio playback device can always play the optimized confrontation audio information while the target person speaks the real verification voice. It should be understood that the working state of the audio playback device can be controlled by a preset program or button, and the present disclosure does not limit the audio playback device and the control method of the audio playback device.

Each verification module of the voice verification device (for example, including a voice recognition module, a speaker verification module, and a playback detection module) can verify the collected verification voice information (i.e., the real verification voice emitted by the target person and the optimized adversarial audio information played) to obtain a verification result. By obtaining the verification result of the voice verification device for the verification voice information, the security of the voice verification device can be judged and the security test result of the voice verification device can be obtained.

Compared with the process shown in FIG3 , in the process shown in FIG4 , the adversarial audio information can be played as a separate sound source by the audio playback device, and the real verification voice of the target person (personnel A) has only undergone one transmission record.

Therefore, by using the method shown in FIG. 4 , the loss caused by multiple transmission records of the voice can be reduced, and the verification voice can pass through the playback detection module of the voice verification device, thereby improving the accuracy of the method for testing the security of the voice verification device.

In the actual application process of the voice verification device, it is possible that an unregistered person A is identified as a registered person B (i.e., a target scenario), or a registered person is not identified (i.e., a non-target scenario). In view of the two security risks in the actual application process of the voice verification device, the security testing method of the voice verification device of the embodiment of the present disclosure is described in detail below for the target scenario and the non-target scenario respectively.

For the target scenario, Figure 5 shows a schematic diagram of the security testing method of the voice verification device according to the embodiment of the present disclosure. As shown in Figure 5, person A is a target person who has not registered the voice verification device, and person B is a person who has registered the voice verification device, wherein the voice or voice features of the registered person B have been pre-stored in the voice verification device.

In the process of testing the security of the voice verification device, since the password content required by the voice verification device is unknown and the password is different each time, it is necessary to generate a universal adversarial audio information that is independent of the voice content.

In a possible implementation, in step S1, the voice of the target person A is easy to obtain, and the voice of the target person A can be collected through any audio collection device (such as a microphone) to obtain the voice information of the target person A.

As shown in FIG. 5 , N voices of target person A may be collected to construct a voice sample set, and the voice sample set may include voice 1 to voice N of target person A, that is, first voice information 1 to first voice information N.

Among them, each first voice information includes different change factors such as voice content, duration, emotion, and pitch. The larger the value of N, the more comprehensive the voice sample set covers the change factors in the voice of the target person A, and the more conducive it is to the learning of the adversarial audio information on the voice sample set.

Therefore, in this way, the optimized adversarial audio information obtained in the subsequent steps can adapt to the voice change factors of the target person A, so that in the process of security testing of the voice verification device, the optimized adversarial audio information can be directly applied to the new voice of the target person A.

After a language sample set including a plurality of first voice information of a target person is obtained in step S1, the adversarial audio information may be fused with each first voice information respectively to obtain a plurality of fused voice information.

In a possible implementation, step S2 includes:

Step S21, pre-processing the countermeasure audio information according to the duration of the plurality of first voice information to obtain first audio information corresponding to the duration of each first voice information;

Step S22, transforming the first voice information and the corresponding first audio information respectively to obtain second voice information and the corresponding second audio information; wherein the transformation process includes a room impulse response;

Step S23, fusing the second voice information and the corresponding second audio information to obtain the fused voice information.

For example, in step S21, assuming that the adversarial audio information is δ, the acquired voice sample set is X={ _x1 , _x2 ,…, _xN }, _xi represents the first voice information of the target person, the value range of i is 1~N, N is a positive integer, wherein the duration of the N first voice information _x1 ~ _xN may be different, and the present disclosure does not limit the duration of each first voice information.

According to the duration of each first voice information x _i , the adversarial audio information δ is preprocessed to obtain the first audio information δ′ corresponding to the duration x _i of each first voice information.

The duration of the antagonistic audio information δ can be set to be shorter than the duration of each first voice information _xi , and the antagonistic audio information δ is preprocessed according to the duration of each first voice information _xi , for example, including repeated expansion processing, to obtain multiple first audio information δ′ after repeated expansion, which are respectively the same as the duration of each first voice information _xi . It should be understood that the present disclosure does not limit the duration of the antagonistic audio information δ and the specific method of preprocessing.

In step S22, each first voice information x _i and the corresponding first audio information δ′ in the voice sample set X are transformed by T to obtain second voice information T(x _i ) and corresponding second audio information T(δ′);

The transformation T may include a room impulse response (RIR), also known as a room transfer function. The transformation T may be measured by a measurement method, for example, a sine sweep signal method may be used to measure the room impulse response of an actual room.

The frequency sweep signal _{x e (} t) can be played and can be expressed as:

In formula (1), f ₁ , f ₂ represent the frequency range of the room impulse response to be measured, T ₁ represents the time length of the swept frequency signal x _e (t), and the value range of the independent variable t of the swept frequency signal x _e (t) is 0 to T ₁ .

Assume that _ye (t) is the signal recorded by the receiving end, and the room impulse response r(t)= _ye (t)* _xe (-t), where * represents the convolution operation and _xe (-t) represents the inverse order of the time dimension of the swept frequency signal _xe (t).

Therefore, corresponding to any audio x, the specific form of the transformation T can be expressed as T(x)=x*r, where x represents any audio to be transformed by T, and r is the room impulse response r(t).

In step S23, the second speech information T( _xi ) and the corresponding second audio information T(δ') are fused to obtain N fused speech information T( _xi )+T(δ').

It should be understood that the present disclosure does not limit the specific method of fusing the second voice information T( _xi ) and the second audio information T(δ′). For example, the fusion can be performed by adding the amplitudes of the second voice information T( _xi ) and the second audio information T(δ′) at corresponding moments.

In this way, the process of the target person speaking to the audio acquisition device of the voice verification device is recorded, and the distortion caused by the voice is modeled by transformation T. By performing transformation T on the first voice information and the corresponding first audio information respectively, the transformation T can be integrated into the optimization process of the adversarial audio information. If the adversarial audio information performs enough transformations T during the optimization process, the adversarial audio information can improve the generalization ability of the transformation T, thereby adapting to changes in the test environment and improving the effectiveness of the security test of the voice verification device.

After obtaining N fused voice information in step S2, in step S3, the N fused voice information can be respectively input into the voice verification device to obtain the first voice feature of the target person.

In one possible implementation, assuming that the fused speech information can be expressed as T( _xi )+T(δ′), the fused speech information T( _xi )+T(δ′) can be input into a speech verification device, and the speaker verification module F of the speech verification device can be used to extract the first speech feature of the target person, i.e., F(T( _xi )+T(δ′)).

Each fused voice information T( _xi )+T(δ') can be input into the speaker verification module F of the voice verification device separately. For example, the fused voice information T( _x1 )+T(δ') is input into the speaker verification module F of the voice verification device separately for the first time, and the fused voice information T( _x2 )+T(δ') is input into the speaker verification module F of the voice verification device separately for the second time, and so on, until the fused voice information T( _xN )+T(δ') is input into the speaker verification module F of the voice verification device separately for the Nth time. In this case, in the subsequent optimization process of the adversarial audio information, each fused voice information will generate a gradient with respect to the adversarial audio information, so that N forward propagations and backward propagations are required to be performed when the adversarial audio information is updated using the gradient.

Alternatively, multiple fused voice information may be input into the voice verification device in batch data. Since the lengths of the N fused voice information are different, the length of each fused voice information in the N fused voice information may be extended to the length of the longest fused voice information. In this case, in the subsequent optimization process of the adversarial audio information, one forward propagation and one backward propagation may be performed.

It should be understood that the present disclosure does not limit the input method of inputting multiple fused voice information into the voice verification device separately.

After obtaining the first voice feature of the target person in step S3, the adversarial audio information may be optimized in step S4 according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device.

In one possible implementation, step S4 includes: based on a first loss function, optimizing the adversarial audio information to obtain adversarial audio information in a first state; based on the first loss function and a second loss function, optimizing the adversarial audio information in the first state to obtain optimized adversarial audio information; wherein the first loss function is used to indicate the recognition error between the first speech feature and the second speech feature, and the second loss function is used to indicate the influence of the adversarial audio information on the speech recognition content.

For example, the adversarial audio information may be optimized through a two-stage optimization method.

In the first stage, the adversarial audio information δ can be optimized based on the first loss function L ₁ to obtain the adversarial audio information δ in the first state. The adversarial audio information δ in the first state obtained in this stage can satisfy the adversarial property.

The first loss function is used to indicate the recognition error between the first speech feature and the second speech feature.

The first loss function L ₁ can be expressed as:

In formula (2), X represents a speech sample set, and the speech sample set X includes N first speech information, _xi represents the i-th first speech information in the speech sample set X, δ′ represents the preprocessed adversarial audio information δ, that is, the first audio information δ′ with the same duration as the first speech information _xi , y represents the audio information of the registered person, F represents the speaker verification module of the speech verification device, F(T( _xi )+T(δ′)) represents the first speech feature of the target person, F(y) represents the second speech feature of the registered person, s(F(T( _xi )+T(δ′)), F(y)) represents the similarity between the first speech feature and the second speech feature, θ represents a preset threshold, and k represents the confidence level. Among them, the method for obtaining the similarity s(·) in the present disclosure can be a cosine similarity method, a Pearson correlation coefficient method, or a Jaccard similarity coefficient method, and the present disclosure does not limit the specific method for obtaining the similarity.

If the similarity s(F(T( _xi )+T(δ′)),F(y)) is greater than a preset threshold value θ, it can be determined that the first voice feature and the second voice feature are from the same person;

If the similarity s(F(T( _xi )+T(δ′)),F(y)) is less than or equal to a preset threshold θ, it can be determined that the first voice feature and the second voice feature come from different persons.

It should be understood that the preset threshold value θ and confidence level κ may be set based on experience, and the present disclosure does not limit the specific values of the preset threshold value θ and confidence level κ.

Therefore, in the first stage, by optimizing the first loss function L ₁ , the similarity between the first speech feature of the target person and the second speech feature of the registered person can be made greater than the threshold, so that the speech verification device determines that the two speech segments are from the same speaker, and the adversarial audio information of the first state can meet the adversarial property.

In the second stage, the adversarial audio information δ in the first state may be optimized based on the first loss function L ₁ and the second loss function L ₂ to obtain the optimized adversarial audio information δ.

For example, the adversarial audio information δ can be optimized based on the weighted sum of the first loss function _L1 and the second loss function _L2 : L= _L1 + _λL2 to obtain the optimized adversarial audio information δ. Wherein, λ is a balance parameter, and the specific value of the balance parameter λ can be set according to an empirical value, and the present disclosure does not limit the specific value of the balance parameter λ.

The optimized adversarial audio information δ obtained in this stage can reduce the impact of the adversarial audio information δ on the speech recognition content while ensuring the adversarial nature.

Among them, the second loss function _L2 is used to indicate the impact of adversarial audio information on speech recognition content.

Since the speech recognition modules of the speech verification device are diverse, they cannot be optimized through a single speech recognition module. In view that there is a pre-processing process of frequency domain feature extraction in the speech recognition module, the impact of frequency domain feature extraction in the speech recognition process can be reduced by counteracting audio information, thereby reducing the impact on the recognition results.

Therefore, the second loss function L ₂ can be expressed as:

L ₂ (δ) = mean(|STFT(δ)|) (3)

In formula (3), δ represents the adversarial audio information, STFT(·) represents the function for extracting the speech frequency domain features, that is, performing short-time Fourier transform (STFT) on the adversarial audio information δ, mean(·) represents the function for obtaining the mean, and the distance of the results of the function STFT(·) for extracting the speech frequency domain features can be averaged.

Therefore, in the second stage, by optimizing the weighted sum L=L ₁ +λL ₂ of the first loss function L ₁ and the second loss function L ₂ , the optimized adversarial audio information can reduce the impact on the original speech recognition content while satisfying the adversarial nature.

In this way, adversarial audio information of the first state can be obtained in the first stage, and the adversarial audio information of this state can meet the adversarial properties. Then, under the premise of ensuring the adversarial properties in the second stage, the impact of the adversarial audio information on the original voice recognition content is reduced, so that the optimized adversarial audio information can effectively detect the security and reliability of the voice verification device.

After the optimized adversarial audio information is obtained through steps S1 to S4, FIG6 shows a schematic diagram of performing a security test on the optimized adversarial audio information according to an embodiment of the present disclosure.

As shown in Figure 6, the audio playback device is used to play the optimized adversarial audio information. During the period when the audio playback device plays the optimized adversarial audio information, the target person reads out the dynamic password, and the microphone of the voice verification device can receive and record the verification voice information, that is, the mixed audio of the optimized adversarial audio information and the target person reading out the dynamic password.

The voice verification device can verify the received verification language to obtain a verification result, and the security test result of the voice verification device can be determined according to the verification result.

In a possible implementation, when the target person is an unregistered person, the optimized adversarial audio information is used to make the verification result output by the voice verification device be a successful verification.

For example, in a target scenario, that is, the voice verification device may mistakenly identify an unregistered target person A as a registered person B. The optimized adversarial audio information can be used to perform a security test on the voice verification device, that is, the adversarial nature of the optimized adversarial audio information can be used.

Among them, the adversarial nature of the optimized adversarial audio information is specifically: inputting the optimized adversarial audio information and the voice information of the target person (unregistered) into the voice verification device, the verification result output by the voice verification device can be successful verification.

In a target scenario, the optimized adversarial audio information and the voice information of the target person (unregistered) are input into the voice verification device. If the verification result output by the voice verification device is a successful verification, it means that the security of the voice verification device is relatively low; if the verification result output by the voice verification device is a failed verification, it means that the security of the voice verification device is relatively high.

Therefore, by utilizing the adversarial properties of the optimized adversarial audio information, the security of the voice authentication device can be effectively tested.

The above describes the security testing method of the voice verification device of the embodiment of the present disclosure in a target scenario. Compared with the target scenario, the voice verification device may have a serious security risk of identifying an unregistered person A as a registered person B. In the non-target scenario, the security risk of the voice verification device failing to identify a registered person is relatively small.

For non-target scenarios, the security testing method for the voice verification device in the above-mentioned target scenario, as shown in FIG5 , can be applied to the security testing method for the voice verification device in the non-target scenario after some setting modifications.

As shown in Figure 5, person A (target person) and person B (registered person) are set to be the same person, that is, the target person is the person who has registered the voice verification device. Since they are the same person, if there is no adversarial audio information, the voice verification device can accept the verification voice of the target person. In the non-target scenario, the addition of adversarial audio information may cause the verification of the target person to fail, and the first loss function can be changed to:

In formula (4), X represents a speech sample set, which includes N first speech information, x _i represents the i-th first speech information in the speech sample set X, δ′ represents the preprocessed adversarial audio information δ, that is, the first audio information δ′ with the same duration as the first speech information x _i , y represents the audio information of the registered person (that is, the registered audio information of the target person A), F represents the speaker verification module of the speech verification device, F(T(x _i )+T(δ′)) represents the first speech feature of the target person, F(y) represents the second speech feature of the registered person (that is, the target person A), s(F(T(x _i )+T(δ′)), F(y)) represents the similarity between the first speech feature and the second speech feature, θ represents a preset threshold, and κ represents the confidence level.

Optimizing L′ ₁ can make the similarity between the target person's first voice feature and the second voice feature that the person has registered lower than a threshold, so that the voice verification device determines that the two voices are not from the same speaker.

It should be understood that the modified method only modifies the above settings, and its method steps are the same as the method steps in the target scenario, which will not be repeated here.

Similarly, optimized adversarial audio information suitable for target-free scenarios is obtained, as shown in FIG6 , and security testing in target-free scenarios can be performed based on the optimized adversarial audio information.

While the audio playback device is playing the optimized adversarial audio information, the target person reads out the dynamic password, and the microphone of the voice verification device can receive and record the verification voice information, that is, the mixed audio of the optimized adversarial audio information and the target person reading out the dynamic password.

The voice verification device can verify the received verification language to obtain a verification result, and the security test result of the voice verification device can be determined according to the verification result.

In a possible implementation, when the target person is a registered person, the optimized adversarial audio information is used to make the verification result output by the voice verification device be a verification failure.

In an untargeted scenario, that is, the voice verification device may not be able to correctly identify the identity of a registered person, the optimized adversarial audio information can be used to perform a security test on the voice verification device, that is, the adversarial nature of the optimized adversarial audio information can be used.

Among them, the adversarial nature of the optimized adversarial audio information is specifically: inputting the optimized adversarial audio information and the voice information of the target person (registered) into the voice verification device can make the verification result output by the voice verification device be verification failure.

In a non-target scenario, the optimized adversarial audio information and the voice information of the target person (registered) are input into the voice verification device. If the verification result output by the voice verification device is a successful verification, it means that the security of the voice verification device is relatively high; if the verification result output by the voice verification device is a failed verification, it means that the security of the voice verification device is relatively low.

Therefore, by utilizing the adversarial properties of the optimized adversarial audio information, the security of the voice authentication device can be effectively tested.

Through the detailed introduction of the security testing method of the voice verification device of the embodiment of the present disclosure in the above target scenario and the non-target scenario, it can be known that the stronger the adversarial nature of the adversarial audio information, the more effective the security of the voice verification device can be tested. In order to better illustrate the test effect of the security testing method of the voice verification device of the present disclosure, three groups of comparative experiments can be used to illustrate the adversarial nature of the adversarial audio information of the present disclosure.

Volunteers can be selected as target personnel, and some dynamic password contents that have never appeared in the voice sample set can be set for the target personnel to read. When the first group of target personnel read out the dynamic password, the audio playback device does not play any content, in order to provide a baseline for the accuracy of content recognition by the voice verification device; when the second group of target personnel speak out the dynamic password, the audio playback device plays Gaussian noise, and the noise amplitude value is similar to the amplitude value of the adversarial audio information, in order to eliminate the influence of random noise on the experiment; when the third group of target personnel speak out the dynamic password, the audio playback device plays the optimized adversarial audio information, in order to evaluate the effectiveness of the adversarial audio information.

In the above three comparative experiments, the second group of experiments playing Gaussian noise did not improve the success rate of the confrontation compared with the first group of experiments, both of which were 0%, and the error rate of speech content recognition increased from 11.42% to 17.77%, an increase of 6.35%; the third group of experiments playing optimized adversarial audio information compared with the first group of experiments, the success rate of the confrontation was 100%, while the error rate of speech content recognition increased from 11.42% to 14.97%, an increase of only 3.55%. This shows that the optimized adversarial audio information has little effect on speech content recognition while ensuring the success rate of the confrontation.

In addition, the present disclosure also conducted two groups of comparative experiments on playback detection. In the first group, the method in the related art as shown in Figure 3 was used, and the verification voice information (i.e., the recorded target person's voice + adversarial audio information) was played by the audio playback device. The second group used the method of the present disclosure as shown in Figure 4. During the period when the target person issued the real verification voice, the audio playback device played the adversarial audio information alone. In the above two groups of comparative experiments on playback detection, the pass rate of the first group of playback detection was 37.7%, and the pass rate of the second group of playback detection was 67.7%. The pass rate of the playback detection of the present disclosure method was 30% higher than that of the related art method.

Therefore, in the disclosed embodiment, the acquired voice sample set may include multiple first voice information of the target person, and the content of each first voice information is different. By learning the adversarial audio information on the voice sample set, the optimized adversarial audio information can be made independent of the content, and even when the dynamic password changes, the adversarial audio information can be directly applied to the new voice; and the method can optimize the adversarial audio information in two stages according to the first voice feature of the target person and the second voice feature of the registered person, so that the optimized adversarial audio information can reduce the impact on the recognition of voice content while meeting the adversarial nature; finally, the adversarial audio information optimized by the method can be played by the audio playback device as a separate sound source, and can pass the playback detection module of the voice verification device in conjunction with the real voice of the target person. The disclosed method can utilize the adversarial nature of the adversarial audio information optimized by the method to effectively detect the security and reliability of the voice verification device, remind users to pay attention to the risks of using the voice verification device, and help the developers of the voice verification device to improve the voice verification device.

It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, the present disclosure also provides a security testing device, an electronic device, a computer-readable storage medium, and a program for a voice verification device, all of which can be used to implement the security testing method for any voice verification device provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

FIG. 7 shows a block diagram of a security testing device of a voice verification device according to an embodiment of the present disclosure. As shown in FIG. 7 , the device includes:

A voice sample set acquisition module 71, used to acquire a voice sample set, wherein the voice sample set includes a plurality of first voice information of a target person;

A fusion module 72, configured to fuse the adversarial audio information with the plurality of first voice information respectively to obtain a plurality of fused voice information;

A feature acquisition module 73, used to input the plurality of fused voice information into the voice verification device respectively to obtain the first voice feature of the target person;

The optimization module 74 is used to optimize the adversarial audio information according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device to obtain optimized adversarial audio information, and the optimized adversarial audio information is used to perform a security test on the voice verification device, wherein the voice verification device is used to verify the identity of the registered person through voice.

In a possible implementation, the device also includes: an audio playback module, used to control the audio playback device to play the optimized adversarial audio information; a verification result acquisition module, used to obtain the verification result of the voice verification device for the verification voice information, the verification voice information including the optimized adversarial audio information and the real verification voice emitted by the target person during the period when the audio playback device plays the optimized adversarial audio information; a security test result determination module, used to determine the security test result of the voice verification device based on the verification result.

In one possible implementation, the fusion module 72 is used to: pre-process the adversarial audio information according to the duration of the multiple first voice information to obtain first audio information corresponding to the duration of each first voice information; transform the first voice information and the corresponding first audio information respectively to obtain second voice information and corresponding second audio information; wherein the transformation processing includes room impulse response; fuse the second voice information and the corresponding second audio information to obtain the fused voice information.

In one possible implementation, the feature acquisition module 73 is used to: optimize the adversarial audio information based on a first loss function to obtain adversarial audio information in a first state; optimize the adversarial audio information in the first state based on the first loss function and the second loss function to obtain optimized adversarial audio information; wherein the first loss function is used to indicate the recognition error between the first speech feature and the second speech feature, and the second loss function is used to indicate the influence of the adversarial audio information on the speech recognition content.

In one possible implementation, when the target person is an unregistered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a successful verification; when the target person is a registered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a failed verification.

In some embodiments, the security testing device of the voice verification device provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the method described in the above method embodiments. Its specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

The embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

The electronic device may be provided as a terminal, a server, or a device in other forms.

8 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input/output (I/O) interface 812 , a sensor component 814 , and a communication component 816 .

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

I/O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open/closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G), a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G) or a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

FIG9 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Referring to FIG9 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft's server operating system (Windows ServerTM), Apple's graphical user interface-based operating system (Mac OSXTM), a multi-user multi-process computer operating system (UnixTM), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

The present disclosure may be a system, a method and/or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.

The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

Various aspects of the present disclosure are described herein with reference to the flowcharts and/or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and/or block diagram and the combination of each box in the flowchart and/or block diagram can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. A method for testing the security of a voice verification device, characterized in that the method comprises: Acquire a voice sample set, wherein the voice sample set includes a plurality of first voice information of a target person; Fusing the adversarial audio information with the plurality of first voice information respectively to obtain a plurality of fused voice information; Inputting the plurality of fused voice information into the voice verification device respectively to obtain the first voice feature of the target person; According to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device, the adversarial audio information is optimized to obtain optimized adversarial audio information, and the optimized adversarial audio information is used to perform a security test on the voice verification device. Wherein, the voice verification device is used to verify the identity of the registered person through voice. 2. The method according to claim 1, characterized in that the method further comprises: Controlling the audio playback device to play the optimized countermeasure audio information; Acquire a verification result of the voice verification device for the verification voice information, wherein the verification voice information includes the optimized adversarial audio information and a real verification voice emitted by the target person during the period when the audio playback device plays the optimized adversarial audio information; A security test result of the voice verification device is determined according to the verification result. 3. The method according to claim 1 is characterized in that, according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device, the adversarial audio information is optimized to obtain the optimized adversarial audio information, comprising: Based on a first loss function, optimizing the adversarial audio information to obtain adversarial audio information in a first state; Based on the first loss function and the second loss function, optimizing the adversarial audio information of the first state to obtain optimized adversarial audio information; Among them, the first loss function is used to indicate the recognition error between the first speech feature and the second speech feature, and the second loss function is used to indicate the influence of the adversarial audio information on the speech recognition content. 4. The method according to claim 1, characterized in that fusing the adversarial audio information with the plurality of first voice information respectively to obtain a plurality of fused voice information comprises: Preprocessing the adversarial audio information according to the durations of the plurality of first voice information to obtain first audio information corresponding to the durations of the respective first voice information; The first voice information and the corresponding first audio information are transformed respectively to obtain the second voice information and the corresponding second audio information; wherein the transformation process includes a room impulse response; The second voice information and the corresponding second audio information are fused to obtain the fused voice information. 5. The method according to claim 2, characterized in that: In the case where the target person is an unregistered person, the optimized adversarial audio information is used to: make the verification result output by the voice verification device be a successful verification; In the case where the target person is a registered person, the optimized adversarial audio information is used to make the verification result output by the voice verification device be verification failure. 6. The method according to claim 1, characterized in that the voice verification device comprises a voice recognition module, a speaker verification module, and a playback detection module; Among them, the voice recognition module is used to identify the content information of the verification voice information, the speaker verification module is used to identify whether the verification voice information features belong to the voice information features of a registered person, and the playback detection module is used to detect whether the verification voice information is voice information played back after recording. 7. A security testing device for a voice verification device, characterized in that the device comprises: A voice sample set acquisition module, used to acquire a voice sample set, wherein the voice sample set includes a plurality of first voice information of a target person; A fusion module, used for fusing the adversarial audio information with the plurality of first voice information respectively to obtain a plurality of fused voice information; A feature acquisition module, used for inputting the plurality of fused voice information into the voice verification device respectively to obtain a first voice feature of the target person; an optimization module, for optimizing the adversarial audio information according to the first voice feature of the target person and the second voice feature of the registered person stored in the voice verification device to obtain optimized adversarial audio information, wherein the optimized adversarial audio information is used to perform a security test on the voice verification device, Wherein, the voice verification device is used to verify the identity of the registered person through voice. 8. The device according to claim 7, characterized in that the device further comprises: An audio playback module, used to control an audio playback device to play the optimized adversarial audio information; A verification result acquisition module, used to obtain a verification result of the voice verification device for the verification voice information, wherein the verification voice information includes the optimized confrontation audio information and the real verification voice emitted by the target person during the period when the audio playback device plays the optimized confrontation audio information; The security test result determination module is used to determine the security test result of the voice verification device according to the verification result. 9. An electronic device, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5. 10. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method according to any one of claims 1 to 5.