CN113763982A - Audio processing method and device, electronic equipment and readable storage medium - Google Patents

Abstract

Embodiments of the present disclosure disclose an audio processing method, an apparatus, an electronic device, and a readable storage medium. The audio processing method includes: acquiring first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone; determining the first audio data and the second audio data The main sound source orientation of the data; a target noise ratio is determined based on the first audio data, the second audio data and the main sound source orientation, the target noise ratio representing the respective differences of the first audio data and the second audio data a ratio of desired signal energy to undesired signal energy; and, based on the target noise ratio, filtering the first audio data and/or the second audio data and based on the filtered first audio data and/or the second audio data The audio data obtains the target audio data, which improves the estimation accuracy of the noise parameters, so that the signal of the desired audio source can be better extracted from the environment.

Description

Audio processing method, apparatus, electronic device and readable storage medium

technical field

The present disclosure relates to the field of computer technologies, and in particular, to an audio processing method, an apparatus, an electronic device, and a readable storage medium.

Background technique

When shooting video, recording, conducting a voice call or remote conference, the signal received by the microphone is the result of the superposition of the desired signal and the undesired noise signal. The actual environment is often accompanied by various types of noise, including steady-state Gaussian white noise and non-steady-state noise, such as canteens, supermarkets, restaurants and other sound places. Noisy, affecting the listening experience. In severe cases, the desired sound may even be masked by noise, and the desired speech content cannot be obtained.

The basic idea of audio noise reduction is to use spectral subtraction, built-in many background noise samples in different environments, and calculate and match the most similar noise samples to deal with different actual environments. However, the effect of suppressing non-steady-state noise is weak. The dual microphone array can locate the sound source and extract the sound source at the desired position through beamforming, which can offset the ambient noise to a certain extent. In addition, the inaccuracy of the method to locate the target sound source in a noisy environment directly affects the noise parameter estimation, which may lead to misjudgment, resulting in large distortion of the expected speech and affecting the listening experience.

SUMMARY OF THE INVENTION

In order to solve the problems in the related art, the embodiments of the present disclosure provide an audio processing method, an apparatus, an electronic device, and a readable storage medium.

In a first aspect, an audio processing method is provided in the embodiments of the present disclosure.

Specifically, the audio processing method includes:

acquiring first audio data collected by the first microphone and second audio data corresponding to the first audio data collected by the second microphone;

Determine the main sound source position of the first audio data and the second audio data. Set the position of the sound source of the condition;

A target noise ratio is determined based on the first audio data, the second audio data, and the main audio source orientation, the target noise ratio representing the desired signal energy and the undesired signal energy of each of the first audio data and the second audio data the ratio of signal energy; and

Based on the target noise ratio, filtering the first audio data and/or the second audio data and obtaining target audio data based on the filtered first audio data and/or the second audio data.

In conjunction with the first aspect, in a first implementation manner of the first aspect of the present disclosure, the method further includes:

Before determining the target noise ratio, the frequency spectrum of the first audio data and the frequency spectrum of the second audio data are acquired.

With reference to the first implementation manner of the first aspect, in the second implementation manner of the first aspect of the present disclosure, the target noise ratio is determined based on the first audio data, the second audio data, and the position of the main audio source, include:

For a specified frequency point, determine a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data;

A target-to-noise ratio for the specified frequency point is determined based on the correlation function and the orientation of the main sound source.

With reference to the second implementation manner of the first aspect, in a third implementation manner of the first aspect of the present disclosure, the determining the target noise ratio of the specified frequency point based on the correlation function and the main sound source position includes:

determining a desired signal component representation and an undesired signal component representation of the real part of the correlation function;

determining a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function;

The assignment is determined based on the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function, and the main sound source orientation The target-to-noise ratio of the frequency point.

With reference to the second implementation manner of the first aspect, in a fourth implementation manner of the first aspect of the present disclosure, the target noise ratio is determined based on the first audio data, the second audio data, and the position of the main audio source, It also includes acquiring the target-to-noise ratio of each frequency point in the frequency spectrum.

With reference to the first implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the first audio data and the second audio data are filtered based on the target noise ratio and based on The filtered first audio data and the second audio data obtain target audio data, including:

based on the target noise ratio, filtering the frequency spectrum of the first audio data and the frequency spectrum of the second audio data;

The time domain representation of the first audio data is obtained from the spectrum of the filtered first audio data as third audio data, and/or the time domain representation of the second audio data is obtained from the spectrum of the filtered second audio data represented as fourth audio data;

Target audio data is acquired based on the third audio data and/or the fourth audio data.

With reference to the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect of the present disclosure, the filtering of the first audio data and the second audio data based on the target noise ratio includes: :

Obtain the azimuth range of the desired sound source;

Based on the azimuth of the main audio source and the azimuth range of the desired audio source, a judgment result of whether the current audio data is desired audio data or undesired audio data is obtained, and the current audio data is the first audio data or the second audio data;

updating spatial filter coefficients based on the judgment result, the current audio data, and the target noise ratio;

The current audio data is filtered by the updated spatial filter coefficients.

With reference to the sixth implementation manner of the first aspect, in the seventh implementation manner of the present disclosure, the spatial filter coefficients are updated based on the judgment result, the current audio data, and the target noise ratio, include:

When the current audio data is desired audio data, update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio;

When the current audio data is undesired audio data, update the noise covariance matrix and the global covariance matrix of the current audio data based on the current audio data and the target noise ratio;

Spatial filter coefficients are updated based on the noise covariance matrix and the global covariance matrix.

With reference to the first aspect, in an eighth implementation manner of the first aspect of the present disclosure, the sound source azimuth whose probability meets the preset condition includes the sound source azimuth with the highest probability.

In a second aspect, an embodiment of the present disclosure provides an audio processing apparatus.

Specifically, the audio processing device includes:

a first acquisition module, configured to acquire first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone;

The first determining module is configured to determine the position of the main sound source of the first audio data and the second audio data, and the main sound source position includes the position of the first audio data and the second audio data. The position of the sound source whose probability meets the preset conditions among the positions of the multiple sound sources;

A second determination module configured to determine a target noise ratio based on the first audio data, the second audio data and the main audio source orientation, the target noise ratio representing the first audio data and the second audio data the respective ratios of desired signal energy to undesired signal energy; and

A second obtaining module, configured to filter the first audio data and/or the second audio data based on the target noise ratio, and obtain target audio based on the filtered first audio data and/or the second audio data data.

In conjunction with the second aspect, in a first implementation manner of the second aspect of the present disclosure, the apparatus further includes:

The third acquisition module is configured to acquire the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target noise ratio.

With reference to the first implementation manner of the second aspect, in the second implementation manner of the second aspect of the present disclosure, the determining module includes:

a first determination submodule, configured to determine, for a specified frequency point, a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data;

The second determination submodule is configured to determine the target noise ratio of the specified frequency point based on the correlation function and the position of the main sound source.

With reference to the second implementation manner of the second aspect, in a third implementation manner of the second aspect of the present disclosure, the second determination submodule includes:

a first determination unit configured to determine a desired signal component representation and an undesired signal component representation of the real part of the correlation function;

a second determination unit configured to determine a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function;

a third determination unit configured to represent desired and undesired signal components based on the real part of the correlation function, the desired and undesired signal component representations of the imaginary part of the correlation function, and the The orientation of the main sound source, and the target noise ratio of the specified frequency point is determined.

With reference to the second implementation manner of the second aspect, in a fourth implementation manner of the second aspect of the present disclosure, the second determination module further includes a first acquisition sub-module configured to acquire each frequency point in the frequency spectrum target-to-noise ratio.

With reference to the first implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the second acquisition module includes:

a filtering submodule configured to filter the frequency spectrum of the first audio data and the frequency spectrum of the second audio data based on the target-to-noise ratio;

A second acquisition sub-module configured to acquire a time-domain representation of the first audio data from the spectrum of the filtered first audio data as third audio data, and/or to acquire from the spectrum of the filtered second audio data a time domain representation of the second audio data as the fourth audio data;

A third obtaining sub-module is configured to obtain target audio data based on the third audio data and/or the fourth audio data.

With reference to the fifth implementation manner of the second aspect, in the sixth implementation manner of the second aspect of the present disclosure, the filtering submodule includes:

a first obtaining unit, configured to obtain the azimuth range of the desired sound source;

a second obtaining unit, configured to obtain a judgment result of whether the current audio data is expected audio data or undesired audio data based on the azimuth of the main sound source and the azimuth range of the desired sound source, where the current audio data is the first audio data audio data or the second audio data;

an update unit configured to update the spatial filter coefficient based on the judgment result, the current audio data and the target noise ratio;

A filtering unit configured to filter the current audio data through the updated spatial filter coefficients.

In conjunction with the sixth implementation manner of the second aspect, in the seventh implementation manner of the second aspect of the present disclosure, the updating unit includes:

A first update subunit, configured to update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio when the current audio data is the desired audio data;

a second update subunit, configured to update the noise covariance matrix of the current audio data and the global covariance matrix;

A third update subunit is configured to update spatial filter coefficients based on the noise covariance matrix and the global covariance matrix.

With reference to the second aspect, in an eighth implementation manner of the second aspect of the present disclosure, the sound source azimuth whose probability meets the preset condition includes the sound source azimuth with the highest probability.

In a third aspect, an embodiment of the present disclosure provides an audio processing method, including:

Obtain N pieces of audio data corresponding to each other collected by N microphones, N≥3;

determining one or more pairs of audio data based on the N audio data;

For each pair of audio data, determine the location of the main sound source of the audio data corresponding to the pair of audio data, where the location of the main sound source includes a sound source whose probability meets a preset condition among the locations of multiple audio sources located for the pair of audio data Orientation; determine a target noise ratio based on the audio data corresponding to the audio data pair and the main audio source orientation, where the target noise ratio represents the respective expected signal energy and undesired signal energy of the audio data corresponding to the audio data pair based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data;

Target audio data is determined based on the filtered audio data obtained from the one or more pairs of audio data.

With reference to the third aspect, in a first implementation manner of the third aspect of the present disclosure, the determining one or more pairs of audio data based on the plurality of audio data includes:

determining the one or more pairs of audio data according to the positional relationship of the N microphones; or

Any two audio data in the plurality of audio data are formed into an audio data pair.

With reference to the first implementation manner of the third aspect, in the second implementation manner of the third aspect of the present disclosure, the determining of the one or more audio data pairs according to the positional relationship of the N microphones includes:

If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair.

In conjunction with the third aspect, in a third implementation manner of the third aspect of the present disclosure, the determining of target audio data based on the filtered audio data obtained from the one or more pairs of audio data includes:

obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or

Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data.

With reference to the third aspect, in a fourth implementation manner of the third aspect of the present disclosure, the sound source azimuth whose probability meets the preset condition includes the sound source azimuth with the highest probability.

In a fourth aspect, an embodiment of the present disclosure provides an audio processing apparatus, including:

a fourth acquisition module, configured to acquire N pieces of audio data corresponding to each other collected by the N microphones, N≥3;

a third determining module configured to determine one or more pairs of audio data based on the N pieces of audio data;

The filtering module is configured to, for each pair of audio data, determine the position of the main sound source of the audio data corresponding to the pair of audio data, and the position of the main sound source includes a probability among the positions of the multiple sound sources located for the pair of audio data The sound source orientation that meets the preset conditions; the target noise ratio is determined based on the audio data corresponding to the audio data pair and the main sound source orientation, and the target noise ratio represents the respective expected signals of the audio data pair corresponding to the audio data. a ratio of energy to undesired signal energy; based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data;

A fourth determination module configured to determine target audio data based on the filtered audio data obtained from the one or more pairs of audio data.

With reference to the fourth aspect, in a first implementation manner of the fourth aspect of the present disclosure, the third determining module includes:

a third determination submodule, configured to determine the one or more audio data pairs according to the positional relationship of the N microphones; or

The fourth determining submodule is configured to form an audio data pair from any two audio data in the plurality of audio data.

With reference to the first implementation manner of the fourth aspect, in the second implementation manner of the fourth aspect of the present disclosure, the third determination submodule is configured as:

If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair.

In conjunction with the fourth aspect, in a third implementation manner of the fourth aspect, the fourth determination module is configured to:

obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or

Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data.

With reference to the fourth aspect, in a fourth implementation manner of the fourth aspect of the present disclosure, the sound source azimuth whose probability meets the preset condition includes the sound source azimuth with the highest probability.

In a fifth aspect, embodiments of the present disclosure provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are processed by the The device executes to implement any one of the first aspect, the first implementation manner to the eighth implementation manner of the first aspect, the third aspect, the first implementation manner of the third aspect to the fourth implementation manner Methods.

In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, implement the first aspect, the first implementation manner of the first aspect to The method described in any one of the eighth implementation manner, the third aspect, and the first implementation manner of the third aspect to the fourth implementation manner.

According to the technical solutions provided by the embodiments of the present disclosure, first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone are obtained; the first audio data and The position of the main sound source of the second audio data, the position of the main sound source includes the position of the sound source whose probability meets the preset condition among the plurality of sound source positions located on the first audio data and the second audio data; The first audio data, the second audio data and the main sound source orientation determine a target noise ratio, the target noise ratio representing the desired signal energy and the undesired signal energy of the first audio data and the second audio data respectively. and, based on the target noise ratio, filtering the first audio data and the second audio data and obtaining the target audio data based on the filtered first audio data and the second audio data, which improves the noise parameter. Estimate the accuracy, so that the signal of the desired sound source can be better extracted from the environment.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the attached image:

1 shows a flowchart of an audio processing method according to an embodiment of the present disclosure;

FIG. 2 shows a flowchart of determining a target noise ratio according to an embodiment of the present disclosure;

3 shows a flowchart of acquiring target audio data according to an embodiment of the present disclosure;

4A shows a flowchart of filtering the first audio data and the second audio data according to an embodiment of the present disclosure;

4B shows a schematic diagram of an azimuth range of a desired sound source according to an embodiment of the present disclosure;

5 shows a schematic diagram of updating spatial filter coefficients according to an embodiment of the present disclosure;

6 shows a schematic diagram of an audio processing method according to another embodiment of the present disclosure;

7 shows a block diagram of an audio processing apparatus according to an embodiment of the present disclosure;

8 shows a block diagram of a second determination module according to an embodiment of the present disclosure;

9 shows a block diagram of a second acquisition module according to an embodiment of the present disclosure;

10 shows a block diagram of a filtering sub-module according to an embodiment of the present disclosure;

11 shows a flowchart of an audio processing method according to another embodiment of the present disclosure;

12A-12C illustrate schematic diagrams of a plurality of microphones according to an embodiment of the present disclosure;

13 shows a block diagram of an audio processing apparatus according to another embodiment of the present disclosure;

FIG. 14 shows a block diagram of an electronic device according to an embodiment of the present disclosure; and

FIG. 15 shows a schematic structural diagram of a computer system suitable for implementing an audio processing method according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts unrelated to describing the exemplary embodiments are omitted from the drawings.

In the present disclosure, it should be understood that terms such as "comprising" or "having" are intended to indicate the presence of features, numbers, steps, acts, components, parts, or combinations thereof disclosed in this specification, and are not intended to exclude a or multiple other features, numbers, steps, acts, components, parts, or combinations thereof may exist or be added.

In addition, it should be noted that the embodiments of the present disclosure and the features of the embodiments may be combined with each other under the condition of no conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

When shooting video, recording, conducting a voice call or remote conference, the signal received by the microphone is the result of the superposition of the desired signal and the undesired noise signal. The actual environment is often accompanied by various types of noise, including steady-state Gaussian white noise and non-steady-state noise, such as canteens, supermarkets, restaurants and other sound places. Noisy, affecting the listening experience. In severe cases, the desired sound may even be masked by noise, and the desired speech content cannot be obtained.

For audio noise reduction, the basic idea of single-mic noise reduction is to use spectral subtraction, obtain the audio data of the noise segment through voice activity detection, further estimate the noise spectrum, and subtract the estimated noise spectrum from the received sound spectrum to obtain the desired speech. Element. Considering the diversity of environments, some devices will have many built-in background noise samples in different environments, and calculate and match the most similar noise samples to deal with different actual environments. Since the actual environment is very random, and the sound components in the same environment are very different, this noise reduction method with built-in noise samples has a certain effect on steady-state noise, but it requires a lot of noise samples, which requires a lot of work, and for Astable noise suppression effect is weak. At the same time, a single microphone cannot locate the sound source, and cannot achieve directional enhancement of the sound within a certain pickup range.

Therefore, in some small communication equipment or recording equipment, the dual-microphone array has become the first choice for enhancing the desired sound and suppressing the noise. The method of addition (delay and sum) corrects the direct delay of the two channels by estimating the sound source localization result, and then superimposes the data of the two channels to offset the ambient noise to a certain extent, but in the case of a large reverberation or a noisy environment. The extraction effect is poor, and spectral subtraction is still needed to achieve the final noise reduction; the minimum variance distortionless response method (MVDR, Minimum variance distortionless response) is an adaptive beamforming method. The positioning results are used to estimate the noise spectrum parameters, and the steering vector is calculated to update the spatial filter coefficients, and then the desired signal is separated from the signal. This method locates the target sound source in a noisy environment. noise) parameter estimation, misjudgment may occur, resulting in large distortion of the expected speech, affecting the listening experience.

The audio processing method provided by the embodiment of the present disclosure obtains first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone; The position of the main sound source of the second audio data, the position of the main sound source includes the position of the sound source whose probability meets the preset condition among the plurality of sound source positions located on the first audio data and the second audio data; The first audio data, the second audio data, and the orientation of the main audio source determine a target noise ratio, the target noise ratio representing the difference between the desired signal energy and the undesired signal energy of the first audio data and the second audio data respectively. and, based on the target-to-noise ratio, filtering the first audio data and/or the second audio data and obtaining target audio data based on the filtered first audio data and/or the second audio data, improving the The estimation accuracy of noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

FIG. 1 shows a flowchart of an audio processing method according to an embodiment of the present disclosure. As shown in FIG. 1, the audio processing method includes the following steps S110-S140:

In step S110, first audio data collected by the first microphone and second audio data corresponding to the first audio data collected by the second microphone are obtained;

In step S120, determine the main sound source orientation of the first audio data and the second audio data, where the main sound source orientation includes multiple audio sources located for the first audio data and the second audio data The position of the sound source whose probability meets the preset conditions in the position;

In step S130, a target noise ratio is determined based on the first audio data, the second audio data and the orientation of the main audio source, the target noise ratio representing the respective expectations of the first audio data and the second audio data The ratio of signal energy to undesired signal energy;

In step S140, based on the target noise ratio, the first audio data and/or the second audio data are filtered, and target audio data is acquired based on the filtered first audio data and/or the second audio data.

According to the technical solutions of the embodiments of the present disclosure, by acquiring the first audio data collected by the first microphone and the second audio data corresponding to the first audio data collected by the second microphone; The position of the main sound source of the second audio data, the position of the main sound source includes the position of the sound source whose probability meets the preset condition among the plurality of sound source positions located on the first audio data and the second audio data; The first audio data, the second audio data, and the main audio source orientation determine a target noise ratio, the target noise ratio representing the difference between the desired signal energy and the undesired signal energy of the first audio data and the second audio data. ratio; based on the target noise ratio, filtering the first audio data and/or the second audio data and obtaining target audio data based on the filtered first audio data and/or the second audio data, The signal-to-noise ratio between the desired signal energy in the sound area and the undesired signal energy outside the pickup area improves the estimation accuracy of noise parameters, so that the signal of the desired sound source in the noisy environment can be better extracted from the environment.

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

According to the embodiment of the present disclosure, the angle between the ray passing through the midpoint of the directional connection line between the two microphones and the direction of the directional connection line can be used to represent the sound source orientation.

According to an embodiment of the present disclosure, the method further includes acquiring the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target-to-noise ratio.

According to the technical solutions of the embodiments of the present disclosure, by acquiring the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target noise ratio, the estimation accuracy of the noise parameter is improved, so that the noise parameter can be better estimated. Extract the signal of the desired source from the environment.

According to the embodiment of the present disclosure, for the audio data received by the first microphone, the first audio data can be obtained by framing it, and the audio data received by the second microphone can be divided into frames to obtain the second audio data, The first audio data and the second audio data are audio frames containing audio data collected in the same time period. The first audio data may be windowed, and a frequency spectrum of the first audio data may be obtained through Fourier transform. For example, the frame may be divided according to the frame length of 20ms and the frame shift of 10ms, that is, the first audio data may be divided into segments such as 0-20ms, 10ms-30ms, 20ms-40ms, ie audio frames. These segments are windowed so that the head and tail changes are continuous, and a segment is converted into a periodic signal. Fourier transform is performed on the windowed result to obtain the spectrum of each audio frame. The Fourier transform length can be preset, for example, set to 1024 or even higher, and the Fourier transform length determines the values of multiple frequency points in the discrete spectrum. The values of multiple frequency points ω _l =2πl/L, l=0,1,2,...,L-1, where L is the Fourier transform length. Similarly, for the second audio data received by the second microphone, it can be similarly windowed, and the frequency spectrum of the second audio data can be obtained through Fourier transform.

According to the embodiments of the present disclosure, the position of the main sound source of the current frame can be detected by using the existing sound source location technology, where the position of the main sound source is the most of the positions of the first audio data and the second audio data. Among the sound source positions, the sound source positions whose probability meets the preset conditions. The audio frame may include multiple sounds incident from different angles, and the sound source localization technology can locate multiple sound source angles and their corresponding probabilities, and use the angle with the highest probability as the main sound source orientation, which can also be used to judge the current frame. Whether to base the desired frame.

FIG. 2 shows a flowchart of determining a target noise ratio according to an embodiment of the present disclosure. As shown in FIG. 2, step S130 may include the following steps S210-S220:

In step S210, for a specified frequency point, a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data is determined;

In step S220, the target noise ratio of the designated frequency point is determined based on the correlation function and the orientation of the main sound source.

According to the technical solutions of the embodiments of the present disclosure, a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data is determined for a specified frequency point; The target noise ratio of the specified frequency point is described, which improves the estimation accuracy of the noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

The correlation function of two signals can be expressed as:

表示互能量谱密度，定义为Φ_uv(ω,k)＝E[U(ω,k)V^*(ω,k)]，E表示求能量的过程，*表示共轭符号。

和

表示自能量谱密度，定义为Φ_uu(ω,k)＝E[|U(ω,k)|²]。其中，ω为频点，k为音频帧的编号，U(ω，k)是信号U的频谱，V(ω，k)是信号V的频谱。in,

Represents the cross-energy spectral density, which is defined as Φ _uv (ω,k)=E[U(ω,k)V ^* (ω,k)], E represents the process of finding energy, and * represents the conjugate symbol.

and

represents the self-energy spectral density, defined as Φ _uu (ω,k)=E[|U(ω,k)| ² ]. Among them, ω is the frequency point, k is the number of the audio frame, U(ω, k) is the frequency spectrum of the signal U, and V(ω, k) is the frequency spectrum of the signal V.

The correlation function of two signals with an incident angle θ can be expressed as:

表示声速，d表示麦克风间距，单位米。where f _s is the sampling rate,

Indicates the speed of sound, and d represents the distance between the microphones, in meters.

The correlation function of the two signals expressed as the target noise ratio can be written as:

和

分别表示带噪信号相关函数，期望信号的相关函数和非期望噪音信号的相关函数，TNR₁和TNR₂分别表示两个麦克风信号的目标噪音比。in,

and

Denote the correlation function of the noisy signal, the correlation function of the desired signal and the correlation function of the undesired noise signal, respectively, and TNR ₁ and TNR ₂ denote the target-to-noise ratio of the two microphone signals, respectively.

估计目标-噪音比记为

所以上式可以修正为：Since the distance between two dual microphone arrays is often only 5 to 20cm, so

The estimated target-to-noise ratio is recorded as

So the above formula can be modified as:

Substituting formula (2) into formula (4), assuming that the angle between the direction of the main sound source to be extracted and the 0° direction of the microphone array is θ, the correlation function can be written as:

where τ=f _s (d/c), β=cosβ ₁ +cosβ ₂ +...+cosβ _N represents the sum of several noise sources other than the angle θ. For example, the main sound source azimuth θ is determined to be 100 degrees through sound source localization, and other azimuths other than 100 degrees in the current frame are determined to be β ₁ , β ₂ , . . . , β _N through sound source localization.

In equations (3), (4) or (5), the first term on the right side of the equation is the expected signal component in the correlation function, and the second term is the undesired signal component in the correlation function. The method of the embodiment of the present disclosure determines the target noise ratio of the specified frequency point based on the desired signal component and the undesired signal component, which improves the estimation accuracy of the noise parameter, so that the desired sound source can be better extracted from the environment signal of.

According to an embodiment of the present disclosure, the determining the target-to-noise ratio of the specified frequency based on the desired signal component and the undesired signal component includes:

determining a desired signal component representation and an undesired signal component representation of the real part of the correlation function;

determining a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function;

The assignment is determined based on the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function, and the main sound source orientation The target-to-noise ratio of the frequency point.

According to the technical solutions of the embodiments of the present disclosure, by determining the desired signal component representation and the undesired signal component representation of the real part of the correlation function; determining the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function; The assignment is determined based on the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function, and the main sound source orientation The target-to-noise ratio of the frequency point improves the estimation accuracy of the noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

α＝ωτcosθ，从而，相关函数的实部R和虚部I可以分别表示为：According to the embodiment of the present disclosure, on the basis of formula (5), we can make

α=ωτcosθ, thus, the real part R and imaginary part I of the correlation function can be expressed as:

After sorting, it can be seen that:

的方程：Through the simultaneous formula (8), it can be listed about

The equation for :

或

Among them, I and R can be calculated by formula (1), α=ωτcosθ is a known quantity, therefore, it can be solved

or

For example, note:

A=I-sinα,

B=cosα-R,

C=Rsinα-Icosα. (10)

The equation can be solved to get:

Substitute Equation (11) into Equation (8) to determine the target noise ratio at the specified frequency point.

According to an embodiment of the present disclosure, the determining a target noise ratio based on the first audio data, the second audio data, and the orientation of the main audio source further includes acquiring the target noise ratio of each frequency point in the frequency spectrum. By performing the above operations on each frequency point, the target noise ratio of each frequency point in the spectrum can be obtained, so that each frequency on the audio frame spectrum can be filtered to obtain the target audio data, so that the target audio data can be better extracted from the environment The signal of the desired source in a noisy environment.

FIG. 3 shows a flowchart of acquiring target audio data according to an embodiment of the present disclosure. As shown in FIG. 3, step S140 may include the following steps:

In step S310, filtering the frequency spectrum of the first audio data and/or the frequency spectrum of the second audio data based on the target noise ratio;

In step S320, the time domain representation of the first audio data is obtained from the spectrum of the filtered first audio data as third audio data and/or the second audio is obtained from the spectrum of the filtered second audio data a time domain representation of the data as fourth audio data;

In step S330, target audio data is acquired based on the third audio data and/or the fourth audio data.

According to an embodiment of the present disclosure, in step S320, the frequency spectrum of the filtered first audio data and/or the frequency spectrum of the filtered second audio data may be correspondingly converted into a third time-domain representation through inverse Fourier transform. audio data and/or fourth audio data. For example, for the spectrum of the filtered first audio data, the time-domain representation of each audio frame can be obtained through inverse Fourier transform, and then a windowing operation can be used to process and superimpose multiple frames before and after to obtain the time-domain representation of the first audio frame. Three audio data. The windowing operation can make the superimposed continuous audio data relatively smooth and continuous between frames.

According to the embodiment of the present disclosure, step S330 can be implemented in various manners. For example, step S330 may be used to superimpose the third audio data and the fourth audio data into one target audio data, and output the superimposed target audio data to the user. Alternatively, step S330 may be adopted to output the third audio data and the fourth audio data together as target audio data to the user. Any one of the data is output to the user as the target audio data.

According to the technical solutions of the embodiments of the present disclosure, filtering the frequency spectrum of the first audio data and/or the frequency spectrum of the second audio data based on the target noise ratio; obtaining a time-domain representation of the first audio data as third audio data, and/or obtaining a time-domain representation of the second audio data from the spectrum of the filtered second audio data as fourth audio data; based on the The third audio data and/or the fourth audio data are used to obtain the target audio data, which improves the estimation accuracy of the noise parameter, so that the signal of the desired audio source can be better extracted from the environment.

4A shows a flowchart of filtering the first audio data and the second audio data according to an embodiment of the present disclosure. As shown in FIG. 4A, step S310 may include the following steps:

In step S410, obtain the azimuth range of the desired sound source;

In step S420, based on the azimuth of the main audio source and the azimuth range of the desired audio source, obtain a judgment result that the current audio data is desired audio data or undesired audio data, and the current audio data is the first audio data or the second audio data;

In step S430, the spatial filter coefficients are updated based on the judgment result, the current audio data and the target noise ratio;

In step S440, the current audio data is filtered through the updated spatial filter coefficients.

According to the embodiment of the present disclosure, two microphones may be provided on the electronic device to form a microphone array. According to the common use state of the electronic device, the range of the direction in which the user's mouth should be positioned relative to the microphone can be pre-defined as the range of the desired sound source.

For example, a directional line can be made between two microphones, which will pass through the midpoint of the directional line between the two microphones, and the angle formed with the direction of the directional line is within a preset range. The angle area of is used as the azimuth range of the desired sound source. The azimuth range of the desired sound source may be symmetrical, for example, 45 degrees to 135 degrees, so the boundary of the azimuth range of the desired sound source may form a quadratic cone in space. For another example, the azimuth range of the desired sound source can be asymmetric, for example, it can be set to 40 degrees to 180 degrees. When the azimuth is 160 degrees, it is determined as expected data, but when the azimuth is 20 degrees, it is determined as undesired data. According to an embodiment of the present disclosure, the orientation of the sound source is defined as the included angle between a line from the position of the sound source to the midpoint of the directional line between the two microphones and the direction of the directional line. In practical applications, the azimuth range of the desired sound source can be set as required.

For example, as shown in FIG. 4B , M ₁ and M ₂ indicate the positions of the two microphones, and M ₀ is the midpoint of the directional connection line between M ₁ and M ₂ . The angular region passing through M ₀ and the angle formed with the direction of the directional connection line is within a preset range (between θ ₁ and θ ₂ ) as the azimuth range of the desired sound source, as shown by the dotted line in FIG. 4B . For example, θ ₁ can be 45 degrees and θ ₂ can be 135 degrees. FIG. 4B is only a two-dimensional schematic diagram. In a practical application in a three-dimensional space, the dotted line can be rotated in the three-dimensional space with a directional connection line as an axis to determine an axially symmetric spatial region as the azimuth range of the desired sound source.

According to an embodiment of the present disclosure, the current audio data is data of an audio frame. For each audio frame, as described above, it is determined whether the orientation of the main audio source for that audio frame is within the orientation range of the desired audio source. If it is within the azimuth range of the expected sound source, it is determined that the frame is an expected frame, and if it is not within the azimuth range of the expected sound source, it is determined that the frame is not an expected frame. According to the judgment result of whether it is a desired frame and the target noise ratio determined in the preceding steps, the spatial filter coefficients can be updated, and the audio frame is filtered based on the updated filter coefficients.

According to the technical solutions of the embodiments of the present disclosure, the azimuth range of the desired audio source is obtained; based on the azimuth range of the main audio source and the azimuth range of the desired audio source, the judgment result of whether the current audio data is desired audio data or undesired audio data is obtained; The judgment result, the current audio data, and the target noise ratio update the spatial filter coefficients; the current audio data is filtered by the updated spatial filter coefficients, which improves the estimation accuracy of noise parameters, so that it can be more accurate. The signal of the desired source is well extracted from the environment.

FIG. 5 shows a schematic diagram of updating spatial filter coefficients according to an embodiment of the present disclosure.

As shown in FIG. 5 , the updating the spatial filter coefficients based on the judgment result, the current audio data and the target noise ratio includes: in the case that the current audio data is expected audio data, based on the The current audio data and the target noise ratio update the global covariance matrix of the current audio data; when the current audio data is undesired audio data, update the current audio data and the target noise ratio based on the current audio data and the target noise ratio. the noise covariance matrix and the global covariance matrix of the current audio data; and the spatial filter coefficients are updated based on the noise covariance matrix and the global covariance matrix.

According to an embodiment of the present disclosure, the noise covariance matrix is used to represent the covariance of the noise signal, and the global covariance matrix is used to represent the covariance of the whole signal (including the noise signal and the non-noise signal).

For example, assuming that the signal value of the current frame at a certain frequency is X, and the target noise ratio is 0.25, it means that the component of (1/(1+0.25)) in the current value X is noise, that is, an undesired component. The elements in the matrix can be calculated by the above method to realize the update of the covariance matrix. According to the minimum variance undistorted response method MVDR algorithm, the noise covariance matrix and the global covariance matrix can be used to determine the coefficients of the air filter filter, so that the current audio data can be filtered by the MVDR algorithm.

According to the technical solutions of the embodiments of the present disclosure, when the current audio data is desired audio data, the global covariance matrix of the current audio data is updated based on the current audio data and the target noise ratio; When the current audio data is undesired audio data, the noise covariance matrix and the global covariance matrix of the current audio data are updated based on the current audio data and the target noise ratio; based on the noise covariance The matrix and the global covariance matrix update the coefficients of the spatial filter, which improves the estimation accuracy of the noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

FIG. 6 shows a schematic diagram of an audio processing method according to another embodiment of the present disclosure.

根据Flag、TNR、Y_1(ω,k)和Y_2(ω,k)，可以确定MVDR算法所需要的空域滤波器系数W(ω,k)。根据W(ω,k)对Y_1(ω,k)和Y_2(ω,k)进行滤波，并通过傅里叶逆变换(IFFT)、叠加，最终得到目标音频数据y(m)，并输出该y(m)。As shown in FIG. 6 , the audio data of channel 1 collected by microphone 1 is y_1(m), and the audio data of channel 2 collected by microphone 2 is y_2(m). The audio frames Y_1(ω,k) and Y_2(ω,k) represented in the frequency domain are obtained by framing, windowing and Fourier transform (FFT) respectively. On the one hand, perform sound source localization on Y_1(ω,k) and Y_2(ω,k) respectively, determine the orientation of the main sound source of each audio frame, and pick up the sound source according to the orientation range of the desired sound source (that is, the pickup range in Figure 6). Tone range decision, to determine whether it is a desired frame, for example, it can be marked by Flag (identifier). On the other hand, the mutual energy spectral density Φ ₁₂ (ω, k) and the self energy spectral density Φ ₁₁ (ω, k) and Φ ₂₂ (ω, k) are calculated for Y_1(ω,k) and Y_2(ω,k) , and further determine the target-to-noise ratio TNR, that is, the aforementioned

According to Flag, TNR, Y_1(ω,k) and Y_2(ω,k), the spatial filter coefficient W(ω,k) required by the MVDR algorithm can be determined. Y_1(ω,k) and Y_2(ω,k) are filtered according to W(ω,k), and the target audio data y(m) is finally obtained through inverse Fourier transform (IFFT) and superposition, and output the y(m).

The technical solutions of the embodiments of the present disclosure can accurately estimate the signal-to-noise ratio between the desired signal and the undesired noise signal in the pickup area by adding a target-to-noise ratio estimator on the basis of MVDR beamforming by using a dual-microphone array, thereby improving noise The estimation accuracy of the parameters is based on the dual microphone array to achieve the directional pickup of the desired sound source in the noisy environment. At the same time, because it does not depend on the accurate result of sound source localization, it only judges whether it is in the incident area of the desired sound source, which reduces the requirements for sound source localization. desired ingredients.

FIG. 7 shows a block diagram of an audio processing apparatus 700 according to an embodiment of the present disclosure. Wherein, the apparatus may be realized by software, hardware or a combination of the two to become part or all of the electronic device.

As shown in FIG. 7 , the audio processing apparatus 700 includes a first obtaining module 710 , a first determining module 720 , a second determining module 730 and a third obtaining module 740 .

a first acquiring module 710, configured to acquire first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone;

The first determination module 720 is configured to determine the main sound source position of the first audio data and the second audio data, and the main sound source position includes locating the first audio data and the second audio data. The position of the sound source whose probability meets the preset conditions among the positions of the multiple sound sources;

The second determining module 730 is configured to determine a target noise ratio based on the first audio data, the second audio data and the main audio source position, where the target noise ratio represents the first audio data and the second audio the ratio of the respective expected signal energy to the undesired signal energy of the data; and

A second obtaining module 740, configured to filter the first audio data and/or the second audio data based on the target-to-noise ratio, and obtain a target based on the filtered first audio data and/or the second audio data audio data.

According to the technical solutions of the embodiments of the present disclosure, the first acquisition module is configured to acquire the first audio data collected by the first microphone and the second audio data corresponding to the first audio data collected by the second microphone; a determining module, configured to determine the main sound source location of the first audio data and the second audio data, the main sound source location including the location of the first audio data and the second audio data. Among the sound source azimuths, a sound source azimuth whose probability meets a preset condition; the second determining module is configured to determine a target noise ratio based on the first audio data, the second audio data and the main sound source azimuth, and the target noise ratio represents The ratio of the desired signal energy to the undesired signal energy of the first audio data and the second audio data; a second acquisition module, configured to based on the target noise ratio, for the first audio data and/or or the second audio data is filtered and the target audio data is obtained based on the filtered first audio data and/or the second audio data, which improves the estimation accuracy of the noise parameter, so that the signal of the desired audio source can be better extracted from the environment .

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

According to an embodiment of the present disclosure, the apparatus further includes a third acquisition module configured to acquire the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target noise ratio.

According to the technical solutions of the embodiments of the present disclosure, the third acquisition module is configured to acquire the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target noise ratio, which improves the estimation of noise parameters accuracy, so that the signal of the desired sound source can be better extracted from the environment.

FIG. 8 shows a block diagram of a second determination module 800 according to an embodiment of the present disclosure.

As shown in FIG. 8 , the second determination module 800 may include a first determination sub-module 810 and a second determination sub-module 820 .

a first determination sub-module 810, configured to determine a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data for a specified frequency point;

The second determination sub-module 820 is configured to determine the target noise ratio of the specified frequency point based on the correlation function and the position of the main sound source.

According to the technical solutions of the embodiments of the present disclosure, the first determination sub-module is configured to determine the correlation function between the spectrum of the first audio data and the spectrum of the second audio data for a specified frequency point; the second The determining submodule is configured to determine the target noise ratio of the specified frequency point based on the correlation function and the orientation of the main sound source, which improves the estimation accuracy of the noise parameter, so that the signal of the desired sound source can be better extracted from the environment.

According to an embodiment of the present disclosure, the second determination submodule 820 may include:

a first determination unit configured to determine a desired signal component representation and an undesired signal component representation of the real part of the correlation function;

a second determination unit configured to determine a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function;

a third determination unit configured to represent desired and undesired signal components based on the real part of the correlation function, the desired and undesired signal component representations of the imaginary part of the correlation function, and the The orientation of the main sound source, and the target noise ratio of the specified frequency point is determined.

According to the technical solutions of the embodiments of the present disclosure, the first determination unit is configured to determine the desired signal component representation and the undesired signal component representation of the real part of the correlation function; the second determination unit is configured to determine the correlation a desired signal component representation and an undesired signal component representation of the imaginary part of the function; a third determination unit configured to represent the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the imaginary signal component representation of the correlation function The expected and undesired signal component representations of the part, and the orientation of the main sound source, determine the target-to-noise ratio of the specified frequency point, improve the estimation accuracy of noise parameters, and thus can better extract the desired signal from the environment. source signal.

According to an embodiment of the present disclosure, the second determination module further includes a first acquisition sub-module configured to acquire the target-to-noise ratio of each frequency point in the frequency spectrum.

According to the technical solutions of the embodiments of the present disclosure, the first obtaining sub-module is configured to obtain the target noise ratio of each frequency point in the frequency spectrum, so that the signal of the desired sound source can be better extracted from the environment.

FIG. 9 shows a block diagram of a second acquisition module 900 according to an embodiment of the present disclosure.

As shown in FIG. 9 , the second obtaining module 900 may include a filtering sub-module 910 , a second obtaining sub-module 920 and a third obtaining sub-module 930 .

a filtering sub-module 910, configured to filter the frequency spectrum of the first audio data and the frequency spectrum of the second audio data based on the target-to-noise ratio;

A second obtaining sub-module 920, configured to obtain a time-domain representation of the first audio data as third audio data from a frequency spectrum of the filtered first audio data, and/or from a frequency spectrum of the filtered second audio data obtaining a time domain representation of the second audio data as the fourth audio data;

The third obtaining sub-module 930 is configured to obtain target audio data based on the third audio data and/or the fourth audio data.

According to the technical solutions of the embodiments of the present disclosure, the filtering sub-module is configured to filter the frequency spectrum of the first audio data and the frequency spectrum of the second audio data based on the target noise ratio; the second obtaining sub-module , configured to obtain a time-domain representation of the first audio data from the spectrum of the filtered first audio data as third audio data, and/or to obtain the second audio data from the spectrum of the filtered second audio data The time domain representation of the data is used as the fourth audio data; the third acquisition sub-module is configured to acquire the target audio data based on the third audio data and/or the fourth audio data, which improves the estimation accuracy of the noise parameter , so that the signal of the desired sound source can be better extracted from the environment.

FIG. 10 shows a block diagram of a filtering sub-module 1000 according to an embodiment of the present disclosure.

As shown in FIG. 10 , the filtering sub-module 1000 may include a first obtaining unit 1010 , a second obtaining unit 1020 , an updating unit 1030 and a filtering unit 1040 .

The first obtaining unit 1010 is configured to obtain the azimuth range of the desired sound source;

The second obtaining unit 1020 is configured to obtain a judgment result of whether the current audio data is desired audio data or undesired audio data based on the position of the audio source of the current audio data and the azimuth range of the desired audio source, where the current audio data is the first audio data or the second audio data;

an update unit 1030, configured to update the spatial filter coefficient based on the judgment result, the current audio data and the target noise ratio;

The filtering unit 1040 is configured to filter the current audio data through the updated spatial filter coefficients.

According to the technical solutions of the embodiments of the present disclosure, the first acquisition unit is configured to acquire the azimuth range of the desired sound source; the second acquisition unit is configured to be based on the audio source position of the current audio data and the azimuth range of the desired audio source , obtain the judgment result that the current audio data is expected audio data or undesired audio data; the updating unit is configured to update the spatial filter coefficient based on the judgment result, the current audio data and the target noise ratio; the filtering unit, It is configured to filter the current audio data through the updated spatial filter coefficients, which improves the estimation accuracy of the noise parameter, so that the signal of the desired audio source can be better extracted from the environment.

According to an embodiment of the present disclosure, the update unit includes:

A first update subunit, configured to update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio when the current audio data is the desired audio data;

a second update subunit, configured to update the noise covariance matrix of the current audio data and the global covariance matrix;

A third update subunit is configured to update spatial filter coefficients based on the noise covariance matrix and the global covariance matrix.

According to the technical solutions of the embodiments of the present disclosure, the first update subunit is configured to update the current audio based on the current audio data and the target noise ratio when the current audio data is desired audio data The global covariance matrix of the data; the second update subunit is configured to update the current audio data based on the current audio data and the target noise ratio when the current audio data is undesired audio data. a noise covariance matrix and the global covariance matrix; a third update subunit is configured to update the spatial filter coefficients based on the noise covariance matrix and the global covariance matrix, which improves the estimation accuracy of noise parameters, Thereby, the signal of the desired sound source can be better extracted from the environment.

FIG. 11 shows a flowchart of an audio processing method according to another embodiment of the present disclosure.

As shown in FIG. 11, the audio processing method includes the following steps S1110-S1140:

In step S1110, N pieces of audio data corresponding to each other collected by N microphones are acquired, N≥3;

In step S1120, one or more pairs of audio data are determined based on the N pieces of audio data;

In step S1130, each audio data pair is processed to obtain filtered audio data corresponding to the audio data pair. Specifically, for each audio data pair, determine the main sound source position of the audio data corresponding to the audio data pair, where the main sound source position includes the probability that the positions of the multiple audio sources located for the audio data pair meet a preset probability The sound source orientation of the condition; based on the audio data corresponding to the audio data pair and the main sound source orientation, a target noise ratio is determined, and the target noise ratio represents the expected signal energy and the The ratio of expected signal energy; based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data;

In step S1140, target audio data is determined based on the filtered audio data obtained from the one or more pairs of audio data.

According to the technical solutions of the embodiments of the present disclosure, N pieces of audio data corresponding to each other collected by N microphones are acquired, N≥3; one or more pairs of audio data are determined based on the N pieces of audio data; for each audio data Yes, determine the position of the main sound source of the audio data corresponding to the audio data pair, and the main sound source position includes the position of the audio source whose probability meets the preset condition among the plurality of sound source positions located for the audio data pair; based on the A target noise ratio is determined for the audio data corresponding to the audio data pair and the position of the main sound source, and the target noise ratio represents the ratio of the expected signal energy to the undesired signal energy of the audio data corresponding to the audio data pair; the target-to-noise ratio, filtering the audio data corresponding to the audio data pair to obtain filtered audio data; determining target audio data based on the filtered audio data obtained from the one or more audio data pairs , which improves the estimation accuracy of noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

The difference between the embodiment shown in FIG. 11 and the embodiments described above with reference to FIGS. 1 to 6 is that in the embodiment shown in FIG. 11 , the number of microphones is three or more. The method of this embodiment of the present disclosure determines one or more audio data pairs based on the N pieces of audio data through step S1120, and transforms the problem into the situation described in FIG. 1 to FIG. In step S1130, the audio in the audio data pair is filtered based on the target noise ratio, and in step S1140, the target audio data is determined according to the filtered result.

Wherein, in step S1120, according to an embodiment of the present disclosure, the determining one or more pairs of audio data based on the plurality of audio data includes:

determining the one or more pairs of audio data according to the positional relationship of the N microphones; or

Any two audio data in the plurality of audio data are formed into an audio data pair.

According to the technical solutions of the embodiments of the present disclosure, the determining one or more pairs of audio data based on the plurality of audio data includes: determining the one or more pairs of audio data according to the positional relationship of the N microphones; Alternatively, any two audio data in the plurality of audio data are formed into an audio data pair, which improves the estimation accuracy of the noise parameter, so that the signal of the desired audio source can be better extracted from the environment.

For example, determining the one or more pairs of audio data according to the positional relationship of the N microphones includes:

If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair.

According to the technical solutions of the embodiments of the present disclosure, determining the one or more pairs of audio data according to the positional relationship of the N microphones includes: if the N microphones are arranged in a linear manner, selecting a distance from the N microphones The audio data corresponding to two microphones whose geometric center points are closest to the array formed by N microphones form an audio data pair, which improves the estimation accuracy of noise parameters, so that the signal of the desired sound source can be better extracted from the environment.

For example, in the embodiment shown in FIG. 12A , an even number of microphones (M ₁ , M ₂ , M ₃ , M ₄ ) are arranged in a linear manner, and the closest point to the geometric center point of the array formed by the N microphones can be selected The two microphones (M ₂ -M ₃ ) of are identified as an audio data pair.

For another example, in the embodiment shown in FIG. 12B , the three microphones M ₁ , M ₂ , and M ₃ are arranged in a linear manner, and M ₁ − which is closest to the geometric center point of the array formed by the N microphones can be selected. One of M ₂ or M ₂ -M ₃ is determined as an audio data pair.

According to the embodiment of the present disclosure, when the N microphones are arranged in a non-linear manner, two microphones close to the center position may also be selected to form an audio data pair.

According to the embodiment of the present disclosure, any two audio data in the plurality of audio data may also be formed into an audio data pair. For example, in the embodiment shown in FIG. 12C , the three microphones M ₁ , M ₂ , M ₃ are arranged in a center-symmetric manner, and M ₁ -M ₂ , M ₂ -M ₃ , M ₁ -M ₃ can be selected The audio data pairs are respectively determined, three audio data pairs are determined in total, and the three audio data pairs are processed respectively in step S1130.

According to an embodiment of the present disclosure, the determining of target audio data based on the filtered audio data obtained from the one or more pairs of audio data includes:

obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or

Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data.

According to the technical solutions of the embodiments of the present disclosure, determining target audio data based on the filtered audio data obtained from the one or more pairs of audio data includes: Weighted summation is performed on the obtained filtered audio data to obtain target audio data; or, in the filtered audio data obtained from the one or more pairs of audio data, a selection corresponding to a preset position is selected. The filtered audio data corresponding to the microphone is used as the target audio data, which improves the estimation accuracy of the noise parameters, so that the signal of the desired audio source can be better extracted from the environment.

According to an embodiment of the present disclosure, after processing one or more pairs of audio data, filtered audio data of at least two microphones are obtained, and the filtered audio data corresponding to one microphone may be selected as the target audio data for output, or, The filtered audio data corresponding to the two or more microphones are weighted and summed to obtain target audio data. For example, in the embodiment shown in FIG. 12A , the filtered audio data corresponding to the microphone M ₂ or the microphone M ₃ can be selected as the target audio data output, or the filtered audio data corresponding to the microphone M ₂ or the microphone M ₃ can be output. Audio data weighting processing, the processing result is output as the target audio data; in the embodiment shown in Figure 12B, the filtered audio data corresponding to the microphone M ₂ can be selected as the target audio data output; as shown in Figure 12C In the embodiment, the filtered audio data corresponding to the microphone M ₁ , the microphone M ₂ and the microphone M ₃ may be weighted and summed, and the processing result may be output as the target audio data.

FIG. 13 shows a block diagram of an audio processing apparatus according to another embodiment of the present disclosure. Wherein, the apparatus may be realized by software, hardware or a combination of the two to become part or all of the electronic device.

As shown in FIG. 13 , the audio processing apparatus 1300 includes a fourth obtaining module 1310 , a third determining module 1320 , a filtering module 1330 and a fourth determining module 1340 .

The fourth acquisition module 1310 is configured to acquire N pieces of audio data corresponding to each other collected by the N microphones, N≥3;

a third determining module 1320, configured to determine one or more pairs of audio data based on the N pieces of audio data;

The filtering module 1330 is configured to, for each pair of audio data, determine the position of the main sound source of the audio data corresponding to the pair of audio data, where the position of the main sound source includes the positions of the multiple sound sources located for the pair of audio data. The sound source orientation whose probability meets the preset conditions; the target noise ratio is determined based on the audio data corresponding to the audio data pair and the main sound source orientation, and the target noise ratio represents the respective expectations of the audio data pair corresponding to the audio data. The ratio of signal energy to undesired signal energy; based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data;

The fourth determination module 1340 is configured to determine target audio data based on the filtered audio data obtained from the one or more pairs of audio data.

According to the technical solutions of the embodiments of the present disclosure, the fourth obtaining module is configured to obtain N pieces of audio data corresponding to each other collected by the N microphones, N≥3; the third determining module is configured to obtain N pieces of audio data corresponding to each other collected by N microphones; The audio data determines one or more pairs of audio data; the filtering module is configured to, for each pair of audio data, determine the position of the main sound source of the audio data corresponding to the pair of audio data, and the position of the main sound source includes A sound source orientation whose probability meets a preset condition among the multiple sound source orientations located by the data pair; a target noise ratio is determined based on the audio data corresponding to the audio data pair and the main sound source orientation, and the target noise ratio represents the audio The ratio of the respective expected signal energy and undesired signal energy of the audio data corresponding to the data pair; based on the target noise ratio, filtering the audio data corresponding to the audio data pair to obtain filtered audio data; fourth; A determination module, configured to determine target audio data based on the filtered audio data obtained from the one or more audio data pairs, improves the estimation accuracy of noise parameters, so that expectations can be better extracted from the environment source signal.

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

According to an embodiment of the present disclosure, the third determining module includes:

a third determination submodule, configured to determine the one or more audio data pairs according to the positional relationship of the N microphones; or

The fourth determining submodule is configured to form an audio data pair from any two audio data in the plurality of audio data.

According to the technical solutions of the embodiments of the present disclosure, the third determination sub-module is configured to determine the one or more audio data pairs according to the positional relationship of the N microphones; or, the fourth determination sub-module is configured to In order to form an audio data pair from any two audio data in the plurality of audio data, the estimation accuracy of the noise parameter is improved, so that the signal of the desired audio source can be better extracted from the environment.

According to an embodiment of the present disclosure, the third determination submodule is configured to:

If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair.

According to the technical solutions of the embodiments of the present disclosure, the third determination submodule is configured to select two closest to the geometric center point of the array formed by the N microphones if the N microphones are arranged in a linear manner The audio data corresponding to the microphones are composed of audio data pairs, which improves the estimation accuracy of the noise parameters, so that the signal of the desired audio source can be better extracted from the environment.

According to an embodiment of the present disclosure, the fourth determination module is configured to:

obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or

Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data.

According to the technical solutions of the embodiments of the present disclosure, the fourth determining module is configured to: obtain target audio data by weighting and summing the filtered audio data obtained from the one or more pairs of audio data Or, in the described filtered audio data obtained from the one or more audio data pairs, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data, which improves the noise parameter. Estimate the accuracy, so that the signal of the desired sound source can be better extracted from the environment.

An embodiment of the present disclosure further provides an electronic device, and FIG. 14 shows a block diagram of the electronic device according to an embodiment of the present disclosure.

As shown in FIG. 14 , the electronic device 1400 includes a memory 1401 and a processor 1402 , wherein the memory 1401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1402 to implement a method according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the memory 1401 is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1402 to implement the following steps:

acquiring first audio data collected by the first microphone and second audio data corresponding to the first audio data collected by the second microphone;

Determine the main sound source position of the first audio data and the second audio data. Set the position of the sound source of the condition;

A target noise ratio is determined based on the first audio data, the second audio data, and the main audio source orientation, the target noise ratio representing the desired signal energy and the undesired signal energy of each of the first audio data and the second audio data the ratio of signal energy; and

Based on the target noise ratio, filtering the first audio data and the second audio data and obtaining target audio data based on the filtered first audio data and the second audio data.

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

According to an embodiment of the present disclosure, the one or more computer instructions are executed by the processor 1402 to further implement:

Before determining the target noise ratio, the frequency spectrum of the first audio data and the frequency spectrum of the second audio data are acquired.

According to an embodiment of the present disclosure, the determining a target noise ratio based on the first audio data, the second audio data and the main audio source azimuth includes:

For a specified frequency point, determine a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data;

A target-to-noise ratio for the specified frequency point is determined based on the correlation function and the orientation of the main sound source.

According to an embodiment of the present disclosure, the determining the target noise ratio of the specified frequency point based on the correlation function and the main sound source orientation includes:

determining a desired signal component representation and an undesired signal component representation of the real part of the correlation function;

determining a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function;

The assignment is determined based on the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function, and the main sound source orientation The target-to-noise ratio of the frequency point.

According to an embodiment of the present disclosure, the determining a target noise ratio based on the first audio data, the second audio data, and the orientation of the main audio source further includes acquiring the target noise ratio of each frequency point in the frequency spectrum.

According to an embodiment of the present disclosure, the filtering of the first audio data and the second audio data based on the target noise ratio and obtaining the target audio data based on the filtered first audio data and the second audio data includes:

based on the target noise ratio, filtering the frequency spectrum of the first audio data and the frequency spectrum of the second audio data;

The time domain representation of the first audio data is obtained from the spectrum of the filtered first audio data as the third audio data, and the time domain representation of the second audio data is obtained from the spectrum of the filtered second audio data as the third audio data. four audio data;

Target audio data is acquired based on the third audio data and the fourth audio data.

According to an embodiment of the present disclosure, the filtering of the first audio data and the second audio data based on the target noise ratio includes:

Obtain the azimuth range of the desired sound source;

Based on the azimuth of the main sound source and the azimuth range of the desired sound source, obtain a judgment result that the current audio data is desired audio data or undesired audio data;

updating spatial filter coefficients based on the judgment result, the current audio data, and the target noise ratio;

The current audio data is filtered by the updated spatial filter coefficients.

According to an embodiment of the present disclosure, the updating of the spatial filter coefficients based on the judgment result, the current audio data, and the target noise ratio includes:

When the current audio data is desired audio data, update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio;

When the current audio data is undesired audio data, update the noise covariance matrix and the global covariance matrix of the current audio data based on the current audio data and the target noise ratio;

Spatial filter coefficients are updated based on the noise covariance matrix and the global covariance matrix.

According to an embodiment of the present disclosure, the memory 1401 is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1402 to implement the following steps:

Obtain N pieces of audio data corresponding to each other collected by N microphones, N≥3;

determining one or more pairs of audio data based on the N audio data;

For each pair of audio data, determine the location of the main sound source of the audio data corresponding to the pair of audio data, where the location of the main sound source includes a sound source whose probability meets a preset condition among the locations of multiple audio sources located for the pair of audio data Orientation; determine a target noise ratio based on the audio data corresponding to the audio data pair and the main audio source orientation, where the target noise ratio represents the respective expected signal energy and undesired signal energy of the audio data corresponding to the audio data pair based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data;

Target audio data is determined based on the filtered audio data obtained from the one or more pairs of audio data.

According to an embodiment of the present disclosure, the sound source location whose probability meets the preset condition includes the sound source location with the highest probability.

According to an embodiment of the present disclosure, the determining one or more pairs of audio data based on the plurality of audio data includes:

determining the one or more pairs of audio data according to the positional relationship of the N microphones; or

Any two audio data in the plurality of audio data are formed into an audio data pair.

According to an embodiment of the present disclosure, the determining of the one or more pairs of audio data according to the positional relationship of the N microphones includes:

If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair.

According to an embodiment of the present disclosure, the determining of target audio data based on the filtered audio data obtained from the one or more pairs of audio data includes:

obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or

Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data.

FIG. 15 shows a schematic structural diagram of a computer system suitable for implementing the audio processing method according to an embodiment of the present disclosure.

As shown in FIG. 15 , the computer system 1500 includes a processing unit 1501 that can perform the above-described implementation according to a program stored in a read only memory (ROM) 1502 or a program loaded from a storage section 1508 into a random access memory (RAM) 1503 various treatments in the example. In the RAM 1503, various programs and data necessary for the operation of the system 1500 are also stored. The processing unit 1501 , the ROM 1502 , and the RAM 1503 are connected to each other through a bus 1504 . An input/output (I/O) interface 1505 is also connected to bus 1504 .

The following components are connected to the I/O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc. ; and a communication section 1509 including a network interface card such as a LAN card, a modem, and the like. The communication section 1509 performs communication processing via a network such as the Internet. Drivers 1510 are also connected to I/O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1510 as needed so that a computer program read therefrom is installed into the storage section 1508 as needed. The processing unit 1501 may be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, and an NPU.

In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a readable medium thereof, the computer program containing program code for performing the above-described method. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 1509, and/or installed from the removable medium 1511.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The units or modules involved in the embodiments of the present disclosure may be implemented in a software manner, or may be implemented in a programmable hardware manner. The described units or modules may also be provided in the processor, and the names of these units or modules do not constitute a limitation on the units or modules themselves in certain circumstances.

As another aspect, the present disclosure also provides a computer-readable storage medium, and the computer-readable storage medium may be a computer-readable storage medium included in the electronic device or computer system in the above-mentioned embodiments; it may also exist independently , a computer-readable storage medium that does not fit into a device. The computer-readable storage medium stores one or more programs used by one or more processors to perform the methods described in the present disclosure.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, and should also cover the above-mentioned technical features without departing from the inventive concept. Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.

Claims (30)

1. an audio processing method, is characterized in that, comprises: acquiring first audio data collected by the first microphone and second audio data corresponding to the first audio data collected by the second microphone; Determine the main sound source position of the first audio data and the second audio data. Set the position of the sound source of the condition; A target noise ratio is determined based on the first audio data, the second audio data, and the main audio source orientation, the target noise ratio representing the desired signal energy and the undesired signal energy of each of the first audio data and the second audio data the ratio of signal energy; and Based on the target noise ratio, filtering the first audio data and/or the second audio data and obtaining target audio data based on the filtered first audio data and/or the second audio data. 2. The method according to claim 1, wherein the method further comprises: Before determining the target noise ratio, the frequency spectrum of the first audio data and the frequency spectrum of the second audio data are acquired. 3. The method according to claim 2, wherein the determining a target noise ratio based on the first audio data, the second audio data and the main audio source azimuth comprises: For a specified frequency point, determine a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data; A target-to-noise ratio for the specified frequency point is determined based on the correlation function and the orientation of the main sound source. 4. The method according to claim 3, wherein the determining the target noise ratio of the specified frequency point based on the correlation function and the main sound source azimuth comprises: determining a desired signal component representation and an undesired signal component representation of the real part of the correlation function; determining a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function; The assignment is determined based on the desired signal component representation and the undesired signal component representation of the real part of the correlation function, the desired signal component representation and the undesired signal component representation of the imaginary part of the correlation function, and the main sound source orientation The target-to-noise ratio of the frequency point. 5. The method according to claim 3, wherein the determining a target noise ratio based on the first audio data, the second audio data and the main audio source position further comprises: Obtain the target-to-noise ratio of each frequency point in the spectrum. 6. The method according to claim 2, wherein the first audio data and/or the second audio data are filtered based on the target noise ratio and based on the filtered first audio data and / or the second audio data to obtain target audio data, including: filtering the frequency spectrum of the first audio data and/or the frequency spectrum of the second audio data based on the target noise ratio; Obtain a time-domain representation of the first audio data from the spectrum of the filtered first audio data as third audio data and/or obtain a time-domain representation of the second audio data from the spectrum of the filtered second audio data as fourth audio data; Target audio data is acquired based on the third audio data and/or the fourth audio data. 7. The method according to claim 6, wherein the filtering the first audio data and the second audio data based on the target noise ratio comprises: Obtain the azimuth range of the desired sound source; Based on the azimuth of the main audio source and the azimuth range of the desired audio source, a judgment result of whether the current audio data is desired audio data or undesired audio data is obtained, and the current audio data is the first audio data or the second audio data; updating spatial filter coefficients based on the judgment result, the current audio data, and the target noise ratio; The current audio data is filtered by the updated spatial filter coefficients. 8. The method according to claim 7, wherein the updating of the spatial filter coefficients based on the judgment result, the current audio data and the target noise ratio comprises: When the current audio data is desired audio data, update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio; When the current audio data is undesired audio data, update the noise covariance matrix and the global covariance matrix of the current audio data based on the current audio data and the target noise ratio; Spatial filter coefficients are updated based on the noise covariance matrix and the global covariance matrix. 9 . The method according to claim 1 , wherein the sound source orientation whose probability meets a preset condition includes the sound source orientation with the highest probability. 10 . 10. An audio processing method, comprising: Obtain N pieces of audio data corresponding to each other collected by N microphones, N≥3; determining one or more pairs of audio data based on the N audio data; For each pair of audio data, determine the location of the main sound source of the audio data corresponding to the pair of audio data, where the location of the main sound source includes a sound source whose probability meets a preset condition among the locations of multiple audio sources located for the pair of audio data Orientation; determine a target noise ratio based on the audio data corresponding to the audio data pair and the main audio source orientation, where the target noise ratio represents the respective expected signal energy and undesired signal energy of the audio data corresponding to the audio data pair based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data; Target audio data is determined based on the filtered audio data obtained from the one or more pairs of audio data. 11. The method of claim 10, wherein the determining one or more pairs of audio data based on the plurality of audio data comprises: determining the one or more pairs of audio data according to the positional relationship of the N microphones; or Any two audio data in the plurality of audio data are formed into an audio data pair. 12. The method according to claim 11, wherein the determining the one or more pairs of audio data according to the positional relationship of the N microphones comprises: If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair. 13. The method of claim 10, wherein the determining of target audio data based on the filtered audio data obtained from the one or more pairs of audio data comprises: obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data. 14 . The method according to claim 10 , wherein the sound source location whose probability meets a preset condition includes the sound source location with the highest probability. 15 . 15. An audio processing device, comprising: a first acquisition module, configured to acquire first audio data collected by a first microphone and second audio data corresponding to the first audio data collected by a second microphone; The first determining module is configured to determine the main sound source orientation of the first audio data and the second audio data, and the main sound source orientation includes the location of the first audio data and the second audio data. The position of the sound source whose probability meets the preset conditions among the positions of the multiple sound sources; A second determination module configured to determine a target noise ratio based on the first audio data, the second audio data and the main audio source orientation, the target noise ratio representing the first audio data and the second audio data the respective ratios of desired signal energy to undesired signal energy; and A second obtaining module, configured to filter the first audio data and/or the second audio data based on the target noise ratio, and obtain target audio based on the filtered first audio data and/or the second audio data data. 16. The apparatus of claim 15, wherein the apparatus further comprises: The third acquisition module is configured to acquire the frequency spectrum of the first audio data and the frequency spectrum of the second audio data before determining the target noise ratio. 17. The apparatus according to claim 16, wherein the second determining module comprises: a first determining submodule, configured to determine, for a specified frequency point, a correlation function between the frequency spectrum of the first audio data and the frequency spectrum of the second audio data; The second determination submodule is configured to determine the target noise ratio of the specified frequency point based on the correlation function and the position of the main sound source. 18. The apparatus according to claim 17, wherein the second determination submodule comprises: a first determination unit configured to determine a desired signal component representation and an undesired signal component representation of the real part of the correlation function; a second determination unit configured to determine a desired signal component representation and an undesired signal component representation of the imaginary part of the correlation function; a third determination unit configured to represent desired and undesired signal components based on the real part of the correlation function, the desired and undesired signal component representations of the imaginary part of the correlation function, and the The orientation of the main sound source, and the target noise ratio of the specified frequency point is determined. 19. The apparatus according to claim 17, wherein the second determining module further comprises: The first obtaining sub-module is configured to obtain the target noise ratio of each frequency point in the frequency spectrum. 20. The apparatus according to claim 16, wherein the second obtaining module comprises: a filtering submodule configured to filter the frequency spectrum of the first audio data and the frequency spectrum of the second audio data based on the target-to-noise ratio; A second acquisition sub-module configured to acquire a time-domain representation of the first audio data from the spectrum of the filtered first audio data as third audio data, and/or to acquire from the spectrum of the filtered second audio data a time domain representation of the second audio data as the fourth audio data; A third obtaining sub-module is configured to obtain target audio data based on the third audio data and/or the fourth audio data. 21. The apparatus according to claim 20, wherein the filtering sub-module comprises: a first obtaining unit, configured to obtain the azimuth range of the desired sound source; a second obtaining unit, configured to obtain a judgment result of whether the current audio data is expected audio data or undesired audio data based on the azimuth of the main sound source and the azimuth range of the desired sound source, where the current audio data is the first audio data audio data or the second audio data; an update unit configured to update the spatial filter coefficient based on the judgment result, the current audio data and the target noise ratio; A filtering unit configured to filter the current audio data through the updated spatial filter coefficients. 22. The apparatus according to claim 21, wherein the updating unit comprises: A first update subunit, configured to update the global covariance matrix of the current audio data based on the current audio data and the target noise ratio when the current audio data is the desired audio data; a second update subunit, configured to update the noise covariance matrix of the current audio data and the global covariance matrix; A third update subunit is configured to update spatial filter coefficients based on the noise covariance matrix and the global covariance matrix. 23 . The device according to claim 15 , wherein the sound source location with the probability meeting a preset condition includes the sound source location with the highest probability. 24 . 24. An audio processing device, comprising: a fourth acquisition module, configured to acquire N pieces of audio data corresponding to each other collected by the N microphones, N≥3; a third determining module configured to determine one or more pairs of audio data based on the N pieces of audio data; The filtering module is configured to, for each pair of audio data, determine the position of the main sound source of the audio data corresponding to the pair of audio data, and the position of the main sound source includes a probability among the positions of the multiple sound sources located for the pair of audio data The sound source orientation that meets the preset conditions; the target noise ratio is determined based on the audio data corresponding to the audio data pair and the main sound source orientation, and the target noise ratio represents the respective expected signals of the audio data pair corresponding to the audio data. a ratio of energy to undesired signal energy; based on the target-to-noise ratio, filtering the audio data to the corresponding audio data to obtain filtered audio data; A fourth determination module configured to determine target audio data based on the filtered audio data obtained from the one or more pairs of audio data. 25. The apparatus according to claim 24, wherein the third determining module comprises: a third determination submodule, configured to determine the one or more audio data pairs according to the positional relationship of the N microphones; or The fourth determining submodule is configured to form an audio data pair from any two audio data in the plurality of audio data. 26. The apparatus according to claim 25, wherein the third determination sub-module is configured to: If the N microphones are arranged in a linear manner, audio data corresponding to two microphones closest to the geometric center point of the array formed by the N microphones are selected to form an audio data pair. 27. The apparatus of claim 24, wherein the fourth determining module is configured to: obtaining target audio data by weighted summing said filtered audio data obtained from said one or more pairs of audio data; or Among the filtered audio data obtained from the one or more pairs of audio data, the filtered audio data corresponding to the microphone at the preset position is selected as the target audio data. 28 . The device according to claim 24 , wherein, the sound source position whose probability meets a preset condition includes the sound source position with the highest probability. 29 . 29. An electronic device, comprising a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement The method steps of any one of claims 1-14. 30. A readable storage medium on which computer instructions are stored, characterized in that, when the computer instructions are executed by a processor, the method steps of any one of claims 1-14 are implemented.

Images