TWI861755B - Microphone array system and noise filtering method for microphone array - Google Patents

Abstract

A noise filtering method for microphone array, performed by a processing device, includes: setting an desired range according to a user command, setting a plurality of virtual ranges based on the desired range, wherein a distance between a center of each one of the virtual ranges and a center of the desired range is not greater than a default distance, and controlling an output audio corresponding to each one of the desired range and the virtual ranges according to audio energy corresponding to the desired range and the virtual ranges, respectively.

Description

Microphone array system and noise filtering method of microphone array

The present invention relates to a microphone array system and a noise filtering method of the microphone array.

With the popularity of online meetings, the market for online meeting equipment is growing, and one of the most important equipment is the microphone. The sound pickup effect of the microphone affects the quality of the meeting. Furthermore, when there are multiple people around a microphone at the same time, the voices of other non-speakers will also be transmitted to the remote participants, causing serious interference to the output audio of the corresponding speaker.

In view of the above, the present invention provides a microphone array system and a noise filtering method of a microphone array.

According to an embodiment of the present invention, a noise filtering method for a microphone array is executed by a processing device, and includes: setting a desired range according to a user instruction; setting a plurality of virtual ranges based on the desired range, wherein the distance between the center of each of the plurality of virtual ranges and the center of the desired range is not greater than a preset distance; and controlling the output audio corresponding to the desired range and each of the plurality of virtual ranges according to the energy of the audio signals corresponding to the desired range and the plurality of virtual ranges.

A microphone array system according to an embodiment of the present invention includes a microphone array and a processing device. The microphone array is used to receive audio signals of a desired range and multiple virtual ranges. The processing device is connected to the microphone array and is used to perform: setting the desired range according to a user instruction; setting the multiple virtual ranges based on the desired range, wherein the distance between the center of each of the multiple virtual ranges and the center of the desired range is not greater than a preset distance; and controlling the output audio signals corresponding to the desired range and each of the multiple virtual ranges according to the energy of the audio signals corresponding to the desired range and the multiple virtual ranges.

In summary, the microphone array system and the noise filtering method of the microphone array according to one or more embodiments of the present invention can provide users with a more flexible sound collection method.

The above description of the disclosed content and the following description of the implementation methods are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the scope of the patent application of the present invention.

The following detailed description of the features and advantages of the present invention is provided in the implementation mode, and the content is sufficient to enable any person skilled in the relevant art to understand the technical content of the present invention and implement it accordingly. Moreover, according to the content disclosed in this specification, the scope of the patent application and the drawings, any person skilled in the relevant art can easily understand the relevant purposes and advantages of the present invention. The following embodiments are to further explain the viewpoints of the present invention in detail, but are not to limit the scope of the present invention by any viewpoint.

Please refer to FIG. 1, which is a block diagram of a microphone array system according to an embodiment of the present invention. The microphone array system 1 includes a microphone array 11 and a processing device 12. The processing device 12 is connected to the microphone array 11 in a wired or wireless manner. The microphone array 11 is used to receive audio signals from its surroundings, and the processing device 12 is used to perform noise filtering on the audio signals received by the microphone array 11, and can transmit the filtered audio signals to another processing device at a remote end. The processing device 12 may include one or more processors, such as a central processing unit, a graphics processor, a microcontroller, a programmable logic controller, or other processors with signal processing functions.

Please refer to FIG. 1 and FIG. 2 , where FIG. 2 is a schematic diagram showing the application of the microphone array to an online conference. In FIG. 2 , the processing device 12 can output the filtered audio to the remote processing device that is jointly conducting the online conference. The microphone array 11 can be a ceiling-mounted microphone array installed on the ceiling, but the present invention is not limited thereto. Taking FIG. 2 as an example, the microphone array 11 and the processing device 12 can be jointly arranged in a housing to be jointly installed on the ceiling, and the processing device 12 can also be separately arranged from the microphone array 11 and communicate wirelessly.

Please refer to FIG. 1 , FIG. 3 , FIG. 4A and FIG. 4B , wherein FIG. 3 is a flow chart of a noise filtering method for a microphone array according to an embodiment of the present invention, and FIG. 4A and FIG. 4B are schematic diagrams showing a desired range and a virtual range. The noise filtering method for a microphone array shown in FIG. 3 can be executed by the microphone array system 1 of FIG. 1 , especially by the processing device 12 of the microphone array system 1 . As shown in FIG. 3 , the noise filtering method of the microphone array includes: step S101: setting a desired range according to a user instruction; step S103: setting a plurality of virtual ranges based on the desired range, wherein the distance between the center of each virtual range and the center of the desired range is not greater than a preset distance; and step S105: controlling the output audio corresponding to the desired range and each of the virtual ranges according to the audio energy corresponding to the desired range and the virtual ranges.

In step S101, the processing device 12 sets an expected range according to a user instruction, wherein the expected range indicates a sound pickup range specified by the user. Taking an online conference as an example, the expected range may be a range centered on the microphone array 11. Taking FIG. 4A as an example, the expected range A0 set according to the user instruction may be circular, and the projection range of the microphone array 11 on the ground may at least partially fall within the projection range of the expected range A0 on the ground. Similarly, taking FIG. 4B as an example, the expected range B0 set according to the user instruction may be elliptical, and the projection range of the microphone array 11 on the ground may at least partially fall within the projection range of the expected range B0 on the ground. The number of the expected ranges may be one or more, and their shapes and sizes may be set according to the needs of the user, and the present invention is not limited thereto.

In step S103, the processing device 12 sets a plurality of virtual ranges around the desired range, and the distance between the center of each virtual range and the center of the desired range is not greater than a preset distance, and the virtual range may overlap with or be adjacent to the desired range. For example, assuming that the desired range is circular, the distance between the center of the desired range and the center of each virtual range may be equal to the diameter length of the desired range. In other words, the diameter length of the desired range may be used as a preset distance, but the present invention is not limited thereto.

Taking FIG. 4A and FIG. 4B as examples, there are multiple virtual ranges A1 to A6 around the desired range A0, and there are multiple virtual ranges B1 to B2 around the desired range B0. The number of virtual ranges may be at least three and not more than six, but the present invention does not limit the number of virtual ranges. In addition, the shapes and sizes of the virtual ranges may be the same or different from each other, and the shapes and sizes of the virtual ranges may be the same or different from the desired ranges, and the present invention does not limit them.

In step S105, the processing device 12 controls the output audio corresponding to the desired range and each virtual range according to the audio energy corresponding to the desired range and the virtual range. The microphone array 11 is used to receive the audio of the desired range and the virtual range, wherein the audio can be an audio signal obtained through beamforming, and the calculation method of beamforming can include delay and sum, minimum variance distortionless response (MVDR) and general sidelobe canceller (GSC). The processing device 12 receives the audio from the microphone array 11 and controls the audio output to the remote processing device. For example, the processing device 12 may increase the energy of the output audio signal in the desired range and/or suppress the energy of the output audio signal in the virtual range.

Please refer to FIG. 1 and FIG. 5 together, wherein FIG. 5 is a flow chart of a method for controlling output audio according to an embodiment of the present invention. The process shown in FIG. 5 can be executed after step S103 of FIG. 2, and can be regarded as a detailed flow chart of an embodiment of step S105 of FIG. 2. As shown in FIG. 5, the method for controlling output audio includes: step S201: determining whether the first energy value of the expected range is greater than a preset value; if the determination result of step S201 is "no", executing step S203: not outputting the output audio of the expected range and the virtual ranges; if the determination result of step S201 is "yes", for each virtual range Execution: Execute step S205: determine whether the second energy value of the virtual range is greater than a preset value; if the determination result of step S205 is "no", execute step S207: output the output audio signal of the expected range; and if the determination result of step S205 is "yes", execute step S209: reduce the audio energy of the output audio signal of the virtual range.

The processing device 12 can calculate the energy sum of the received audio signal energy in the desired range and the received audio signal energy in all virtual ranges, and divide the received audio signal energy in the desired range by the energy sum to obtain the first energy value. In step S201, the processing device 12 can determine whether the first energy value is greater than a preset value. If the processing device 12 determines that the first energy value is not greater than the preset value, it indicates that there may be no sound in the desired range. Therefore, the processing device 12 can execute step S203 and does not output any audio to the remote processing device.

If the processing device 12 determines that the first energy value is greater than the preset value, the processing device 12 further determines one by one (or simultaneously) whether the second energy value of the received audio signal of each virtual range is greater than the preset value, wherein the processing device 12 divides the energy of the received audio signal corresponding to each of the virtual ranges by the energy sum value to obtain the second energy value of each virtual range. Taking a virtual range as an example, if the processing device 12 determines that the second energy value of the virtual range is not greater than the preset value, it means that there is sound in the expected range but no sound in the virtual range. Therefore, the processing device 12 may execute step S207 to output the audio of the desired range to the remote processing device, and may not output the audio of the virtual range to the remote processing device. If the processing device 12 determines that the second energy value of the virtual range is greater than the preset value, it means that multiple sounds exist at the same time. Therefore, the processing device 12 may execute step S209 to reduce the energy of the output audio of the virtual range. For example, the processing device 12 may process the sound received in the virtual range as a noise source, and transmit the output audio of the noise-filtered range to the remote processing device.

In other words, in the embodiment of FIG5 , when there are multiple virtual ranges, the processing device 12 may control the corresponding output audio according to the second energy value of each virtual range. Therefore, when the determination result of step S201 is “yes”, the processing device 12 may not output the output audio of some virtual ranges, and reduce the audio energy of the output audio of other virtual ranges; not output the output audio of all virtual ranges; or reduce the audio energy of the output audio of all virtual ranges.

In addition, the processing device 12 can also directly use the energy value of the received audio signal as the first energy value and the second energy value to implement the embodiment of FIG. 5 . Specifically, the processing device 12 can use the value corresponding to the energy of the received audio signal in the expected range as the first energy value, and use the values corresponding to the energy of the received audio signal corresponding to each of the virtual ranges as the second energy value of each virtual range. Therefore, in step S201, the processing device 12 can determine whether the first energy value as the energy value is greater than a preset value; and in step S205, the processing device 12 can determine whether the second energy value as the energy value is greater than a preset value.

It should also be noted that in the implementation method where the first energy value and the second energy value are proportional values, the default value can be a pure value; in the implementation method where the first energy value and the second energy value are simply energy values, the unit of the default value can be decibel, and the values of the default values in these two implementation methods can be different from each other. In addition, the first energy value, the second energy value and the default value can also be implemented in different frequency scales, such as Mel scale.

In addition to the above embodiments, the processing device 12 may also calculate the original energy value and the previous energy value of each of the desired range and each virtual range by the following formula (1) to obtain the first energy value and the second energy value. The energy value in formula (1) may be a proportional value (pure value) or an energy value (decibel). Formula (1) Where is the first energy value or the second energy value; is the default value. , in the embodiment where the energy value is a proportional value, For example, 0.2; is the previous energy value; is the original energy value, wherein the time point corresponding to the previous energy value is earlier than the time point corresponding to the original energy value.

Furthermore, assuming is the first energy value, then the previous energy value and original energy value are the energy values of two consecutive received audio signals within the expected range, where the original energy value is based on the current audio signal energy within the expected range. The previous energy value It is based on the energy of the received audio signal at the previous time point within the expected range; assuming is the second energy value, then and original energy value is the energy value of two consecutive received audio signals in a corresponding virtual range, where the original energy value It is based on the current audio signal energy of the virtual range. The previous energy value It is obtained based on the energy of the received audio signal at the previous time point in the virtual range.

The default value can be adjusted according to the number of virtual ranges and the expected reaction rate. The reaction rate refers to the rate of change from the previous energy value to the original energy value. For example, when the number of virtual ranges is larger, the default value can be adjusted to a smaller value; when the expected reaction rate is faster, the default value can be adjusted to a smaller value. Further, the default value can be regarded as how much of the previous energy value is expected to be retained when calculating the first energy value or the second energy value.

Please refer to FIG. 1 and FIG. 6 together, wherein FIG. 6 is a flow chart of a method for reducing the audio energy of an output audio signal in a virtual range according to an embodiment of the present invention. The process shown in FIG. 6 can be executed after the judgment result of step S205 in FIG. 5 is "yes", and can be regarded as a detailed flow chart of an embodiment of step S209 in FIG. 5. As shown in FIG. 6, the method for reducing the audio energy of an output audio signal in a virtual range includes: step S301: subtracting the second frequency spectrum of the output audio signal in the virtual range from the first frequency spectrum of the output audio signal in the desired range to generate a third frequency spectrum; and step S303: outputting the output audio signal corresponding to the third frequency spectrum.

For a virtual range whose second energy value is greater than a preset value, in step S301, the processing device 12 subtracts the second frequency spectrum of the virtual range from the first frequency spectrum of the expected range to generate a third frequency spectrum, wherein the frequency spectrum may be an amplitude spectrum or a power spectrum. In step S303, the processing device 12 outputs the audio corresponding to the third frequency spectrum to a remote processing device.

Furthermore, if the second energy values corresponding to the plurality of virtual ranges are all greater than a preset value, the processing device 12 may calculate an average spectrum of the second spectrum of the virtual ranges and subtract the average spectrum from the first spectrum to generate a third spectrum.

In addition, the microphone array system may further include an adaptive filter connected to the processing device 12 in a wired or wireless manner. Therefore, the processing device 12 may also control the adaptive filter to reduce the audio energy of the output audio in the virtual range after the determination result of step S205 in FIG. 5 is "yes".

Please refer to FIG. 7(a) to FIG. 7(c), which are example diagrams showing the audio energy corresponding to the desired range and the virtual range. The energy values in FIG. 7(a) to FIG. 7(c) are based on proportional values as examples, and the default value is 0.2.

In FIG. 7( a ), the audio energy corresponding to the desired range is lower than a preset value, and although the audio energy of the virtual range 4 is higher than the preset value, according to one or more embodiments of the present invention, the processing device may not output any output audio to the remote processing device (ie, mute).

In FIG. 7( b ), the audio energy corresponding to the desired range is higher than the preset value, while the audio energy of other virtual ranges is lower than the preset value. According to one or more embodiments of the present invention, the processing device may only output the output audio corresponding to the desired range to the remote processing device.

In FIG. 7( c ), the audio energy corresponding to the desired range and the virtual range 5 are both higher than the preset value. The processing device may first reduce the audio energy corresponding to the virtual range 5 according to one or more of the above embodiments, and then transmit the output audio signal to the remote processing device according to the reduced audio energy corresponding to the virtual range 5 and the audio energy corresponding to the desired range.

In summary, the microphone array system and the noise filtering method of the microphone array according to one or more embodiments of the present invention can provide users with a more flexible sound collection method, and the sound within the undesired range can be filtered out or suppressed.

Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any changes and modifications made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention. Please refer to the attached patent application for the scope of protection defined by the present invention.

1: Microphone array system 11: Microphone array 12: Processing device A0, B0: Expected range A1-A6, B1, B2: Virtual range S101, S103, S105, S201, S203, S205, S207, S209, S301, S303: Steps

FIG. 1 is a block diagram of a microphone array system according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the application of a microphone array to an online conference. FIG. 3 is a flow chart of a noise filtering method of a microphone array according to an embodiment of the present invention. FIG. 4A and FIG. 4B are schematic diagrams showing a desired range and a virtual range. FIG. 5 is a flow chart of a method for controlling output audio according to an embodiment of the present invention. FIG. 6 is a flow chart of a method for reducing audio energy of output audio of a virtual range according to an embodiment of the present invention. FIG. 7(a) to FIG. 7(c) are example diagrams showing audio energy corresponding to the desired range and the virtual range, respectively.

S101, S103, S105: Steps

Claims (10)

A noise filtering method for a microphone array is performed by a processing device, the method comprising: setting a desired range according to a user instruction; setting a plurality of virtual ranges based on the desired range, wherein the distance between the center of each of the virtual ranges and the center of the desired range is not greater than a preset distance; and controlling the output audio corresponding to the desired range and each of the virtual ranges according to the audio energy corresponding to each of the desired range and the virtual ranges, wherein the desired range and the virtual range are spatial ranges, and the virtual range overlaps or is adjacent to the desired range. The noise filtering method of the microphone array as described in claim 1, wherein the output audio of each of the expected range and the virtual ranges is controlled according to the audio energy corresponding to each of the expected range and the virtual ranges, including: determining whether a first energy value of the expected range is greater than a preset value; when the first energy value is greater than the preset value, executing for each of the virtual ranges: determining whether a second energy value of the virtual range is greater than the preset value; and if the second energy value is greater than the preset value, reducing the audio energy of the output audio of the virtual range; and when the first energy value is not greater than the preset value, not outputting the output audio of the expected range and the virtual ranges. The noise filtering method of the microphone array as described in claim 2, wherein reducing the audio energy of the output audio in the virtual range includes: Subtracting a second frequency spectrum of the output audio in the virtual range from a first frequency spectrum of the output audio in the desired range to generate a third frequency spectrum; and outputting the output audio corresponding to the third frequency spectrum. The noise filtering method of the microphone array as described in claim 2 further comprises: performing calculation on an original energy value and a previous energy value of each of the expected range and the virtual ranges through formula (1) to obtain the first energy value and the second energy value, δ(t) ' = αδ(t-1) + (1- α )δ(t) Formula (1) wherein δ(t) ' is the first energy value or the second energy value, α is the default value, 0<α

1, δ(t-1) is the previous energy value, δ(t) is the original energy value, wherein the time point corresponding to the previous energy value is earlier than the time point corresponding to the original energy value. A noise filtering method for a microphone array as described in claim 1, wherein the microphone array is a ceiling microphone array. A microphone array system includes: a microphone array for receiving audio signals of a desired range and multiple virtual ranges; and a processing device connected to the microphone array for executing: setting the desired range according to a user instruction; setting the virtual ranges based on the desired range, wherein the distance between the center of each of the virtual ranges and the center of the desired range is not greater than a preset distance; and controlling the output audio signals corresponding to the desired range and each of the virtual ranges according to the audio energy corresponding to each of the desired range and the virtual ranges, wherein the desired range and the virtual range are spatial ranges, and the virtual range overlaps or is adjacent to the desired range. A microphone array system as described in claim 6, wherein the processing device executes the control of the output audio of each of the expected range and the virtual ranges according to the audio energy corresponding to each of the expected range and the virtual ranges, including: determining whether a first energy value of the expected range is greater than a preset value; when the first energy value is greater than the preset value, executing for each of the virtual ranges: determining whether a second energy value of the virtual range is greater than the preset value; and if the second energy value is greater than the preset value, reducing the audio energy of the output audio of the virtual range; and when the first energy value is not greater than the preset value, not outputting the output audio of the expected range and the virtual ranges. A microphone array system as described in claim 7, wherein the processing device performs the step of reducing the audio energy of the output audio in the virtual range, including: subtracting a second frequency spectrum of the output audio in the virtual range from a first frequency spectrum of the output audio in the desired range to generate a third frequency spectrum; and outputting the output audio corresponding to the third frequency spectrum. A microphone array system as described in claim 7, wherein the processing device is further used to perform: performing calculations on an original energy value and a previous energy value of each of the desired range and the virtual ranges to obtain the first energy value and the second energy value through formula (1), δ(t) ' = αδ(t-1)+(1- α )δ(t) Formula (1) wherein δ(t) ' is the first energy value or the second energy value, α is the default value, 0<α

1, δ(t-1) is the previous energy value, δ(t) is the original energy value, wherein the time point corresponding to the previous energy value is earlier than the time point corresponding to the original energy value. A microphone array system as described in claim 6, wherein the microphone array is a ceiling microphone array.