TWI775401B - Two-channel audio processing system and operation method thereof - Google Patents

Abstract

The present application relates to a two-channel audio processing system and its operation method. The audio processing system It is configured in a head-mounted device, and the audio processing system includes at least a head-related position impulse response function unit and a localization mixing unit, wherein the head-related position impulse response function unit is used for according to the head-mounted device. The inertial measurement signal generates a head-related impulse function signal, and the positioning mixing unit is used for performing audio localization according to the multi-channel input audio signal and the head-related impulse function signal to generate a multi-channel output audio signal, and for The multi-channel output audio is down-mixed to obtain a dual-channel output audio, thereby improving the convenience for users to realize multi-channel audio data with a dual-channel audio processing system.

Description

Two-channel audio processing system and operation method thereof

The present application relates to an audio processing system and an operation method thereof, especially a dual-channel audio processing system and an operation method thereof.

With the mature development of consumer electronic products and changes in the needs of users, various products are constantly being introduced, such as electronic devices such as earphones and head-mounted displays. Users can choose suitable electronic devices according to their needs.

At the same time, users also have higher demands for the experience of audio and video. Generally speaking, users can obtain better audio experience by selecting multi-channel audio data. However, currently the most commonly used audio data is two-channel audio data, and multi-channel audio data is not popular. In addition, multi-channel audio data are all preset fixed audio data, so the multi-channel audio data cannot be based on real-time data. The user can make corresponding sound field adjustments according to the environmental conditions when the user is listening.

Therefore, in order to allow users to have a better sound effect experience, it is indeed necessary to propose a better solution.

In view of the above-mentioned deficiencies of the prior art, the main purpose of the present application is to provide a two-channel audio processing system and method, which adjusts multiple audio channels according to the displacement information sensed by the user's head-mounted device. The sound field positioning of channel audio data converts multi-channel audio data into two-channel audio data output, thereby realizing the sound effect of multi-channel audio data with two-channel audio data, allowing users to have better sound effects Experience, and enhance the convenience of users to achieve multi-channel audio data with a two-channel audio processing system.

The main technical means adopted to achieve the above-mentioned purpose is the operation method of the aforementioned two-channel audio processing system, and the audio processing system is configured on a head-mounted device, and the method includes the following steps: obtaining a multi-channel input audio and a head-related impulse function signal; perform audio localization on the multi-channel input audio with the head-related impulse function signal and generate a multi-channel output audio; and perform downmixing on the multi-channel output audio and obtain a Two-channel audio output.

From the above steps, after the method obtains the multi-channel input audio and the head-related impulse function signal, it performs audio localization to generate the corresponding multi-channel output audio, and converts the multi-channel output audio into binaural The method can immediately reflect the displacement state of the user's head, generate corresponding sound field changes, and realize the sound effect of the multi-channel audio data with the two-channel audio data, so that the user has a better sound effect. It improves the user's convenience of realizing multi-channel audio data with a two-channel audio processing system.

The main technical means adopted to achieve the above purpose is the aforementioned two-channel audio processing system, and the audio processing system is configured on a head-mounted device, and the audio processing system includes: a head-related position impulse response function unit, with generating a head-related impulse function signal according to the inertial measurement signal of the head-mounted device; and a positioning sound mixing unit electrically connected with the head-related position impulse response function unit, Wherein, the localization mixing unit is used to obtain a multi-channel input audio signal and the head-related impulse function signal, and perform audio localization accordingly to generate a multi-channel output audio signal, and perform downscaling on the multi-channel output audio signal. Mix to get a two-channel output audio.

By the above structure, after the location mixing unit obtains the multi-channel input audio and the head-related impulse function signal, it performs audio localization to generate the corresponding multi-channel output audio, and converts the multi-channel output audio into It outputs audio in two channels, whereby the structure can instantly reflect the displacement state of the user's head, and generate corresponding sound field changes. The user has a better sound effect experience, and enhances the convenience for users to realize multi-channel audio data with a two-channel audio processing system.

100: Audio Processing System

110: Head-related position impulse response function unit

130: Positioning Mixing Unit

131: 3D positioning unit

133: Downmix unit

150: Space Processing Unit

151: Reverb unit

153: Distance Adjuster Unit

155: Echo Control Unit

157: Equalizer unit

159: Volume gain unit

170: Up mixing unit

190: Inertia Correction Unit

S _axis : inertial measurement signal

S _c : Inertia correction signal

S _HRIR : head related impulse function signal

S _mi : Multi-channel input audio

S _mo : Multi-channel output audio

S _sa : Two-channel adjusted audio

S _si : Two-channel audio input

S _so : Two-channel output audio

S101, S103, S105, S107, S109, S111, S1071, S1073, S1075, S1077: Steps

1 is a schematic diagram of a system architecture according to an embodiment of the present application; FIG. 2 is a schematic diagram of another system architecture according to an embodiment of the present application; FIG. 3 is a schematic diagram of another system architecture according to an embodiment of the present application; FIG. 5 is a schematic flowchart of another step according to an embodiment of the present application; and FIG. 6 is a schematic flowchart of another step according to an embodiment of the present application.

Regarding the embodiment of the dual-channel audio processing system of the present application, please refer to FIG. 1 , the audio processing system 100 is a processing system configured in a head-mounted device, and at least includes a head-related position impulse response function unit 110 and a positioning mixing unit 130 .

The head-related position impulse response function unit 110 is used for generating a head-related impulse function signal according to the inertial measurement signal of the head-mounted device, and the positioning mixing unit 130 and the head-related position impulse response function unit 110 are electrically connected connected to obtain a multi-channel input audio signal S _mi and the head-related impulse function signal, and perform audio localization on the received multi-channel input audio signal S _mi according to the head-related impulse function signal, and generate a corresponding Multi-channel output audio is performed, and down-mixing is performed on the multi-channel output audio to obtain a two-channel output audio.

Because the audio processing system 100 of the present application can immediately reflect the displacement state of the user's head according to the inertial measurement signal, and the head-related impulse function signal defines the real-time multi-channel sound field change corresponding to the user Therefore, the present application can generate corresponding sound field changes in real time according to the displacement state of the user's head, and also output two-channel output audio suitable for the existing head-mounted device, which not only allows the user to have a better sound effect experience, but also improves the The user realizes the convenience of multi-channel audio data with a two-channel audio processing system.

Please refer to FIG. 2 for another embodiment of the two-channel audio processing system of the present application. The audio processing system 100 may further include a spatial processing unit 150 , a one-liter mixing unit 170 and an inertial correction unit 190 .

The spatial processing unit 150 is used for receiving a binaural input audio _Ssi , and performing spatial sound effects processing on the binaural input audio _Ssi , that is, using reverberation and an equalizer to enhance the sense of space and distance of the sound source sense, and output the corresponding two-channel adjusted audio S _sa . The up-mixing unit 170 is electrically connected to the spatial processing unit 150 for receiving the two-channel adjustment audio S _sa , and generating virtual multi-channel outputs according to the two-channel adjustment audio S _sa to separate out corresponding Multi-channel multi-channel input audio S _mi . The inertial correction unit 190 is configured to receive the inertial measurement signal S _axis from the multi-axis sensor of the head-mounted device, and use a Kalman filter or an attitude analysis algorithm (MahonyAHRS) to measure the inertial measurement The signal S _axis is corrected, and an inertia correction signal S _c corresponding to an Eulerian coordinate is generated.

Further, the head-related position impulse response function unit 110 is used for receiving the inertial correction signal S _c , and according to the inertial correction signal S _c to calculate or obtain the head-related impulse function required for the three-dimensional spatial positioning of the audio, and generate corresponding The head-related impulse function signal S _HRIR .

Further, the localization mixing unit 130 further includes a three-dimensional localization unit 131 and a downmixing unit 133 . The three-dimensional positioning unit 131 is electrically connected to the up-mixing unit 170 and the head-related position impulse response function unit 110 for receiving the multi-channel input audio signal S _mi , and according to the head-related impulse function signal S The _HRIR and the corresponding spatial position (channel) configure the audio, and generate the corresponding multi-channel output audio S _mo . The down-mixing unit 133 is electrically connected to the three-dimensional positioning unit 131 for receiving the multi-channel output audio S _mo , and generates a two-channel output audio S _so including sound field localization through the mixing gain operation of each channel .

The multi-channel referred to in this embodiment is 5.1 channels, and this application is not limited thereto, and the channel system with more than two-channel channels can be referred to as multi-channel in this application.

In one embodiment, the head-related position impulse response function unit 110, the localization mixing unit 130, the spatial processing unit 150, the up-mixing unit 170, and the inertial modification unit 190 may be one or more microprocessors. It can be realized by circuit units, and the present application is not limited by this.

In an embodiment, the two-channel input audio S _si comes from an electronic device communicatively connected to the head-mounted device, such as a smart phone, a tablet computer or a notebook computer, and the present application is not limited thereto.

In one embodiment, the inertial measurement signal S _axis includes sensing results of a multi-axis gyroscope, a multi-axis accelerometer, and a multi-axis magnetometer, and the present application is not limited thereto.

To further illustrate the spatial processing unit 150, please refer to FIG. 3, the spatial processing unit 150 includes a reverberator unit 151, a distance adjuster unit 153, an echo control unit 155, an equalizer unit 157 and a volume gain unit 159. The reverberator unit 151 is used for receiving the two-channel input audio _Ssi , and editing and adjusting the two-channel input audio Ssi to have an echo effect, so as to create a sense of sound space of the multi-channel audio. The distance adjuster unit 153 is used to receive the binaural input audio S _si edited and output by the reverberator unit 151 , and to extend the direct sound in the binaural input audio S _si to achieve a sense of distance. The echo control unit 155 is used to receive the two-channel input audio S _si edited and output by the distance adjuster unit 153 , and use the adjusted echo to enhance the solidity and sound effect of the sound. The equalizer unit 157 is used for receiving the two-channel input audio _Ssi edited and output by the echo control unit 155, and for enhancing the low-frequency effect of the two-channel input audio _Ssi . The volume gain unit 159 is used for receiving the two-channel input audio S _si edited and output by the equalizer unit 157 , adjusting the gain of the received audio, and outputting the two-channel adjusted audio S _sa .

In one embodiment, the equalizer unit 157 is, for example, a shelving filter, and the present application is not limited thereto.

In this way, the spatial processing unit 150 can perform preprocessing such as sound effect adjustment and editing on the received two-channel input audio S _si .

Therefore, when the user wears the head-mounted device, the head-mounted device can instantly change the sound field of the two-channel input audio according to the displacement state of the user, and realize the multi-channel audio transmission with the two-channel audio. The sound effect allows users to experience the sound effect of the stereo sound field effect with different two-channel audio at any time.

Therefore, the audio processing system 100 of the present application can immediately reflect the displacement state of the user's head according to the inertial measurement signal, and the head-related impulse function signal defines the real-time multi-channel audio corresponding to the user. Therefore, the present application can generate corresponding sound field changes in real time according to the displacement state of the user's head, and also output two-channel output audio suitable for the existing head-mounted device, which not only allows the user to have a better sound effect experience, It also improves the convenience for users to realize multi-channel audio data with a two-channel audio processing system.

The operation method of a two-channel audio processing system can be summarized from the above-mentioned embodiments and application methods of the present application, and the audio processing system (as shown in FIG. 1 or FIG. 2 ) is configured on a head-mounted device, as shown in FIG. 4 . As shown, the method includes the following steps: the audio processing system 100 obtains a multi-channel input audio signal and a head-related impulse function signal (S107); the audio processing system 100 uses the head-related impulse function signal to the multi-channel Perform audio localization on the input audio and generate a multi-channel output audio (S109); and the audio processing system 100 performs downmixing on the multi-channel output audio and obtains a two-channel output audio (S111).

Further, please refer to FIG. 2 and FIG. 5 at the same time, the method further includes the following steps: obtaining two-channel input audio _Ssi (S101); performing sound effect processing to generate a two-channel adjustment audio _Ssa (S103); the audio processing The spatial processing unit 150 of the system 1 is used for receiving the two-channel input audio _Ssi , and performing spatial sound effects processing on the two-channel input audio _Ssi , and outputting the corresponding two-channel adjustment audio _Ssa ; Mixing to obtain a multi-channel input audio (S105); the up-mixing unit 170 of the audio processing system 100 is used to receive the two-channel adjustment audio S _sa and separate the two-channel adjustment audio S _sa into corresponding multi-channel audio multi-channel input audio S _mi of the channel; obtain the multi-channel input audio S _mi and a head related impulse function signal S _HRIR (S107); perform audio localization and generate a multi-channel output audio (S109); the The three-dimensional positioning unit 131 of the audio processing system 100 receives the multi-channel input audio signal S _mi , and adjusts and configures the audio according to the head-related impulse function signal S _HRIR and the corresponding spatial position, and generates corresponding multi-channel output. and perform downmixing and obtain a two-channel output audio ( _S111 ); the downmixing unit 133 of the audio processing system 100 receives the multi-channel output audio _Smo and generates the two-channel output audio S _so .

In this preferred embodiment, when the above steps are executed to the step of "the audio processing system 100 obtains the multi-channel input audio and a head related impulse function signal (S107)", as shown in FIG. 6, the method further Including the following sub-steps: obtaining an inertial measurement signal S _axis (S1071); the audio processing system 100 receives the inertial measurement signal S _axis from the multi-axis sensor of the head-mounted device; generates an inertial correction signal S _c ( S1073); the inertial correction unit 190 of the audio processing system 100 receives the inertial measurement signal S _axis , and performs corresponding adjustment and correction to generate the inertial correction signal S _c ; generates the head-related impulse function signal S _HRIR (S1075 ); the head-related position impulse response function unit 110 of the audio processing system 100 is used to receive the inertial correction signal S _c , and generate a corresponding head-related impulse function signal S _HRIR according to the inertial correction signal S _c ; and obtain the The multi-channel input audio S _mi and the head-related impulse function signal S _HRIR ( S1077 ), the localization mixing unit 130 of the audio processing system 100 obtains the multi-channel input audio S _mi from the one-liter mixing unit 170 and the The head related impulse function signal S _HRIR .

Therefore, the dual-channel audio processing system and its operation method of the present application can instantly reflect the displacement state of the user's head according to the inertial measurement signal, and the head-related impulse function signal defines the real-time corresponding to the user's Multi-channel sound field changes, so the application can generate corresponding sound field changes in real time according to the displacement state of the user's head, and output two-channel output audio suitable for existing head-mounted devices, which not only allows users to have better It enhances the user's convenience in realizing multi-channel audio data with a two-channel audio processing system.

100: Audio Processing System

110: Head-related position impulse response function unit

130: Positioning Mixing Unit

S _mi : Multi-channel input audio

S _so : Two-channel output audio

Claims (8)

An operation method of a two-channel audio processing system, the audio processing system is configured on a head-mounted device, the operation method includes: receiving a two-channel input audio; performing sound effect processing on the two-channel input audio to generate an echo the two-channel input audio with echo; perform sound effect processing on the two-channel input audio with echo to adjust an audio depth of the two-channel input audio with echo; Audio performing sound effect processing to enhance a bass audio; performing sound effect processing to adjust an audio gain on the two-channel input audio with echo that has enhanced the adjusted audio depth of the bass audio to generate a two-channel adjusted audio; Performing up-mixing on the channel adjustment audio to obtain a multi-channel input audio signal; receiving the multi-channel input audio signal and an inertial measurement signal; generating a head-related impulse function signal according to the inertial measurement signal; The impulse function signal performs audio localization on the multi-channel input audio and generates a multi-channel output audio; and performs downmixing on the multi-channel output audio and obtains a two-channel output audio. The operation method of the two-channel audio processing system according to claim 1, wherein the step of obtaining a multi-channel input audio signal, an inertial measurement signal and a head-related impulse function signal further comprises: the inertial The measurement signal is corrected to generate an inertial correction signal corresponding to an Euler coordinate; and the head-related impulse function signal is generated according to the inertial correction signal. The operation method of the two-channel audio processing system according to claim 2, wherein the inertial measurement signal is generated by a multi-axis sensor of the head-mounted device. A dual-channel audio processing system, the audio processing system is configured in a head-mounted device, comprising: a head-related position impulse response function unit for generating a head-related position according to an inertial measurement signal of the head-mounted device impulse function signal; a spatial processing unit for receiving a two-channel input audio signal, and the spatial processing unit includes: a reverberator unit for receiving the two-channel input audio; a distance adjuster unit, and the The reverberation unit is electrically connected to receive the two-channel input audio output by the reverberation unit; an echo control unit is electrically connected to the distance adjuster unit to receive the dual-channel output audio from the distance adjuster unit. channel input audio; an equalizer unit electrically connected to the echo control unit for receiving the two-channel input audio output from the echo control unit; a volume gain unit electrically connected to the equalizer unit, used to receive the two-channel input audio output from the equalizer unit, and output the two-channel adjusted audio accordingly; a one-liter mixing unit is electrically connected to the spatial processing unit for receiving the two-channel adjusted audio audio, and generate a multi-channel input audio corresponding to the multi-channel according to the two-channel adjustment audio; and a positioning mixing unit, electrically connected with the head-related position impulse response function unit; Wherein, the localization mixing unit is used for obtaining the multi-channel input audio and the head-related impulse function signal, and performing audio localization accordingly to generate a multi-channel output audio, and performing downscaling on the multi-channel output audio. Mix to get a two-channel output audio. The two-channel audio processing system according to claim 4, wherein the localization mixing unit further comprises: a three-dimensional localization unit for generating the multi-channel audio according to the multi-channel input audio and the head-related impulse function signal. channel output audio; and a down-mixing unit electrically connected with the three-dimensional positioning unit for performing down-mixing on the multi-channel output audio to obtain the dual-channel output audio. The two-channel audio processing system of claim 5, wherein the audio processing system further comprises: an inertial correction unit electrically connected to the head-related position impulse response function unit for receiving the inertial amount A measurement signal is obtained, the inertial measurement signal is corrected, and a corresponding inertial correction signal is generated. The two-channel audio processing system according to claim 6, wherein the head-related position impulse response function unit is used for receiving the inertial correction signal and generating the corresponding head-related impulse function signal. The two-channel audio processing system of claim 4, wherein the equalizer unit is a shelving filter.

Images