CN1253464A - 3D sound regeneration equipment and method for many listeners - Google Patents

Abstract

The present invention is a 3D sound reproduction device for multiple listeners, comprising: an inverse filter module for filtering input sound signals so that the listeners can have the same virtual sound source, for A time-division multiplexing module for sequentially selecting a sound signal filtered by the reverse filter module for outputting the sound signal selected by the time-division multiplexing device as a plurality of loudspeakers for sound. Thus, the 3D sound reproduction device can simultaneously provide the same 3D sound to a plurality of listeners.

Description

Three-dimensional sound reproduction device and method for multiple listeners

The present invention relates to a three-dimensional (3D) sound reproduction device, and in particular, to a 3D sound reproduction device and a method thereof that provide the same 3D sound to a plurality of listeners in parallel.

In the audio industry, many efforts have been made to arrange loudspeakers on a one-dimensional point or on a two-dimensional plane in order to reproduce sound with full presence. That is, the monaural system at the initial development period, the later stereo system, and the latest Dolby Surround system are all used to reproduce signals with a sense of presence. However, with the development of the multimedia industry, the purpose of the technology for recording and reproducing auditory information (ie, sound signals as well as visual information) has changed from faithfully reproducing signals with a sense of presence to reproducing signals in a 3D sound space. Speakers can be placed anywhere in the 3D sound space.

Today, most audio equipment can reproduce stereo signals rather than mono signals. When a stereo signal is reproduced, the range of presence felt for the reproduced signal is limited by the installation position of the speakers. Accordingly, in order to improve the range of presence, much research has been conducted on improving speaker performance and generating virtual signals using signal processing.

A typical system based on the above research results is the Dolby Surround sound system using the surround reproduction method of a group of 5 speakers. In this system, the virtual signal output to the rear speaker is processed independently, that is, the signal is delayed according to the signal's spatial environment and the reduced amplitude signal is transmitted to the rear speaker to generate a virtual signal. Currently, most home VCRs and DVD players use a technology called a "Dolby Pro Logic" system. Thanks to the equipment using the above technology, it is possible to reproduce the same sound quality as in a theater at home.

As described above, although the sound more faithful to the sense of presence can be obtained by increasing the number of channels, the same number of speakers as the number of channels is required, and accordingly, problems of cost and installation space arise.

These issues can be improved by using research findings on how humans hear and perceive sounds that exist in 3D space. In particular, in the study of sound discrimination by human ears, the inventors have studied the problem that two ears of a person greatly affect the discrimination of sound sources in 3D space.

The above study deals with the interaction of input signals into the two ears, i.e., the difference in auditory intensity of the signal amplitudes sensed by the right and left ears produced by the difference in hearing time in the transmitted sound or the difference in the auditory intensity of the signals input to the right and left ears. The difference in the phase of the sound of the ear. Based on the results of research on human binaural ears, the characteristics of human beings to identify a sound source existing at a certain point in space are simulated. Such a discriminative characteristic is called a head-related transfer function (hereinafter referred to as "HRTF").

HRTF is a filter coefficient for simulating a path from a sound source to an eardrum, characterized by having a value that varies with the relative position between the sound source and the head. In this case, HRTF is described as an impulse response or transfer function in the middle ear of a part of the human body in which signals are transmitted to both ears when a sound source exists at one point in space. By using the HRTF, it is possible to perform a process of transferring sound from the position where it exists to other arbitrary positions in the 3D space.

At the same time, people have also conducted a lot of research on how the human auditory sense can distinguish the 3D sound space. In response to this, a virtual sound source has recently been proposed and its practical application has been investigated.

Typically, listeners hear stereo sound best at the position of the vertices of an equilateral triangle having the base as a line connecting the two speakers. However, since the position of the listener cannot be restricted to this point, a space problem will arise. In addition, it is very difficult to adjust the balance of the sound according to the position of the listener.

The Japanese company Aiwa solved this problem by adding a "unidirectional" speaker capable of producing a strong sound directed towards the listener into a typical speaker setup. The greatest feature of the above speakers is that the listener can enjoy balanced stereo sound at any position towards the speaker device. In a typical loudspeaker system, if the listener moves to the left relative to the loudspeaker device, the sound produced by the right end loudspeaker will attenuate. However, since the single directional speaker contained in the speaker device is internally inclined at 45°, the right end speaker will produce a strong sound to the left and a weak sound to the right. Conversely, a single directional speaker in a left speaker setup will produce a weak sound to the left and a strong sound to the right. Thus, the sound produced by the left and right speakers is balanced when the listener is located on the left or right.

In 1993, JVC, another company in Japan, developed a speaker system that provides virtual reality sound. With this system, it is possible to listen to the sound from the back where no speaker actually exists, using only two speakers arranged in the front. The speaker described above uses the human auditory characteristic. Humans unconsciously use both ears to search for the direction of sound. The speed of sound transmission is 340m/s, and the distance between the two ears is about 20cm. Therefore, the difference in the time of sound transmission to the two ears is at most 1/500s. The difference in the volume of the sound reaching the two ears is also an important parameter for distinguishing the direction of the sound. Humans use information from the difference between the two ears and the eyes to identify the source of a sound. In this way, if the timing of sound transmission to both ears can be controlled, the sound produced from only two speakers can fill the entire room, making the listener feel as if he/she is sitting in a theater.

However, all technologies developed so far for 3D sound are aimed at a single listener. That is, current audio reproduction systems will provide a stereo effect when a single listener is located at the apex of an equilateral triangle whose base is a line connecting the two speakers. However, the same and simultaneous stereophonic effect cannot be provided in the case of multiple listeners.

Such a problem would become severe in the case of a home theater system. As shown in Figure 1, when all family members sit around one sound source, a general home theater system cannot provide good stereo sound to all family members.

More recently, it has been proposed to provide presence and space by equipping the "Dolby Pro Logic" system with more speakers instead of two-channel reproduction. However, in the system described above, multiple listeners will be confined to the center of the circle connecting each speaker to enjoy the full 3D effect. Furthermore, in order to provide multi-channel audio, a corresponding plurality of speakers and an amplifier for driving each speaker are provided. Thus, problems of cost and installation space will arise.

In order to solve the above-mentioned problems, an object of the present invention is to provide a 3D sound reproducing apparatus and method thereof capable of providing the same 3D sound to a plurality of listeners at the same time regardless of their positions.

Correspondingly, in order to achieve the above object, the 3D sound reproduction device for multiple listeners provided by the present invention includes: an inverse filter module that filters the input signal so that each listener can have the same virtual sound source. A time-division multiplexing module for sequentially selecting a sound signal filtered by the above reverse filter module at time intervals, and a plurality of speakers for outputting the sound signal selected by the above-mentioned time-division multiplexing device as sound.

Another aspect of the present invention is to provide a method for reproducing an input sound signal and providing the same 3D sound to a plurality of listeners through two or more fixed number of loudspeakers, the method comprising the steps of: (a) Obtain the speaker transfer function simulating the path between the above loudspeaker and one of the listener's ears, (b) by multiplying the inverse matrix of the speaker transfer function with the effective source transfer function simulating the path between the effective source and one of the listener's ears And the step of obtaining a filter value, (c) a step of sequentially selecting a filter value at a predetermined interval, and (d) performing convolution processing on the input sound signal using the selected filter value and converting the convolution processing The result of the step is output to the speaker.

The above objects and advantages of the present invention will be further revealed by the detailed description of the preferred embodiments with reference to the accompanying drawings.

FIG. 1 is an explanatory diagram of a situation in which a plurality of listeners are located in a general stereoscopic reproduction system.

FIG. 2 is a block diagram showing the structure of the 3D sound reproduction system for multiple listeners of the present invention.

FIG. 3 shows an example of the relationship between a sound source in one virtual space and two speakers included in a two-channel playback system.

Fig. 4 is a diagram showing the speaker position compensation relationship for generating a virtual sound source in a two-channel reproduction system described by the concept of a transfer function.

FIG. 5 shows an example of the relationship between a virtual sound source and an inversely filtered real sound source in a two-channel reproduction system.

FIG. 6 is a block diagram showing the structure of the loudspeaker position compensation system of FIG. 4 constituted by a filter matrix.

Figure 7 shows the arrangement of loudspeakers and heads used in experiments to accurately simulate HRTF at multiple listener positions.

The present invention will be described in detail below in conjunction with the accompanying drawings.

As shown in FIG. 2 , the 3D sound reproduction device for multiple listeners of the present invention includes an inverse filter module 100 , a time division multiplexing device 200 and multiple speakers 300 .

The inverse filter module 100 includes a plurality of inverse filter sections 10 , 20 and 30 for filtering an input sound signal in order to have the same effective sound source for a plurality of listeners 400 . The time division multiplexing device 200 selects one of the sound signals filtered by the inverse filter module 100 so as to conform to a predetermined cycle. The speaker 300 outputs the audio signal selected by the time division multiplexing device 200 as audio.

The method of the present invention requires a HRTF measurement model corresponding to each position of multiple listeners. This is because the positions of multiple listeners are expected to vary widely and away from the standard position compared to the standard position of one listener in the middle of two loudspeakers. Thus, accurate HRTF models are required for both the loudspeaker and each listener.

HRTFs used in the present invention are described below.

HRTF is a filter coefficient obtained by simulating the transmission path from a sound source to a human eardrum. In addition, it also refers to a transfer function on the frequency plane showing the sound transmission from a sound source to the human ear canal in free space, and it also refers to the degree of frequency distortion related to the human head, auricle and body.

Due to the structure of the ear, the spectrum of the signal will be distorted by the irregular shape of the ear before the signal reaches the ear canal. Since the amount of distortion is related to the direction and distance of the sound, this change in frequency content will serve as an important factor in identifying the direction of the sound by a human. HRTF indicates the degree of frequency distortion.

Thus, the HRTF is determined by the position of the sound source, and the HRTFs of the left ear and the right ear are different from each other for the sound source at the same position. Also, since the shape of the face and the pinnae of each person are different from each other, the HRTF is different for each human body.

3D sound can be reproduced by using HRTF. That is, when the HRTF is located at a specific position and the input audio signal is convoluted, the sound appears to be generated at the specific position.

[Formula 1]

y[n]=h[n] ^* x[n]=IFFT{H[k] X[k]}

As shown in Equation 1, in general, the convolution of two signals h[n] and x[n] in the time domain with the two signals H[k] and X processed by FFT (Fast Fourier Transform) [k] IFFT (Inverse Fast Fourier Transform) multiplication in the frequency domain is the same. Perform FFT processing on the given HRTF in advance. In general, the above method is chosen because multiplication in the frequency domain is much faster to process than convolution in the time domain.

First, an HRTF corresponding to the initial position information of the speaker is obtained, and then another HRTF corresponding to the virtual sound source is obtained, and a matrix operation is performed. This matrix operation provides the correlation between the positions of the loudspeakers and the positions of the effective sound sources. In this way, since the relationship between the speakers at any position can be obtained by matrix operation, the quality of the reproduced sound can be independent of the position of the speakers.

First, a 3D sound reproduction method in the case of only one signal listener will be described.

As shown in Figure 3, assuming that a listener's position is at the center of a circle connecting two speakers that reproduce 3D sound as the required data, 6 HRTFs are required: 4 from each speaker to the listener's two ears. HRTFs and two HRTFs from a virtual sound source to the listener's ears. In Figure 3, L and R represent the respective positions of the left and right speakers, and vs represents the virtual position from which the listener expects to hear the sound.

Although all sound is actually produced from the two speakers, the listener perceives the sound as if it was produced at a particular location in the 3D space. This is achieved by convolving the input signal with the HRTF of the particular location from which the listener wishes to hear the sound, by altering the sound itself produced by the two speakers.

To vary the HRTF between the two speakers and the two ears, an inverse filter is used. Here, the signal output from the left end speaker is not transmitted to the left ear, and the signal output from the right end speaker is not transmitted to the right ear. This is the cross-interference cancellation method. After canceling the signals generated by the two speakers, the HRTF for the direction in which the listener expects the sound is convolved with the input signal. In this way, the listener feels as if the sound is not produced from the speaker but from a specific position.

Referring to Fig. 4, block C110 is the filter matrix for simulating the sound transmission path from two loudspeakers to the two ears of people, and block D120 is the sound transmission path simulated from the virtual sound source that the listener wishes to hear in the two ears. Filter matrix for the path of the sound. Block H130 is an inverse filter matrix for compensating the relationship between the virtual sound source and the two loudspeakers set up, where the input signal is convolved before being output to the loudspeakers. Figure 5 shows the concept of the above relationship.

Fig. 6 shows the calculation method of the inverse filter H. That is, when the two input signals are L and R respectively, the final output signals Y _L and Y _R transmitted from the speaker to the two ears can be described as follows:

[Formula 2] $\left[\begin{array}{c}{Y}_L\\ Y_R\end{array}\right]=\begin{array}{c}\left[\begin{array}{cc}{C}_{\mathrm{LL}}& C_{\mathrm{RL}}\\ C_{\mathrm{LR}}& C_{\mathrm{RR}}\end{array}\right]\·\left[\begin{array}{cc}{h}_{\mathrm{LL}}& h_{\mathrm{RL}}\\ h_{\mathrm{LR}}& h_{\mathrm{RR}}\end{array}\right]\&Center\; Dot;\left[\begin{array}{c}L\\ R\end{array}\right]\end{array}$

In addition, given that the virtual output values in the position where the listener wishes to hear the sound are V _L and _VR , then the above relationship can be described as follows: [Equation 3] $\left[\begin{array}{c}{V}_L\\ V_R\end{array}\right]=\left[\begin{array}{cc}{\mathrm{D.}}_{\mathrm{LL}}& {\mathrm{D.}}_{\mathrm{RL}}\\ {\mathrm{D.}}_{\mathrm{LR}}& {\mathrm{D.}}_{\mathrm{RR}}\end{array}\right]\&Center\; Dot;\left[\begin{array}{c}L\\ R\end{array}\right]$

As a result, in an ideal situation, Equation 2 and Equation 3 would be the same. In practice, the agreement is quite good if the difference between the two formulas is small. Assuming that the above two formulas are the same, the inverse filter H matrix can be obtained from the following formula. [Formula 4] $\left[\begin{array}{cc}{h}_{\mathrm{LL}}& h_{\mathrm{RL}}\\ h_{\mathrm{LR}}& h_{\mathrm{RR}}\end{array}\right]={\left[\begin{array}{cc}{C}_{\mathrm{LL}}& C_{\mathrm{RL}}\\ C_{\mathrm{LR}}& C_{\mathrm{RR}}\end{array}\right]}^{-1}\&CenterDot;\left[\begin{array}{cc}{\mathrm{D.}}_{\mathrm{LL}}& {\mathrm{D.}}_{\mathrm{RL}}\\ {\mathrm{D.}}_{\mathrm{LR}}& {\mathrm{D.}}_{\mathrm{RR}}\end{array}\right]$ $=\frac{1}{C_{\mathrm{LL}}{C}_{\mathrm{RR}}-C_{\mathrm{LR}}{C}_{\mathrm{RL}}}{\left[\begin{array}{cc}{C}_{\mathrm{RR}}& -C_{\mathrm{RL}}\\ -C_{\mathrm{<figure-callout\; id="1"\; label="LR\; C\; LL"\; filenames="A9812367800111.png"\; state="\{\{state\}\}">LR</figure-callout>}}& C_{\mathrm{LL}}{}_{}\end{array}\right]}^1\&Center\; Dot;\left[\begin{array}{cc}{\mathrm{D.}}_{\mathrm{LL}}& {\mathrm{D.}}_{\mathrm{RL}}\\ {\mathrm{D.}}_{\mathrm{LR}}& {\mathrm{D.}}_{\mathrm{RR}}\end{array}\right]$

The playback method for multiple listeners will be described below.

In the reproduction method for the case of multiple listeners, an accurate HRTF model corresponding to each listener's position should first be proposed. Since a typical HRTF model like Kemar provided by MIT simulates the transfer function when a listener is located at the center, it cannot actually be applied in the present invention. Therefore, to measure the HRTF corresponding to the position of the listener, it is necessary to arrange the test equipment as shown in Fig. 7 . Here, the distance between each listener is set to 30cm and the positions of the two loudspeakers deviate 30° to the left and right, which is a standard stereo reproduction position. By using the above-obtained HRTF for each listener position, each inverse filter is recalculated to obtain an inverse filter module 100 including a plurality of inverse filter sections 10, 20, and 30 corresponding to each listener .

The time division method of the core part of the present invention will be described below.

Reverse filters independently processed for each listener are sequentially selected at regular time intervals, and sound signals partially processed by the selected reverse filters are reproduced through the two speakers. Because although the scenes formed by the actual switching are not continuous, yet, due to the phenomenon of persistence of vision, our eyes feel as if it is a continuous scene that is continuous within a certain time interval, so the above method is feasible. That is, although the result of each filter processing is independent of the position of each listener, when the processing results are sequentially output to the speaker at predetermined time intervals, each listener can feel the From her position he/she hears a continuous sound.

Here, the most important thing is the regeneration time interval for each position. If the regeneration interval is set too long for a certain location, other listeners in other locations will not hear the sound. In addition, if the reproduction time interval is too short, the listener cannot have sufficient time to listen to a complete sound.

The effect of the present invention is as follows.

In order to reproduce an input sound signal through two speakers so that the same 3D sound is provided for a plurality of listeners, a method for simulating for each listener the path from the two speakers to the two ears of each listener is obtained Loudspeaker transfer function. Here, the position of the listener can be determined within a specific range without being limited to a central position.

In addition, the filter value is obtained by multiplying the transfer function of the virtual sound source simulating the path between the virtual sound source and one of the listener's ears, and the inverse matrix of the speaker transfer function. The input sound signal is convolved with a filter value.

One filter value is sequentially selected at predetermined intervals and output to the speaker. Since the listener needs at least 20ms time interval to distinguish the sound, the regeneration interval of each listener position in the present invention is at least greater than 20ms. In addition, if there are a large number of listeners, the time division multiplexing method of the present invention has a limit in the number of listeners since it will take a lot of time to process the signal for all the listeners.

In the described embodiment of the invention, the construction time-division interval can be variably adjusted according to the total number of listeners.

In addition, the number of loudspeakers in the above description is limited to two, however, the present invention can also use a plurality of loudspeakers. Therefore, it should be noted that the present invention is not limited to the above-mentioned embodiments, and variations and modifications using known techniques can be made within the spirit of the present invention and the scope defined by the appended claims.

As described above, according to the present invention, it is possible to enjoy 3D sound using only two speakers and to simultaneously provide the same 3D sound effect to a plurality of listeners.

Especially, when enjoying an audio/video home theater system in a room, all family members sitting freely in front of the speaker can simultaneously listen to the same 3D sound and enjoy realistic sound.

Claims (4)

1. A 3D sound reproduction device for multiple listeners, characterized in that it comprises: The reverse filter module is used to filter the input sound signal so that each listener can have the same virtual sound source; A time division multiplexing module for sequentially selecting a sound signal filtered by the above-mentioned reverse filter module at predetermined intervals; and a plurality of speakers for outputting the audio signal selected by the time division multiplexing device as audio. 2. The device as claimed in claim 1, characterized in that: The inverse filter module includes a plurality of inverse filter sections equal to the number of listeners, each of the above filter sections having a filter characteristic obtained by combining the simulated above loudspeaker with the corresponding inverse filter The value obtained by multiplying the inverse matrix C ^-1 of the loudspeaker transfer function C of the path between one ear of the listener and the virtual sound source transfer function D simulating the path between a virtual sound source and the listener's ear. 3. A method of regenerating an input sound signal through two or more fixed numbers of loudspeakers so as to provide the same 3D sound to a plurality of listeners, the method being characterized in comprising the steps of: (a) obtaining a loudspeaker transfer function simulating the path between said loudspeaker and one ear of each listener; (b) obtaining a filter value obtained by multiplying the inverse matrix of the loudspeaker transfer function with a virtual sound source transfer function simulating the path between the virtual sound source and one of the listener's ears; (c) the step of sequentially selecting a filter value within a valid interval; (d) Convolving the input sound signal using the selected filter value and outputting the result of the convolution processing to the speaker. 4. The method as recited in claim 3, characterized in that: The aforementioned predetermined interval in the aforementioned step (c) varies in proportion to the number of the aforementioned listeners.

Images