TWI899919B - Method for creation of linearly interpolated head related transfer functions - Google Patents

Abstract

Systems, devices, and methods are described for determining a “coupled” pair of Left/Right ear Head Related Transfer Functions (HRTFs) that are adapted from an original pair of Left/Right ear HRTFs, wherein the inter-aural delay of the coupled HRTFs is formed using all-pass filters that provide the correct inter-aural delay at low frequencies. The all-pass filters are adapted to limit the inter-aural phase difference at high frequencies. Furthermore, a low-complexity process is described for rapid generation of suitable all-pass filters.

Description

Method for constructing linearly interpolated head-related transfer functions

The present invention relates to creating a modified head-related transfer function (HRTF) from an original HRTF.

Unless otherwise indicated herein, the methods described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

A binaural audio signal consists of two audio channels intended for playback to a listener through each of their two ears (left and right). Binaural playback can be achieved via speakers placed close to each ear or via headphones (including over-ear and in-ear headphones).

Binaural signals can be generated by processing a source audio signal using a pair of head-related transfer function (HRTF) filter responses. HRTF responses can be defined in a variety of ways, including as time-domain impulse responses or as frequency-domain responses. HRTF responses are typically grouped in pairs to provide a response for each ear sensor.

When used to process an audio signal, an HRTF filter pair can be used to provide a listener with an experience that simulates the sound (at each ear) that would occur if the audio signal were presented from a specific arrival direction. Different HRTF filter pairs will produce the illusion of different sound source directions.

A pair of reference HRTF filters associated with a particular arrival direction can be determined by measuring the acoustic transfer function from a sound source located at a certain distance in the same direction to a listener's ear. Alternatively, the reference HRTF filters can be determined by other means, including numerical simulation or acoustic measurement of a human body model.

A pair of modified HRTF filters can differ from a pair of acoustically measured HRTF filters while still providing a listener with the desired impression of a sound originating from the same direction. Specifically, the phase difference between the high-frequency portions of the left and right modified HRTF filters can be substantially different from the phase difference between the high-frequency portions of the left and right reference HRTF filters, without a significant loss in perceived listener experience. This is possible because interaural phase differences in the high-frequency range are largely insignificant to a listener's perception.

A HRTF set function is a function that determines the left and right ear HRTF filters given a direction of arrival: ₍ hl ( t ) ,hr ₍ t ))← H ( x,y,z ) (1)

In Equation 1, the HRTF set function H ( x,y,z ) has an arrival direction in the form of a 3D unit vector ( x,y,z ), and the function returns a pair of left/right ear HRTF filters.

It is with respect to these and other considerations that the present invention made herein is presented.

Techniques for processing audio signals are described. Various examples described herein provide systems, methods, and/or devices for creating and using modified HRTF filters with alternative high-frequency phase responses.

According to some exemplary embodiments, an audio processing method for a control system comprising one or more processors may involve: obtaining, by the control system, a first set of head-related transfer functions (HRTFs); and transforming, by the control system, the first set of HRTFs into a second set of HRTFs. In some exemplary embodiments, the transformation may involve replacing a delayed component of the first set of HRTFs with an all-pass filter in the second set of HRTFs. In some exemplary embodiments, the transformation may involve adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that: for frequencies below a threshold frequency associated with the corresponding all-pass filter, each interaural phase response is substantially linear, and for frequencies above the threshold frequency associated with the corresponding all-pass filter, each interaural phase has a reduced interaural phase difference.

In some example embodiments, the method may involve outputting the second set of HRTFs. According to some example embodiments, outputting the second set of HRTFs may involve storing the second set of HRTFs, transmitting the second set of HRTFs to a device configured to process audio data, providing the second set of HRTFs for further processing, or a combination thereof.

According to some example embodiments, the method may involve: defining, by the control system, a set of basic filters based on the second set of HRTFs. The set of basic filters may have fewer components than the second set of HRTFs. In some example embodiments, the method may involve: obtaining, by the control system, a bit stream of input audio data in an input audio format; and combining, by the control system, the input audio data with one or more basic filters of the set of basic filters to generate left audio data and right audio data.

In some example embodiments, the method may involve: outputting, by the controller, the left audio data and the right audio data. According to some example embodiments, outputting the left audio data and the right audio data may involve storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing, by the control system, the left audio data and the right audio data to a set of speakers for playback, providing the left audio data and the right audio data for further processing, or a combination thereof.

According to some exemplary embodiments, the transformation may also involve: obtaining a left-ear HRTF and a right-ear HRTF from the first set of HRTFs; identifying a left-ear undelayed pulse response and a left-ear delay from each of the left-ear HRTFs; and identifying a right-ear undelayed pulse response and a right-ear delay from each of the right-ear HRTFs. In some exemplary embodiments, the transformation may also involve: generating a left-ear all-pass filter, each of the left-ear all-pass filters being based at least in part on an instance of the left-ear delays. According to some exemplary embodiments, the transformation may also involve: generating a right-ear all-pass filter, each of the right-ear all-pass filters being based at least in part on an instance of the right-ear delays. In some exemplary embodiments, the transformation may also involve combining instances of the left-ear and right-ear undelayed pulse responses with corresponding instances of the left-ear and right-ear all-pass filters to generate HRTF pairs for the second set of HRTFs.

In some exemplary embodiments, the method may also involve generating modified left-ear delay values and right-ear delay values based on one or more of the extracted left-ear delay and right-ear delay. The left-ear all-pass filters and the right-ear all-pass filters may be based on the modified left-ear delay values and the modified right-ear delay values.

According to some example embodiments, generating the modified left-ear delay values and right-ear delay values may involve determining a difference between an extracted left-ear delay and an extracted right-ear delay. In some such example embodiments, generating the modified left-ear delay values and right-ear delay values may involve determining a maximum expected difference between an extracted left-ear delay and an extracted right-ear delay. According to some example embodiments, a difference between an extracted left-ear delay and an extracted right-ear delay may be equal to a difference between a corresponding modified left-ear delay value and a modified right-ear delay value.

In some exemplary embodiments, the modified left-ear delay values and the modified right-ear delay values may correspond to a smoothing function. According to some exemplary embodiments, each pair of the modified left-ear delay values and the modified right-ear delay values may include a lower delay value and a higher delay value. In some examples, the lower delay value may have a smaller delay variation than the higher delay value. In some exemplary embodiments, the non-delayed pulse responses may be minimum phase filter responses.

According to some exemplary embodiments, extracting each left ear non-delayed pulse response, each right ear non-delayed pulse response, each left ear delay, and each right ear delay from each of the left ear and right ear HRTFs may involve: determining a frequency response of an original HRTF filter of the first set of HRTFs; determining an amplitude response of the original HRTF filter; and determining a minimum phase frequency response of a new non-delayed minimum phase filter. In some such exemplary embodiments, extracting each left-ear undelayed pulse response, each right-ear undelayed pulse response, each left-ear delay, and each right-ear delay from each of the left-ear and right-ear HRTFs may involve: determining a phase response of the original HRTF filter and a phase response of the new undelayed minimum phase filter; and determining a delay associated with the original HRTF filter based at least in part on the phase response of the original HRTF filter and the phase response of the new undelayed minimum phase filter. In some such exemplary embodiments, determining the minimum phase frequency response may involve performing a Hilbert transform on the amplitude response of the original HRTF filter. According to some example embodiments, determining the delay associated with the original HRTF filter may also be based at least in part on a delay measurement frequency in a range of 300 Hz to 1600 Hz.

In some exemplary embodiments, the set of elementary filters may have at least an order of magnitude fewer components than the second set of HRTFs. According to some exemplary embodiments, for frequencies above the threshold frequency, an all-pass phase response may deviate from a linearly ramped phase response and may be smoothed to near zero phase.

According to some example embodiments, the control system may correspond to at least a portion of a codec of an immersive voice and audio service (IVAS).

According to some further embodiments, a non-transitory computer-readable medium may store instructions that, when executed by one or more processors, cause the one or more processors to perform the operations of any of the methods disclosed herein.

According to some additional exemplary embodiments, an audio processor device may be configured to process input audio data. In some exemplary embodiments, the audio processor device may include a receiver unit configured to receive the input audio data and a computer unit. According to some exemplary embodiments, the computer unit may be configured to extract a first set of head-related transfer functions (HRTFs) and transform the first set of HRTFs into a second set of HRTFs. In some exemplary embodiments, the transformation may involve replacing a delayed component of the first set of HRTFs with an all-pass filter in the second set of HRTFs. In some exemplary embodiments, the transformation may involve adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that: for frequencies below a threshold frequency associated with the corresponding all-pass filter, each interaural phase response is substantially linear, and for frequencies above the threshold frequency associated with the corresponding all-pass filter, each interaural phase has a reduced interaural phase difference.

In some example embodiments, the computer unit may be configured to output the second set of HRTFs. According to some example embodiments, outputting the second set of HRTFs may involve storing the second set of HRTFs, transmitting the second set of HRTFs to a device configured to process audio data, providing the second set of HRTFs for further processing, or a combination thereof.

According to some exemplary embodiments, the computer unit may be further configured to define a set of basic filters based on the second set of HRTFs. The set of basic filters may have fewer components than the second set of HRTFs. In some exemplary embodiments, the computer unit may be further configured to receive a bit stream of input audio data in an input audio format and combine the input audio data with one or more basic filters in the set of basic filters to generate left audio data and right audio data.

In some exemplary embodiments, the computer unit may be further configured to output the left audio data and the right audio data. According to some exemplary embodiments, outputting the left audio data and the right audio data may involve storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing the left audio data and the right audio data to a set of speakers for playback by the control system, providing the left audio data and the right audio data for further processing, or a combination thereof.

According to some exemplary embodiments, the audio processor device may include a storage device configured to store the first HRTFs, the second HRTFs, the left audio data, the right audio data, the input audio data, or a combination thereof. In some such exemplary embodiments, the storage device may include a random access memory, a read-only memory, a non-transitory computer-readable medium, or a combination thereof.

In some example embodiments, the audio processor device may correspond to at least a portion of a codec of an immersive voice and audio service (IVAS).

The embodiments described herein may generally be described as techniques, where the term "technique" may refer to system(s), device(s), method(s), computer-readable instruction(s), module(s), component(s), hardware logic, and/or operation(s), as suggested by the context used herein.

Features and technical benefits other than those explicitly described above will become apparent upon reading the following [Implementation Methods] and examining the associated drawings. This [Invention Content] is provided to introduce a series of technologies in a simplified form and is not intended to identify the key or essential features of the claimed subject matter as defined by the accompanying patent applications.

100: Configuration

101: Equipment

105: Interface System

110: Control System

111: Pulse Response

115: Select memory system

120: Select microphone system

125: Select speaker system

130: Select sensor system

135: Select a display system

138: Delayed processing block

140: Delay

140L: Delay

140R: Delay

141: Central Processing Unit (CPU)

141L: New Delay

141R: New Delay

142: Read-Only Memory (ROM)

143: Random Access Memory (RAM)

144: Bus

145: Input/Output (I/O) Interface

146: Input unit

147: Output unit

148: Storage Unit

149: Communication unit

150:Driver

151: Removable Media

160: Electronic Processor

161:Memory

162:Encoding software

163: Decoding software

170: Immersive Voice and Audio Services (IVAS) Codec

171:IVAS Encoder

172: Spatial Encoder

173: Core Audio Codec

174:IVAS Decoder

175: Core Audio Decoder

176: Spatial Decoder/Renderer

180: median value

181:s Planar filter pole

182: Filter Pole

200: Head

211: Pulse Response

211L:L HRTF

211R:R HRTF

250: HRTF transformation sub-block

251: HRTF processing block

251L: Left HRTF processing block

251R: Right HRTF processing block

252: All-pass generator

252L: All-pass filter generator

252R: All-pass filter generator

253: Volume Procedure

253L: Left HRTF generation area

253R: Right HRTF generation area

272: Delayed processing block

273: Mapping Block

274: Transformation Block

275: Full-pass operation block

311: Non-delayed pulse response

311L: Non-delayed pulse response

311R: Non-delayed pulse response

411: Linear Phase Response

412: Alternative Phase Response

500: Configuration

501: Configuration

503: More detailed view

511:Causal All-Pass Filter

512: All-pass filter

512L: All-pass filter

512R: All-pass filter

520: Original HRTF Library

521: HRTF Transformation Block

522: Basic filter generation block

523: Basic Filter

524: Arrival Direction

525: Weighting coefficient generation block

526: Weighting coefficient

527: Weighting coefficient and basic filter combination block

528: Left ear HRTF filter

529: Right Ear HRTF Filter

530: Audio generation block

531: Audio signal

532: Audio input and basic filter combination block

533: Left ear audio signal

534: Right ear audio signal

541: Modified HRTF library

611: Left ear HRTF

711: Modified Right Ear HRTF

711L: Left ear HRTF

711R: Right Ear HRTF

771A: Delay

771B: Delay

771C: Delay

772A: Delay

772B: Delay

772C: Delay

801: X-axis

802: Y axis

803: Z axis

811: Pulse Response

911: Pulse Response

2100: Methods

2105: Block

2110: Block

2115: Block

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: FIG1A is a block diagram showing an example of components of an apparatus capable of implementing various aspects of the present invention; FIG1B is a schematic block diagram of an example device architecture that can be used to implement various aspects of the present invention; FIG1C is a schematic block diagram of an example CPU implemented in the device architecture of FIG1B that can be used to implement various aspects of the present invention; FIG1D is a schematic block diagram of an immersive voice and audio service (IVAS) for encoding and decoding an IVAS bit stream according to one or more embodiments. A block diagram of a codec (codec) framework; FIG1E is a diagram showing a Cartesian coordinate system centered on a listener's head; FIG2 is a graph showing a left-ear reference HRTF; FIG3 is a graph showing a right-ear reference HRTF; FIG4 is a graph showing a right-ear HRTF filter with delay removed; FIG5 is a graph showing the phase responses of two alternative filters; FIG6 is a graph showing the phase responses of two alternative filters; FIG7 is a graph showing a modified left-ear HRTF; Figure 8 shows a graph of a modified HRTF for the right ear; Figure 9 shows a graph of a HRTF for the left ear with an added delay; Figure 10 shows a graph of a modified HRTF for the right ear; Figure 11 shows a diagram of a modified HRTF filter; Figure 12 shows a diagram of the formation of a full-pass pulse response with an associated delay; Figure 13 shows the conversion of an original HRTF library into a more compact HRTF base set; Figure 14 shows a diagram of a compact HRTF base set for efficient HRTF computation. FIG15 is a diagram showing a compact HRTF basis set for processing a scene-based audio signal; FIG16 shows additional details of the HRTF transform block of FIG13 to FIG15 according to some embodiments; FIG17 shows additional details of the HRTF transform word block of FIG16 according to some embodiments; FIG18, FIG19, and FIG20 show examples of functions that may be implemented by the delay processing block of FIG17; and FIG21 is a flow chart summarizing an example of a method that may be performed by one of the apparatus or systems disclosed herein.

Cross-reference to related applications

This application claims priority to U.S. Provisional Application No. 63/455,539, filed on March 29, 2023, U.S. Provisional Application No. 63/595,752, filed on November 2, 2023, and U.S. Provisional Application No. 63/567,376, filed on March 19, 2024, the entire contents of which are incorporated herein by reference.

The present invention relates to creating a modified HRTF from an original HRTF, such that the modified HRTF can be more efficiently approximated by a linear blend while preserving the psychoacoustic properties of the original HRTF. Described herein are techniques for processing HRTF filters to produce modified HRTF filters suitable for use in a filter set based on linear interpolation. In the following description, for purposes of explanation, several examples and specific details are set forth to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention, as defined by the claims, may encompass some or all of the features of these examples, alone or in combination with other features described below, and may further encompass modifications and equivalents of the features and concepts described herein.

In the following description, various systems, devices, methods, programs, and processes are described in detail. Although certain steps may be described in a particular order, this order is for convenience and clarity only. A particular step may be repeated more than once, may occur before or after other steps (even if those steps would otherwise be described in a different order), and may occur concurrently with other steps. A second step may be performed after a first step only if the first step must be completed before starting the second step. When the context is unclear, this will be explicitly stated.

Throughout this document, the terms "and," "or," and "and/or" are used. These terms should be understood to have an inclusive meaning. For example, "A and B" may mean at least the following: "both A and B," "at least both A and B." As another example, "A or B" may mean at least the following: "at least A," "at least B," "both A and B," "at least both A and B." As another example, "A and/or B" may mean at least the following: "A and B," "A or B." When an exclusive or is intended, this will be explicitly stated (e.g., "A or B," "at most one of A and B").

The term "including" and its variations should be understood as open-ended terms meaning "including, but not limited to." The terms "one exemplary embodiment" and "an exemplary embodiment" should be understood to mean "at least one exemplary embodiment." The term "another embodiment" should be understood to mean "at least one other embodiment." The terms "determined," "determines," or "determining" should be understood to mean to obtain, receive, calculate, compute, estimate, predict, or infer. Furthermore, in the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art to which this invention belongs.

This document describes various processing functions associated with structures such as blocks, components, assemblies, and circuits. Generally, these structures can be implemented by a processor controlled by one or more computer programs.

Various abbreviations may be used throughout the present invention and in the associated patent claims and/or drawings, as listed below. Other commonly used abbreviations and technical terms may be excluded from this list for the sake of brevity. Therefore, a brief list of abbreviations is provided below for the reader's reference.

IVAS - Immersive Voice and Audio Services

HRTF-Head Related Transfer Function

LPC - Linear Predictive Coding

CLDFB-Complex Low Delay Filter Bank

SBA-Scene-Based Audio

SPAR - Spatial Reconstruction, a spatial audio coding technique

DirAC - Directional Audio Coding, another spatial audio coding technology

MD-Metadata

BS-Bitstream

HOA-High-end Stereo Speakers

FOA-First-order stereo speakers

MDFT-Modified Discrete Fourier Transform

MDCT-Modified Discrete Cosine Transform

FIG1A is a block diagram illustrating an example of components of a device capable of implementing various aspects of the present invention. As with other figures provided herein, the types and numbers of components shown in FIG1A are provided by way of example only. Other embodiments may include more, fewer, and/or different types and numbers of components. According to some examples, device 101 may be or may include a device configured to perform at least some of the methods disclosed herein, such as a smart audio device, a laptop, a cellular phone, a tablet device, a smart home hub, etc. In some such embodiments, device 101 may be or may include a server configured to perform at least some of the methods disclosed herein.

In this example, apparatus 101 includes an interface system 105 and a control system 110. In some embodiments, interface system 105 can be configured to provide a first set of HRTFs to control system 110. In some examples, interface system 105 can be configured to output one or more results of control system 110 processing the first set of HRTFs (e.g., a second set of HRTFs, a set of basic filters based on the second set of HRTFs, audio data processed by one or more of the basic filters (e.g., left ear audio data and right ear audio data), etc.).

The interface system 105 may include one or more network interfaces and/or one or more external device interfaces (such as one or more Universal Serial Bus (USB) interfaces). According to some embodiments, the interface system 105 may include one or more wireless interfaces. The interface system 105 may include one or more devices for implementing a user interface, such as one or more microphones, one or more speakers, a display system, a touch sensor system, and/or a gesture sensor system. In some examples, the interface system 105 may include one or more interfaces between the control system 110 and a memory system (such as the optional memory system 115 shown in FIG. 1A ). However, in some examples, control system 110 may include a memory system.

The control system 110 may, for example, include a general purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and/or discrete hardware components.

In some embodiments, control system 110 may reside on more than one device. For example, a portion of control system 110 may reside on a device within an environment (e.g., a laptop, a tablet, a smart audio device, etc.) and another portion of control system 110 may reside on a device outside the environment (e.g., a server). In other examples, a portion of control system 110 may reside on a device within an environment and another portion of control system 110 may reside on one or more other devices within the environment.

In some implementations, the control system 110 can be configured to at least partially perform the methods disclosed herein. According to some examples, the control system 110 can be configured to receive a first set of HRTFs and transform the first set of HRTFs into a second set of HRTFs. The second set of HRTFs can be more efficiently approximated than the first set of HRTFs by linear blending while preserving the psychoacoustic properties of the first set of HRTFs. In some such examples, the transformation can involve replacing the delayed components of the first set of HRTFs with all-pass filters in the second set of HRTFs. According to some such examples, the transformation may involve adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that: for frequencies below a threshold frequency associated with the corresponding all-pass filter, each interaural phase response is substantially linear, and for frequencies above the threshold frequency associated with the corresponding all-pass filter, each interaural phase has a reduced interaural phase difference.

In some examples, control system 110 may be configured to define a set of basic filters based on the second set of HRTFs. The basic filter set may have fewer components than the second set of HRTFs. In this context, a "component" of the second set of HRTFs is one of the HRTFs in the second set of HRTFs. Similarly, a "component" of the basic filter set is one of the basic filters of the basic filter set. According to some examples, the basic filter set may have at least an order of magnitude fewer components than the second set of HRTFs. For example, in some examples, the second set of HRTFs may have hundreds or thousands of components, while the basic filter set may include fewer than 100 components, fewer than 50 components, or even fewer than 20 components.

According to some examples, the control system 110 can be configured to receive a bit stream of input audio data in an input audio format via the interface system 105. The input audio format can be, for example, a stereo audio format, an audio object-based audio format (such as Dolby Atmos ^™ ), a channel-based audio format, etc. In some examples, the control system 110 can be configured to combine the input audio data with one or more basic filters of the basic filter set to generate left audio data and right audio data, such as left ear audio data and right ear audio data. In some such examples, the control system 110 can be configured to output the left audio data and the right audio data via the interface system 105. Outputting the left audio data and the right audio data may involve storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing the left audio data and the right audio data to a set of speakers for playback, providing the left audio data and the right audio data for further processing, or a combination thereof.

In some examples, the control system 110 can be configured to implement at least a portion of a codec for an immersive voice and audio service (IVAS). Some examples are described herein with reference to FIG. 23 .

Some or all of the examples described herein may be executed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, and the like. The one or more non-transitory media may, for example, reside in the optional memory system 115 and/or the control system 110 shown in FIG. 1A . Thus, various innovative aspects of the subject matter described herein may be implemented in one or more non-transitory media on which software is stored. The software may, for example, include instructions for controlling at least one device to process audio data. The software may, for example, be executed by one or more components of a control system, such as control system 110 in FIG. 1A .

In some examples, device 101 may include the optional microphone system 120 shown in FIG. 1A . Optional microphone system 120 may include one or more microphones. In some implementations, one or more of the microphones may be part of or associated with another device (e.g., a speaker in a speaker system, a smart audio device, etc.).

According to some embodiments, device 101 can include optional speaker system 125, as shown in FIG1A . Optional speaker system 125 can include one or more speakers. A speaker is sometimes referred to as a "speaker." In some examples, at least some of the speakers of optional speaker system 125 can be positioned arbitrarily. For example, at least some of the speakers of optional speaker system 125 can be placed in positions that do not correspond to a speaker layout specified by any standard (e.g., Dolby 5.1, Dolby 5.1.2, Dolby 7.1, Dolby 7.1.4, Dolby 9.1, Hamasaki 22.2, etc.). In some of these examples, at least some of the speakers of the selected speaker system 125 may be placed in spatially convenient locations (e.g., locations where there is room to accommodate the speakers) rather than in any standard prescribed speaker layout.

In some embodiments, device 101 may include the optional sensor system 130 shown in FIG1A . Optional sensor system 130 may include a touch sensor system, a gesture sensor system, one or more cameras, etc.

In some embodiments, device 101 may include optional display system 135, shown in FIG1A . Optional display system 135 may include one or more displays, such as one or more light-emitting diode (LED) displays. In some examples, optional display system 135 may include one or more organic light-emitting diode (OLED) displays. In some examples where device 101 includes display system 135, sensor system 130 may include a touch sensor system and/or a gesture sensor system proximate to one or more displays of display system 135. According to some such implementations, the control system 110 can be configured to control the display system 135 to present a graphical user interface (GUI), such as a GUI associated with implementing one of the methods disclosed herein.

FIG1B illustrates a schematic block diagram of an exemplary device architecture 101 (in this example, a device 101) that may be used to implement various aspects of the present invention. The device 101 of FIG1B is an example of the device 101 of FIG1A. The architecture 101 includes, but is not limited to, server and client devices, systems, and the like that may be configured to perform the methods described with reference to any or all of FIG11 through FIG17 and FIG21. As shown, the architecture 101 includes a central processing unit (CPU) 141 that is capable of executing various programs based on a program stored in, for example, a read-only memory (ROM) 142 or a program loaded from, for example, a storage unit 148 into a random access memory (RAM) 143. CPU 141 can be, for example, an electronic processor. In these examples, CPU 141 is an example of control system 110 in FIG. 1A , and ROM 142 and RAM 143 are examples of memory system 115 . RAM 143 also stores data required by CPU 141 to execute various programs, as needed. CPU 141, ROM 142, and RAM 143 are connected to each other via bus 144 . Input/output (I/O) interface 145 is also connected to bus 144 . Bus 144 and I/O interface 145 are examples of interface system 105 in FIG. 1A .

The following components are connected to the I/O interface 145: an input unit 146, which may include a keyboard, a mouse, or the like; an output unit 147, which may include a display, such as a liquid crystal display (LCD) and one or more speakers; a storage unit 148, which may include a hard drive or another suitable storage device; and a communication unit 149, which may include a network interface card, such as a network card (e.g., wired or wireless).

In some implementations, the input unit 146 includes one or more microphones located in different locations (depending on the host device) that enable capture of audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

In some embodiments, output unit 147 includes a system with a varying number of speakers. Output unit 147 (depending on the capabilities of the host device) can present audio signals in a variety of formats, such as mono, stereo, immersive, binaural, and other suitable formats.

In some embodiments, communication unit 149 is configured to communicate with other devices (e.g., via a network). Drive 150 is also connected to I/O interface 145 as needed. Removable media 151 (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a flash drive, or another suitable removable medium) is mounted on drive 150, allowing a computer program read therefrom to be installed into storage unit 148 as needed. Those skilled in the art will understand that although device 101 is described as including the aforementioned components, in actual applications, some of these components may be added, removed, and/or replaced, and all such modifications or changes fall within the scope of the present invention.

According to exemplary embodiments of the present invention, the aforementioned procedures may be implemented as a computer software program or on a computer-readable storage medium. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the method. In such embodiments, the computer program may be downloaded and installed from a network via communication unit 149 and/or installed from removable media 151, as shown in FIG. 1B .

FIG1C illustrates a schematic block diagram of an exemplary CPU 141 implemented in the device architecture 101 of FIG1B , which may be used to implement various aspects of the present invention. CPU 141 includes an electronic processor 160 and a memory 161 . Electronic processor 160 is electrically and/or communicatively coupled to memory 161 for bidirectional communication. Memory 161 stores encoding software 162 and decoding software 163 . Memory 161 may be, for example, a ROM, RAM, or another non-transitory computer-readable medium. Electronic processor 160 may implement encoding software 162 stored in memory 161 to perform, among other things, method 2100 of FIG21 . Additionally, the electronic processor 160 may implement decoding software 163 stored in the memory 161 to perform the methods described with reference to, among other things, any or all of Figures 11 to 17 and 21.

In general, various exemplary embodiments of the present invention may be implemented in hardware or dedicated circuitry (e.g., control circuitry), software, logic, or any combination thereof. For example, the units discussed above may be executed by the control circuitry (e.g., CPU 141 in combination with other components of FIG. 1B ), and thus, the control circuitry may perform the actions described herein. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device (e.g., control circuitry). Although various aspects of exemplary embodiments of the present invention may be depicted and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

Additionally, the various blocks shown in the flowcharts may be viewed as method steps, and/or operations resulting from the operation of computer program code, and/or as a plurality of coupled logic circuit elements constructed to perform (some) related functions. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code configured to perform the method described above.

In the context of this invention, a machine-readable medium is any tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be non-transitory and can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include the following: an electrical connection having one or more wires, a portable computer disk, a hard drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Computer program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. Such computer program code can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device having a control circuit system, so that when executed by the computer or other programmable data processing device, the program code causes the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code can be packaged as a separate software and executed entirely on one computer, partially on one computer, partially on one computer and partially on a remote computer, or entirely on a remote computer or server, or distributed across one or more remote computers and/or servers.

Example IVAS codec framework

FIG1D is a block diagram of an Immersive Voice and Audio Service (IVAS) codec/decoder (“codec”) framework 170 for encoding and decoding an IVAS bitstream, according to one or more embodiments. IVAS is expected to support a range of audio service capabilities, including, but not limited to, mono-to-stereo upmixing and fully immersive audio encoding, decoding, and presentation. IVAS is also expected to be supported by a wide range of devices, terminals, and network nodes, including, but not limited to, mobile and smartphones, tablets, personal computers, conference phones, meeting rooms, virtual reality (VR) and augmented reality (AR) devices, home theater devices, and other suitable devices.

In this example, the IVAS codec 170 includes an IVAS codec 171 and an IVAS decoder 174. In some examples, the IVAS codec 171, the IVAS decoder 174, or both can be implemented by one or more instances of the control system 110 of FIG. 1A , the CPU 141 of FIG. 1B and FIG. 1C , or the like. In some examples, the IVAS codec 171 can be implemented by the encoding software 162 of FIG. 1C , and the IVAS decoder 174 can be implemented by the decoding software 163 of FIG. 1C . According to some examples, a control system implementing the IVAS codec 171, the IVAS decoder 174, or both can also be configured to perform some or all of the operations disclosed herein, such as the methods described with reference to one or more of FIG. 11 to FIG. 17 and FIG. 21 .

According to this example, IVAS encoder 171 includes a spatial encoder 172 that receives N channels of input spatial audio (e.g., FOA, HOA). In some embodiments, spatial encoder 172 can be configured to implement spatial reconstruction (SPAR), directional audio coding (DirAC), another spatial audio coding technique, or a combination thereof. In this example, the output of spatial encoder 172 includes a spatial metadata (MD) bitstream (BS) and a spatial downmix of N_dmx channels. According to this example, the spatial MD is quantized and entropy coded. In some embodiments, quantization can include fine, medium, coarse, and extra-coarse quantization strategies, and entropy coding can include Huffman or arithmetic coding. In some implementations, the framework may allow no more than three quantization levels in a given operating mode; however, as the bit rate decreases, in some such implementations, the three levels generally become coarser to meet the bit rate requirements. According to this example, the core audio encoder 173, which may be based on a mono Enhanced Voice Service (EVS) coding unit, for example, is configured to encode the N_dmx channels ( N_dmx = 1 to 16 channels) of the spatial downmix into an audio bitstream, which is combined with the spatial MD bitstream into an IVAS-encoded bitstream that is transmitted to the IVAS decoder 174.

In this example, the IVAS decoder 174 includes a core audio decoder 175 (e.g., an EVS decoder) that decodes the audio bitstream extracted from the IVAS bitstream to recover the N_dmx audio channels. According to this example, the spatial decoder/renderer 176 (e.g., SPAR/DirAC) decodes the spatial MD bitstream extracted from the IVAS bitstream to recover the spatial MD, and uses the spatial MD and a spatial upmix to synthesize/render the output audio channels for playback on various audio systems with different speaker configurations and capabilities.

FIG1E shows an example of a coordinate system with reference to a listener's head. A head-related transfer function (HRTF) filter can be used to process audio signals to generate binaural audio signals, providing a listener with the illusion that sounds are arriving from a specified arrival direction. The arrival direction can be defined by an ( x, y, z ) unit vector, where Cartesian coordinates can be defined as shown in FIG1E . According to the example shown in FIG1E , the origin of the coordinate system is approximately at the center of the listener's head 200, with the X-axis 801 pointing forward (in the direction of the listener's nose), the Y-axis 802 pointing to the left of the listener, and the Z-axis 803 pointing upward toward the top of the listener's head.

An audio signal s ( t ) can be processed using HRTF filters to provide the listener with the illusion of a sound arriving from a direction defined by a free unit vector ( x ,y,z ). This process generates two ear signals , el ( _t ) and er ( _t ) _, by convolving the input audio _{signal with each of a pair of HRTF filters, hl} ( t ) and hr (t ) :

The HRTF filters ( hl ( _t ) and hr ₍ t )) can be derived from the direction vector ( x,y,z ) as follows: ( hl ( _t ) ,hr ₍ t ))← H ( x,y,z ) (3)

In this paper, H ( x,y,z ) is referred to as a HRTF set function because it is suitable for computing HRTF filters for a set of ( x,y,z ) direction vectors. The set of ( x,y,z ) vectors for which the HRTF set function generates valid HRTF filters is referred to as the domain of the HRTF set function.

In the explanations given below, time-domain pulse responses are used to represent filter responses. Those skilled in the art will appreciate that equivalent storage and manipulation of filter responses can be implemented in other domains, including (but not limited to) the frequency domain.

An HRTF set function can be used to create a discrete library of HRTFs that defines the left and right ear HRTF responses for a set of N ( x,y,z ) unit vectors:

And when evaluating the HRTF set function in Equation 4, the HRTF discrete library can be written as:

It is desirable to provide a component for defining a HRTF set function, whereby each output HRTF filter generated by the HRTF set function _is formed by a linear combination of elementary filters. A linear HRTF set function can be defined according to Equation 6, where _the el ( t ) and er ( t ) filters are calculated as:

According to Equation 6, a set of K left-ear basic filters ( t ) and K right ear basic filters ( t ) and the gain function ( x,y,z ) and A weighted linear combination defined by ( x, y, z ).

In an alternative embodiment, a symmetric HRTF set function can be defined using a smaller set of basic filters and gain functions (where the left ear HRTF filter for direction ( x, y, z ) is the same as the right ear HRTF for direction ( x, -y , z )):

Without loss of generality, we can examine the first line of Equation 7, understanding that the following explanation will also apply to the second line of Equation 7 and/or Equation 6.

For a set of N arrival directions (( xn _, yn , zn ₎ , n = 1.. N ), we can rewrite _the first line of Equation 7 in matrix form (also omitting the subscript l in e _l ( t ) to simplify the equation):

We rewrite Equation 8 in a simpler form: E ( t ) = G × B ( t ) (9)

In Equation 9, the row vector E ( t ) defines a set of N left- _ear HRTF filter responses of N unit vectors ( ₍ xn , yn _, zn ) , n = 1..N), and the row vector B ( t ) defines a set of K filter responses. In some embodiments, a goal is to determine the filter responses B ( t ) such that the resulting HRTF filter E ( t ) is a close approximation of the set of original HRTF filter responses Eorig ( t ₎ .

Various methods for determining a suitable filter B ( t ) are known, and one example is found as follows: B ( t ) = G ⁺ × Eorig ( t ) (10) where G ⁺ refers to the pseudo-inverse of the matrix G (as defined in Equation ₉ ).

It will be appreciated that other methods may be employed, wherein the objective of each method may be to minimize the magnitude of the difference E ( t ) - Eorig ( t ₎ .

A large number ( K ) of elementary filters may be required to provide a reasonable approximation ( E ( t ) E _orig ( t )). The difficulty with using a linear mixing procedure (according to Equations 6, 7, or 8) is that the high-frequency components of the HRTF filter can often be difficult to define using linear mixing.

In some embodiments, an original HRTF filter set E _orig ( t ) is modified to produce a modified HRTF filter set E _mod ( t ) , where the modified HRTF filters differ from the original filters in their phase response at high frequencies. For each of the N directions, we can use a Fourier transform to define the frequency response of the original HRTF and the modified HRTF: R _orig,n ( f ) = F { E _orig,n ( t )} R _mod,n ( f ) = F { E _mod,n ( t )} (11)

The frequency response functions R _orig,n ( f ) and R _mod,n ( f ) are complex-valued, and so we can then say: Such that for frequencies less than Fp _Hz , the modified filter closely matches the original filter, and the amplitude of the modified filter closely matches the original filter at higher frequencies. In various non-limiting examples, the transition frequency Fp can be equal to about 1200 Hz, and can generally _be in a range of, for example, 1000 Hz. Fp < 3000 Hz . In some applications, it may be desirable to reduce the number of basis functions ( K ) and may be necessary _to allow values _of Fp less than 1000 Hz, such as 950 Hz, 900 Hz, 850 Hz, 800 Hz, 750 Hz, 700 Hz, 650 Hz, 600 Hz, 550 Hz, or as low as 500 Hz. In other example applications, the transition frequency may be in another range greater than 3000 Hz, such as, for example, 3050 Hz, 3100 Hz, 3150 Hz, 3200 Hz, 3250 Hz, 3300 Hz, etc.

Figures 2, 3, and 4 show examples of pulse responses of HRTF filters. Figure 2 shows the pulse response 111 of a left ear HRTF filter for the arrival direction: (x, y, z ) = (This is a direction of the front left.) Similarly, FIG3 shows the impulse response 211 of the right ear HRTF for the same arrival direction. From inspection of FIG3 , it can be seen that the impulse response 211 includes a delay of 0.4 ms.

Figure 4 shows a (delay-free) pulse response 311 created by removing a 0.4 ms delay from the pulse response 211 in Figure 3 . Figure 5 shows an example of a graph indicating phase response versus frequency. The 0.4 ms delay in Figure 3 can also be defined as a linear phase response 411 in Figure 5 . Additionally, an alternative phase response 412 is plotted in Figure 5 , whereby this alternative phase curve 412 closely matches the linear phase response 411 for frequencies between 0 and 1400 Hz.

In some embodiments, the original pulse response 211 of FIG3 can be modified by removing the large delay of 0.4 ms to produce the delay-free pulse response 311 of FIG4, and the phase response 412 of FIG5 can be applied to the pulse response 311 to produce a new filter pulse response with the correct phase response for frequencies below 1400 Hz. Unfortunately, this can result in a new pulse response that is non-causal, because in order to implement this filter in a real-time audio program, an additional delay of 3 ms can be added to produce the pulse response: for example, see the pulse response 911 shown in FIG10. To maintain compatibility with the right ear response, this example pulse response 111 (left ear response) will also need to have a 3ms delay added, resulting in the pulse response 811 of FIG9 .

Some disclosed examples involve modifying the original HRTF filters for both the left and right ears to provide an interaural phase difference similar to the interaural phase difference shown in the phase response 412 of FIG. 5 , without the side effect of an undesirable delay (e.g., the 3 ms delay discussed above with reference to FIG. 9 and FIG. 10 ).

FIG6 shows an example of a causal all-pass filter. FIG7 and FIG8 show examples of modified HRTFs that can be generated by the causal all-pass filter. In some embodiments, the causal all-pass filters 511 and 512 of FIG6 can be applied to the original left-ear and right-ear impulse responses 111 and 311, respectively, to generate the modified left-ear HRTF 611 of FIG7 and the modified right-ear HRTF 711 of FIG8, respectively.

FIG11 illustrates an example of an HRTF processing block. According to some examples, the blocks of FIG11 can be implemented by the control system 110 of FIG1A , for example, according to instructions stored on a computer-readable medium. In FIG11 , configuration 100 illustrates a raw HRTF pulse response 211 h ( t ), which is received and processed by HRTF processing block 251 to determine a large delay 140 d (the delay inherent in pulse response 211). According to this example, HRTF processing block 251 also generates a delay-free HRTF h' ( t ), such that h' ( t ) = h ( t + d ).

In the example shown in FIG11 , the all-pass generator 252 generates the all-pass filter 512 α ( t ) in response to a delay of 140 d , and the convolution process 253 combines the undelayed pulse response 311 and the all-pass filter 512 to generate the modified right-ear HRTF 711 m ( t ).

The all-pass filter 512 α ( t ) can be defined as a function as follows: α ( t ) = C ( d,t ) (13) where the function C ( d,t ) defines the operation of the all-pass generator 252. We are interested in the phase response of C ( d,t ): Φ ( f,d ) = arg( F { C ( d,t )}( f )) (14) where Φ ( f,d ) represents the phase response of the all-pass filter generated by the all-pass generator 252 at frequency f for a delay value d .

Let us define Φ ₀ ( t ) = arg( F { C ( 0 , t ) } ( f )), which is the all-pass phase response generated by all-pass generator 252 at delay d = 0. We may refer to this as zero-delay all-pass. In some embodiments, we may need the all-pass phase response to satisfy:

The left side of Equation 15 represents the phase difference between a zero-delay all-pass filter and an all-pass filter defined for a delay d . This phase difference is equivalent to the phase response 412 of Figure 5 . The right side of Equation 15 represents the expected linear phase ramp for a delay d . This is equivalent to the linear phase response 411 of Figure 5 .

Thus, Equation 15 expresses the requirement that, in this example, the all-pass phase response 412 should match the linear phase response 411 for frequencies up to Fp _. Furthermore, Equation 15 defines an upper limit dmax _for the range of delay values within which the all-pass generator function C ( d,t ) is expected _to produce valid results. A typical _{value for} dmax is dmax = 0.7 ms (milliseconds), but in some applications, dmax can be some other value, such as a value between 0.6ms and 0.8ms, a value between 0.5ms and 0.8ms, a value between 0.6ms and 0.9ms, a value between 0.5ms and 1.0ms, and so on.

In some embodiments, a finite set _of M delay values ( d1 , d2 _, ... , dM ₎ is selected spanning the range _from 0 to dmax , and suitable full - pass responses ( α1 ( _t ) , α2 _, ... ( t ) , αM ₍ t )) can be pre-calculated according to an optimization procedure. In this case, the full-pass generator function C ( d,t ) can be implemented by using a lookup table or an interpolation function using the M pre-stored full-pass responses.

In some further embodiments, each of the all-pass filters (e.g., the m-th all-pass filter α _m ( t )) can be defined as an infinite impulse response (IIR) filter having T pairs of conjugated poles and their corresponding pairs of conjugated zeros. A basic set of T filter poles p _{1 ,m} , p _{2 ,m} , ... , d _T,m can be selected, and then the filter α _m ( t ) can be defined as having poles ( p _{1 ,m} , ,p _{2 ,m} , , ... ,p _T,m , ) and zero point ( , , , , ... , , ) is an all-pass filter.

Therefore, when the basis set of all M all-pass responses ( α ₁ ( t ) ,α ₂ , ...( t ) ,α _M ( t )) is defined as an IIR filter of order 2 T , the set of M all-pass responses is fully defined in terms of [ T × M ] complex basis poles: The complex values of the matrix P in Equation 16 can be derived by an optimization procedure (such as the MATLAB FMINSEARCH function). In some examples, the optimization procedure can be guided by first computing a cost function (also referred to herein as an error function) for the all-pass filters ( α ₁ ( t ) , α ₂ , ... ( t ) , α _M ( t )) from the basis of the matrix P , then computing the corresponding phase responses ( Φ ₁ ( f ) , Φ ₂ ( f ) , ... , Φ _M ( f )) according to Equation 14, and then measuring how well the relative phase differences between all pairs of all-pass filters match the expected delay differences. For example, the error function can be defined as:

Given a matrix P representing the basis of a set of M all-pass filters (each all-pass filter has T complex basis poles) and a corresponding set of delay values ( d ₁ , d ₂ , ... , d _M ), a polynomial approximation can be formed such that the basis can be defined as a polynomial function of d . This polynomial approximation procedure can be implemented according to known methods, including (but not limited to) the MATLAB POLYFIT function.

In some embodiments, the number of complex bases is T = 3, and a 4th-degree polynomial can be used to calculate the base value as a function of the delay d . According to this embodiment, the process for calculating an all-pass filter α ( t ) is implemented by the following sequence of operations:

_1. Given a delay d , calculate the basis ( p1 , p2 _, p3 ₎ as follows : pt = _c1 , _t + c2 _{, td} + c3 _{, td2} + ^c4 _, td3 ⁺ c5 _, ^td4

2. Form a base p1 _, ,p ₂ , ,p ₃ , One of the IIR filters, and the corresponding zero point: , , , , ,

3. Calculate the pulse response of the IIR filter to form the all-pass response α ( t )

The three steps presented above demonstrate the use of polynomial functions as a convenient way to compute the poles of a filter. Of course, a polynomial can only give an approximation to the "best" poles, and it is well known that small errors in the pole locations can lead to large changes in the resulting filter response. Some alternative methods involve applying a nonlinear function (such as Equation 18) to the polynomial values (J ₁ and J ₁₊₁ ). This nonlinear function can be defined so that small errors in the polynomial values (J ₁ and J ₁₊₁ ) will no longer lead to large errors in the pole locations. In addition, some of these examples involve computing the pole locations in the "s-domain" and then mapping them to the "z-domain." According to some examples, this mapping can be implemented using the MATLAB function "bilinear." Other choices of nonlinear mapping functions can be used, and these nonlinear mapping functions can produce poles in the z-domain, s-domain, or other domains.

FIG12 illustrates an additional transformation process that may be implemented by the all-pass generator 252 of FIG11 , according to some embodiments. In the example shown in FIG12 , the all-pass generator 252 receives a selected delay d 140 that is processed by the delay processing block 272 to generate a set of intermediate values 180. In this example, the intermediate values 180 are then mapped by additional nonlinear processing to form a set of filter poles 182. The filter poles 182 are then processed by the all-pass operation block 275 to form the all-pass filter 512 α ( t ) 512 of FIG11 and FIG12 .

According to some examples corresponding to the blocks shown in FIG12 , delay processing block 272 applies a polynomial function as described above to output a set of intermediate values 180. In some examples, intermediate values 180 may be a set of numbers: J ₁ . In some such examples, mapping block 273 applies a nonlinear mapping procedure to the set of intermediate values 180, for example by implementing Equation 18, to produce an output. In one example, the output is an s-plane filter pole. In some examples, bilinear transform block 174 converts the s-domain pole positions to produce an output. In one example, the output includes the z-domain pole positions (filter poles 182). In the example shown in Figure 12, the all-pass operation block 275 operates on the output, which in this example is the output of the all-pass filter 512, α(t) (an impulse response). In alternative examples, the output can be a phase response, a frequency response, or any other method we can use to define the output of an all-pass filter response.

When a simple function such as a polynomial is used to generate filter poles, as those skilled in the art will appreciate, small errors in the polynomial output can translate into large errors in the final all-pass response when the poles are very close to the unit circle. Nonlinear processing, such as that employed in FIG12 to transform the median value 180 into the filter pole 182, enables more efficient implementation of the processing in delay processing block 272.

In some embodiments, the processing of the delay processing block 272 is implemented as a set of L polynomial functions that generate the L median values 180. For example, J _l = Poly _l ( d )( l 1.. L ). The intermediate values 180 can then be used to generate s-plane filter poles 181 by mapping block 273 in this example. For example, two intermediate values J _l and J _{l + 1} can be used to define a complex s-plane pole P _n :

Alternatively, a single intermediate value (e.g., J _l ) may be used by mapping block 273 to define a single true s-plane pole P _n , according to P _n =-2π J _l .

The s-plane filter pole 181 can then be transformed (in this example, by transform block 274) to the z-plane pole (filter pole 182). For example, MATLAB BILINEAR can be used by transform block 274 to apply this transformation: P(N) = bilinear(P(N), 1, 1, 48000), or this transformation: P(N) = bilinear(P(N), 1, 1, 48000, FP), where Fp represents the upper frequency limit (as used in Equation 15) and 48000 is the sampling rate according to the present embodiment. It should be understood that alternative sampling rates can be used, including (but not limited to) 16000, 32000, 44100, or 96000.

Those skilled in the art will appreciate that other nonlinear processing methods can be employed to facilitate mapping a selected delay d 140 to a set of all-pass poles (filter poles 182). In an alternative embodiment, the polynomial function applied by the delay processing block 272 can be used to define the frequency and Q of the poles, and the nonlinear mapping procedure applied by the mapping block 273 can convert the frequency and Q values to a respective pole location. In another embodiment, the nonlinear mapping procedure applied by the mapping block 273 can determine the z-domain pole locations, eliminating the need for the bilinear transformation of the transformation block 274.

It will also be appreciated that by forming additional conjugate poles (for each complex pole in the set of filter poles 182) and by forming each filter zero as the inverse of each corresponding pole, an all-pass filter response can be derived by the all-pass operation block 275 in this example, and this all-pass filter will be causal.

FIG13 illustrates a process for generating a set of basic filters from a set of HRTFs. In some examples, the blocks of FIG13 may be implemented, at least in part, by the control system 110 of FIG1A . FIG13 shows a configuration 500 in which an original HRTF library 520 is processed (in this example, by an HRTF transformation block 521) to generate a modified HRTF library 541. In this example, the interaural delay components inherent in the HRTF filters of the original HRTF set are replaced by all-pass filters that satisfy _Equation 15, and the modified HRTF library reduces the interaural phase at frequencies greater than Fp . The modified HRTF library 541 (in this example, from the basic filter generation block 522) is processed according to a fitting procedure such as that of Equation 10 to generate a set of basic filters 523.

The set of base filters 523 has fewer components than the modified set of HRTFs. In this context, a "component" of the set of base filters 523 is one of the base filters of the set of base filters 523, and a "component" of the modified set of HRTFs is one of the HRTFs in the modified set of HRTFs. According to some examples, the set of base filters 523 may have at least an order of magnitude fewer components than the modified set of HRTFs. For example, in some examples, the modified set of HRTFs may have hundreds or thousands of components, while the set of base filters 523 may include fewer than 100 components, fewer than 50 components, or even fewer than 20 components. Thus, the set of base filters 523 forms a compact representation of the original HRTF library 520.

FIG14 illustrates a process for generating a set of basic filters from a set of HRTFs and forming left and right HRTF filters using the set of basic filters. In some embodiments, the blocks of FIG14 may be implemented at least in part by the control system 110 of FIG1A. FIG14 shows a configuration 501 in which an original HRTF library 520 is processed by an HRTF transformation block 521 to generate a modified HRTF library 541, which is then processed by a basic filter generation block 522 to generate a set of basic filters 523. An arrival direction 524 (which may be based on spherical coordinates (θ , ), unit vector ( x, y, z ), or other forms known in the art) are processed by weighting coefficient generation block 525 to form weighting coefficients 526. In some embodiments, the weighting coefficients can be defined according to the spherical harmonic translation equation, and the basic filter can also be adapted to be compatible with the spherical harmonic translation equation, for example, g _k ( x, y, z ) in Equation 7.

In this example, weighting coefficient and base filter combination block 527 combines weighting coefficient 526 with base filter 523 to form left-ear and right-ear HRTF filters (528 and 529, respectively) that represent the modified HRTF for a specified direction of arrival. When the base filters represent a symmetric HRTF set, weighting coefficient and base filter combination block 527 can be implemented, for example, according to Equation 7. When the base filters represent an HRTF set that includes asymmetry, weighting coefficient and base filter combination block 527 can be implemented, for example, according to Equation 6.

FIG15 illustrates a process for generating a set of basic filters from a set of HRTFs and forming left and right audio signals using the set of basic filters. In some examples, the blocks of FIG15 may be implemented, at least in part, by the control system 110 of FIG1A . FIG15 shows a configuration 502 in which an original HRTF library 520 is processed by an HRTF transformation block 521 to generate a modified HRTF library 541, which is then processed by a basic filter generation block 522 to generate a set of basic filters 523. According to this example, an audio generation block 530 generates an audio signal 531 in a form associated with a scene-based audio format, such as dubbing or high-order dubbing. The audio generation block 530 may be or may include an audio decoder adapted to generate a multi-channel audio bitstream from a transmitted or stored encoded bitstream. Alternatively, the audio generation block 530 may be or may include an audio capture and/or processing device adapted to generate a scene-based audio signal 531 representing a spatial audio scene.

According to this example, audio input and basic filter combination block 532 is adapted to combine audio signal 531 with basic filter 523 to generate left and right ear audio signals ( 533 , 534 , respectively). In some examples, audio input and basic filter combination block 532 can be configured to implement a convolution process, which can be implemented based on time-domain or frequency-domain methods known in the art.

FIG16 shows additional details of the HRTF transformation block of FIG13-15 according to some embodiments. In some examples, the block of FIG16 may be implemented at least in part by the control system 110 of FIG1A . FIG16 shows a more detailed view 503 of the process in the upper half of FIG13-15 (converting from a "raw" HRTF library 520 to a "modified" HRTF library 541 ). According to this example, each left/right HRTF pair is processed by a corresponding HRTF transformation sub-block 250 .

FIG17 shows additional details of the HRTF transformation sub-block of FIG16 according to some embodiments. In some examples, the block of FIG17 can be implemented at least in part by the control system 110 of FIG1A. FIG17 shows an example of the HRTF transformation sub-block 250, in which the L and R HRTFs (211L and 211R) are processed by the left HRTF processing block 251L and the right HRTF processing block 251R, respectively, to extract the undelayed pulse response 311L/R and the delay 140L/R. The two delays 140L/R are then processed by the delay processing block 138 to generate new simplified delays 141L/R. The simplified delay is processed by all-pass filter generators 252L and 252R to form all-pass filters 512L/R. According to this example, the modified left and right HRTF generation blocks 253L and 253R are configured to combine the non-delayed pulse response 311L/R with the all-pass filter 512L/R to form a modified HRTF pair 711L/R.

Instance-delayed definition of each ear

The delay processing block 138 of FIG17 is configured to respond to the difference between delays 140L/R by generating a new delay 141L/R. FIG18, FIG19, and FIG20 illustrate examples of functions that can be implemented by the delay processing block 138 of FIG17. For FIG18, FIG19, and FIG20, the corresponding functions for modifying delays d' _L 771A, 771B, 771C and d' _R 772A, 772B, 772C are: Based on FIG18: According to Figure 19: d' _L =max(0 ,d _L - d _R ) d' _R =max(0 ,d _R - d _L ) (20) According to Figure 20:

In Equation 21, d _max represents the maximum expected value of | d _L - d _R |.

An important property of the function implemented by the delay processing block 138 is that it produces modified delays d' _L and d' _R that satisfy d' _L - d' _R = dL _{- dR} , such that the interaural delay difference between the left (L) and right (R) HRTFs is preserved.

The function of Figure 20 (which is the same function used in the example MATLAB code, shown below) is defined to have the following properties: (a) the delay for both ears is a smooth function, and (b) the ear with lower delay (which is typically also the ear with larger amplitude) will have smaller delay variations (because the slope of the curve is lower when the delay is lower).

In some further embodiments, the left HRTF processing block 251L and the right HRTF processing block 251R may be adapted to generate undelayed pulse responses 311L and 311R, respectively, which are minimum phase filter responses.

According to some embodiments, the left HRTF processing block 251L and the right HRTF processing block 251R may be implemented according to the following steps:

1. Determine the frequency response of an original HRTF filter, for example, R _orig,n ( f ) as defined in Equation 11.

2. Determine the amplitude response of the original HRTF filter: M ( f ) = | R _orig,n ( f )|

3. Determine the frequency response of a new (undelayed) minimum phase filter using a method known in the art, using the Hilbert transform:

4. Determine the phase response of the original filter and the undelayed filter: Ψ _orig ( f ) = unwrap ( angle ( R _orig,n ( f )))

Ψ _minp ( f ) = unwrap ( angle ( R' ( f ))) where the angle ( ) function extracts the phase component of a complex frequency response, and the unwrap ( ) operation is known in the art as a method for removing discontinuities in the extracted phase response by adding a multiple of 2π at each frequency (e.g., MATLAB UNWRAP() function).

5. Determine the delay associated with the original HRTF filter:

In some examples, the delay measurement frequency f _d is selected to be a value in the range of 300 Hz to 1600 Hz. In a specific example, f _d = 1200 Hz . In some other examples, f _d may be a value in the range of 300 Hz to 1200 Hz, a value in the range of 600 Hz to 1800 Hz, a value in the range of 1000 Hz to 1400 Hz, or a value in some other frequency range.

In some embodiments, as determined according to the above method, the delay d and the minimum phase response R' ( f ) are used to determine the modified HRTF response Rmod _,n ( f ) according to the following steps:

1. For each original left and right ear HRTF filter (for a given direction of _arrival ), use the above procedure to determine the delay d and minimum phase response R' ( f ), and denote them as: Left ear delay = dL Left ear undelayed filter = R'L (f ₎

Right ear delay = d _R Right ear undelayed filter = R' _R ( f )

2. Determine the maximum interaural delay difference: Let d _max define the maximum interaural delay for all arrival directions ( n =1.. N ).

3. Determine new left-ear and right-ear delay values d' _L and d' _R such that: d' _L - d' _R = dL - dR _, where _the left-ear and right-ear delay values d' _L and d' _R can be determined according to Equation 21.

4. Determine the frequency response of the all-pass filter: where C ( d,t ) can be the function described in Equation 13, which determines the phase response of the all-pass filter associated with the delay d .

5. Determine the newly modified HRTF filter: Left ear:

Right ear:

Example MATLAB Implementation

The following MATLAB implementation provides additional details according to the disclosed method. An example of a MATLAB function that determines a set of HRTF base filters from an existing HRTF library is shown below: FUNCTION IR_DATA = GENERATE_HOA_HRIRS_MOD_LENS(ORDER, SOFA_PATH, ... SOFA_FILE_NAME, IR_LEN) % HRIR CONVERTOR - TAKES SPHERE SAMPLED HRIRS AND CONVERTS THEM TO % HOA HRIRS. % % ORDER - HOA ORDER TO BE CONVERTED TO. % SOFA_PATH - PATH TO THE DIRECTORY THAT CONTAINS THE SOFA FILES TO BE % CONVERTED. % SOFA_FILE - FILE NAME OF THE HRTFS TO BE CONVERTED % IR_LEN - LENGTH OF THE IRS TO BE USED. % % LOAD IN THE SUPPORT COEFS LOAD('HRTF_SUPPORT_COEFS.MAT', 'HRTF_SUPPORT_COEFS'); RMSSPHERE = HRTF_SUPPORT_COEFS(ORDER).RMSSPHERE; LR_ODD = HRTF_SUPPORT_COEFS(ORDER).LR_ODD; XYZ_TO_PAN = HRTE_SUPPORT_COEFS(ORDER). VS_HI_RES.VS_HI_RES; N = 512; % FETCH THE HRTFS, AND FIGURE OUT THE ITD FOR EVERY DIRECTION H = HRTF_LIBRARY_LOADER(); H.READSOFA(CHAR(FULLFILE(SOFA_PATH, SOFA_FILE_NAME))); IRS_HI_RES = H.XYZ_TO_IR(VS_HI_RES); FRS_HI_RES = M_DFT(IRS_HI_RES, N); % FREQ FRS_HI_RES_MINP = MAG2MIN_PHASE(FRS_HI_RES); EXCESS_PHASE = SQUEEZE(UNWRAP(DIFF(ANGLE(FRS_HI_RES), 1, 2) - ... DIFF(ANGLE(FRS_HI_RES_MINP), 1, 2))); BIN1200 = CEIL(1200/24000*N); ITD_HI_RES = EXCESS_PHASE(BIN1200,:)' / ((BIN1200-0.5)/N*24000*2*PI); MAXDEL = MAX(ITD_HI_RES, [], 'ALL'); % CREATE 2 EARS EAR_DELS_HI_RES = (REPMAT(ITD_HI_RES, 1, 2) .* [0.5 -0.5]) + ... 0.5*MAXDEL .* (1 - 2/PI*COS(ITD_HI_RES * PI/2 / MAXDEL)); MRS_HI_RES = ABS(FRS_HI_RES_MINP); % GENERATE PERMUTATION [~, PERM] = ISMEMBERTOL(... VS_HI_RES', VS_HI_RES'.*[1,-1,1], ... 1E-4, "BYROWS",TRUE); MRS_HI_RES(:,2,:) = MRS_HI_RES(:,2,PERM); NEW_FREQRESP_L = MAG2MIN_PHASE(SQUEEZE(MEAN(MRS_HI_RES, 2))) .* ... M_DFT(GET_ALLPASS_IRS(ALLPASS, EAR_DELS_HI_RES(:, 1) * 48000), N, 1); % CREATE SOLVING WEIGHTS WEIGHTS = ABS(NEW_FREQRESP_L); WEIGHTS(WEIGHTS < 0.1) = 0.1; WEIGHTS = 1 ./ (SQRT(SQRT(WEIGHTS))); % SOLVE TO COMPUTE THE HOA FREQUENCY RESPONSES. [M, ~] = SIZE(XYZ_TO_PAN); FREQRESP_HOA = ZEROS(M, N); FOR K=1:N AW = NEW_FREQRESP_L(K,:) .* WEIGHTS(K,:); BW = FREQRESP_HOA = FREQRESP_HOA .* ... MAG2MIN_PHASE(((FREQRESP_HOA_ABS2' * RMSSPHERE.^2) .^ (-0.5)), 1).'; % CONVERT BACK TO IRS IR_HOA = M_IDFT(FREQRESP_HOA.', [], 1); IR_HOA = CAT(3, IR_HOA, (IR_HOA .* (1-2*LR_ODD)')); % PUT MATRIX DIMENSIONS IN THE RIGHT ORDER IR_HOA = PERMUTE(IR_HOA, [3, 1, 2]); % GET THE IRS TO THE RIGHT LENGTH IR_HOA = IR_HOA(:,1:IR_LEN,:) .* ... SIN(INTERP1([0,150/192*IR_LEN,IR_LEN+1],[1,1,0]*PI/2, 1:IR_LEN)); IR = PERMUTE(IR_HOA, [2, 1, 3]); IR_DATA = IR; = GET_ALLPASS_IRS(ALLPASS, DELS) XSET = POLY2XSET(ALLPASS.PROTO_POLY, (DELS(:)'-16)*ALLPASS.UPPER_FREQ/ALLPASS.PROTO_BW); IR = MAKEIRS(XSET, ALLPASS.UPPER_FREQ); END FUNCTION XSET = POLY2XSET(P, DELS_SAMPLES) XSET = ZEROS(SIZE(P,1), NUMEL(DELS_SAMPLES)); FOR K = 1:SIZE(P,1) MAP2POLES(X,BW) P = MAP_2_S_POLES(X) * BW; FOR K = 1:SIZE(P(:,:),2) P(:,K) = BILINEAR(P(:,K),P(:,K),1,48000,BW); END END FUNCTION IRS = MAP_POLES2IRS(P) IRS = ZEROS(512,SIZE(P(:,:),2)); FOR K = 1:SIZE(IRS,2) [~,A] = ZP2TF(P(:,K),P(:,K),1); IRS(:,K) = FILTER(FLIPLR(A),A, [1;ZEROS(511,1)]); END END FUNCTION P = MAP_2_S_POLES(X) ORDER = SIZE(X,1); IF ORDER==0 P=[]; ELSEIF ORDER==1 P=-2*PI*X; ELSE ANG = ATAN(X(2,:))/2+PI/4; P = [-2*PI*X(1,:).*EXP(1I*[1;-1]*ANG) ; MAP_2_S_POLES(X(3:END,:))]; END END

FIG21 is a flow chart outlining an example of a method that may be performed by an apparatus or system such as those disclosed herein. As with other methods described herein, the blocks of method 2100 are not necessarily performed in the order indicated. In some embodiments, one or more of the blocks of method 2100 may be performed concurrently. Furthermore, some embodiments of method 2100 may include more or fewer blocks than shown and/or described. The blocks of method 2100 may be performed by one or more devices, which may be (or may include) a control system, such as control system 110 shown in FIG1A and described above.

In this example, method 2100 is an audio processing method. According to this example, block 2105 involves obtaining a first set of HRTFs from a control system. The first set of HRTFs can be, for example, the original HRTF library 520 of Figures 13 to 16.

In this example, block 2110 involves, by the control system, transforming the first set of HRTFs into a second set of HRTFs. For example, the second set of HRTFs may be the modified HRTF library 541 of Figures 13-16. According to this example, the transformation process of block 2110 involves replacing the delay components of the first set of HRTFs with the all-pass filters in the second set of HRTFs. In this example, the transformation process of block 2110 also involves adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that each interaural phase response is substantially linear for frequencies below a threshold frequency associated with the corresponding all-pass filter, and each phase response reduces interaural phase differences for frequencies above the threshold frequency associated with the corresponding all-pass filter. The critical frequency may be, for example, the frequency at which the alternative phase curve 412 deviates from the linear phase response 411 of FIG. 5 . The critical frequency may be, for example, a frequency in the range of 1300 Hz to 1500 Hz, a frequency in the range of 1000 Hz to 1600 Hz, a frequency in the range of 1200 Hz to 1600 Hz, etc. In some examples, the critical frequency may be 1400 Hz.

In this example, block 2115 involves outputting a result of adjusting the phase response of each of the all-pass filters in the second set of HRTFs. Outputting the result may, for example, involve storing the result, transmitting the result, providing the result for further processing, or a combination thereof.

According to some examples, method 2100 may involve additional steps, such as those described herein with reference to FIG. 13 . In some such examples, method 2100 may also involve the control system defining a set of basic filters based on the second set of HRTFs. The set of basic filters may have fewer components than the second set of HRTFs. In some examples, the set of basic filters may have fewer components than the second set of HRTFs by at least an order of magnitude.

In some examples, method 2100 may also involve a process such as the process described herein with reference to FIG. 14 or FIG. 15 . In some such examples, method 2100 may also involve a control system receiving a bit stream of input audio data in an input audio format, and combining the input audio data with one or more basic filters of a basic filter set to generate left audio data and right audio data. According to some such examples, method 2100 may also be configured to output the left audio data and right audio data from the control system. Outputting the left audio data and the right audio data may involve storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing the left audio data and the right audio data to a set of speakers for playback by the control system, providing the left audio data and the right audio data for further processing (e.g., providing to other modules implemented by the control system to another control system), or a combination thereof.

According to some examples, the transformation process of block 2110 may involve processes such as those described herein with reference to Figures 16 and 17. According to some such examples, block 2110 may involve deriving a left-ear HRTF and a right-ear HRTF from the first set of HRTFs, identifying a left-ear non-delayed pulse response and a left-ear delay from each of the left-ear HRTFs, and identifying a right-ear non-delayed pulse response and a right-ear delay from the right-ear HRTFs. In some such examples, block 2110 may involve generating left-ear all-pass filters, each of which is based at least in part on an instance of the left-ear delay, and generating right-ear all-pass filters, each of which is based at least in part on an instance of the right-ear delay. In some such examples, block 2110 may involve combining instances of the left-ear and right-ear undelayed pulse responses with corresponding instances of the left-ear and right-ear all-pass filters to generate HRTF pairs for the second set of HRTFs.

In some such examples, block 2110 may involve generating modified left-ear delay and right-ear delay values based on one or more of the extracted left-ear delay and right-ear delay. The left-ear all-pass filter and the right-ear all-pass filter may be based on the modified left-ear delay and right-ear delay values, respectively. In some examples, generating the modified left-ear delay and right-ear delay values may involve determining a difference between an extracted left-ear delay and an extracted right-ear delay. According to some examples, generating the modified left-ear delay and right-ear delay values may involve determining a maximum expected difference between an extracted left-ear delay and an extracted right-ear delay.

According to some examples, the difference between an extracted left-ear delay and an extracted right-ear delay may be equal to the difference between a corresponding modified left-ear delay value and a modified right-ear delay value. In some examples, the modified left-ear delay value and the modified right-ear delay value may correspond to a smoothing function, such as the smoothing function shown in FIG. 20 . According to some examples, each pair of modified left-ear delay value and modified right-ear delay value may include a lower delay value and a higher delay value. In some such examples, the lower delay value may have a smaller delay variation than the higher delay value.

In some examples, the undelayed pulse response can be a minimum phase filter response.

According to some examples, extracting each left ear non-delayed pulse response, each right ear non-delayed pulse response, each left ear delay, and each right ear delay from each of the left ear and right ear HRTFs may involve: determining a frequency response of an original HRTF of a first set of HRTFs; determining an amplitude response of the original HRTF; determining a minimum phase frequency response of a new non-delayed minimum phase filter; determining a phase response of the original HRTF and a phase response of the new non-delayed minimum phase filter; and determining a delay associated with the original HRTF filter based at least in part on the phase response of the original HRTF filter and the phase response of the new non-delayed minimum phase filter. In some such examples, determining the minimum phase frequency response may involve performing a Hilbert transform on the magnitude response of the original HRTF filter. According to some examples, determining the delay associated with the original HRTF filter may also be based at least in part on a delay measurement frequency in a range of 300 Hz to 1600 Hz.

In some examples, for frequencies above a threshold frequency, an all-pass phase response may deviate from a linearly ramped phase response and may smooth out to near zero phase. The alternative phase curve 412 of FIG. 5 provides an example of this.

According to some examples, a control system configured to implement method 2100 is also configured to implement at least a portion of a codec for an immersive voice and audio service (IVAS). FIG1D shows one such example.

The above description illustrates various embodiments of the present invention and examples of how the present invention may be implemented. The above examples and embodiments should not be construed as exclusive examples, but rather are presented to illustrate the flexibility and advantages of the present invention as defined by the following claims. Based on the above disclosure and the following claims, a person skilled in the art will recognize that other configurations, embodiments, implementations, and equivalents may be employed without departing from the spirit and scope of the present invention as defined by the following claims.

2100: Methods

2105: Block

2110: Block

2115: Block

Claims (26)

A method for audio processing for a control system comprising one or more processors, the method comprising: obtaining, by the control system, a first set of head-related transfer functions (HRTFs); transforming, by the control system, the first set of HRTFs into a second set of HRTFs, wherein the transforming comprises: replacing a delay component of the first set of HRTFs with an all-pass filter in the second set of HRTFs; adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that: each interaural phase response is substantially linear for frequencies below a threshold frequency associated with the corresponding all-pass filter, and each phase response has a reduced interaural phase difference for frequencies above the threshold frequency associated with the corresponding all-pass filter; and Output the second set of HRTFs. The audio processing method of claim 1, wherein outputting the second set of HRTFs involves storing the second set of HRTFs, transmitting the second set of HRTFs to a device configured to process audio data, providing the second set of HRTFs for further processing, or a combination thereof. The audio processing method of claim 1 or claim 2 further comprises: Defining, by the control system, a set of basic filters based on the second set of HRTFs, wherein the set of basic filters has fewer components than the second set of HRTFs; Obtaining, by the control system, a bit stream of input audio data in an input audio format; Combining, by the control system, the input audio data with one or more basic filters in the set of basic filters to generate left audio data and right audio data; and Outputting, by the control system, the left audio data and the right audio data. As in claim 3, the audio processing method, wherein outputting the left audio data and the right audio data involves storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing the left audio data and the right audio data to a set of speakers for playback by the control system, providing the left audio data and the right audio data for further processing, or a combination thereof. The audio processing method of claim 1 or claim 2, wherein the transformation further comprises: obtaining a left-ear HRTF and a right-ear HRTF from the first set of HRTFs; identifying a left-ear non-delayed pulse response and a left-ear delay from each of the left-ear HRTFs; identifying a right-ear non-delayed pulse response and a right-ear delay from each of the right-ear HRTFs; generating a left-ear all-pass filter, each of the left-ear all-pass filters being based at least in part on an instance of the left-ear delays; generating a right-ear all-pass filter, each of the right-ear all-pass filters being based at least in part on an instance of the right-ear delays; and The instances of the left and right ear undelayed pulse responses are combined with corresponding instances of the left and right ear all-pass filters to generate HRTF pairs for the second set of HRTFs. The audio processing method of claim 5 further includes generating modified left-ear delay values and right-ear delay values based on one or more of the extracted left-ear delays and right-ear delays, wherein the left-ear all-pass filters and the right-ear all-pass filters are based on the modified left-ear delay values and the right-ear delay values. The audio processing method of claim 6, wherein generating the modified left-ear delay values and right-ear delay values involves determining a difference between an extracted left-ear delay and an extracted right-ear delay. The audio processing method of claim 6, wherein the example of generating the modified left-ear delay values and right-ear delay values involves determining a maximum expected difference between an extracted left-ear delay and an extracted right-ear delay. The audio processing method of claim 6, wherein a difference between an extracted left ear delay and an extracted right ear delay is equal to a difference between a corresponding modified left ear delay value and a modified right ear delay value. The audio processing method of claim 6, wherein the modified left-ear delay values and the modified right-ear delay values correspond to a smoothing function. The audio processing method of claim 6, wherein each pair of the modified left-ear delay values and the modified right-ear delay values comprises a lower delay value and a higher delay value and wherein the lower delay value has a delay variation smaller than the higher delay value. The audio processing method of claim 5, wherein the non-delayed pulse responses are minimum phase filter responses. The audio processing method of claim 5, wherein extracting each left ear non-delayed pulse response, each right ear non-delayed pulse response, each left ear delay, and each right ear delay from each of the left and right ear HRTFs involves: Determining a frequency response of an original HRTF filter of the first set of HRTFs; Determining an amplitude response of the original HRTF filter; Determining a minimum phase frequency response of a new non-delayed minimum phase filter; Determining a phase response of the original HRTF filter and a phase response of the new non-delayed minimum phase filter; and Determining a delay associated with the original HRTF filter based at least in part on the phase response of the original HRTF filter and the phase response of the new non-delayed minimum phase filter. The audio processing method of claim 13, wherein determining the minimum phase frequency response involves performing a Hilbert transform of the amplitude response of the original HRTF filter. The audio processing method of claim 13, wherein determining the delay associated with the original HRTF filter is also based at least in part on a delay measurement frequency in a range of 300 Hz to 1600 Hz. The audio processing method of claim 3, wherein the set of basic filters has at least one order of magnitude fewer components than the second set of HRTFs. The audio processing method of claim 1 or claim 2, wherein for frequencies above the threshold frequency, an all-pass phase response deviates from a linear ramp phase response and smoothly approaches zero phase. The audio processing method of claim 1 or claim 2, wherein the control system corresponds to at least a portion of a codec of an immersive voice and audio service (IVAS). A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the operations of the method of any one of claims 1 to 18. An audio processor device for processing input audio data, the audio processor device comprising: a receiver unit configured to receive the input audio data; a computer unit configured to: acquire a first set of head-related transfer functions (HRTFs); convert the first set of HRTFs into a second set of HRTFs, wherein the conversion comprises: replacing a delayed component of the first set of HRTFs with an all-pass filter in the second set of HRTFs; adjusting a phase response of each of the all-pass filters in the second set of HRTFs such that: each interaural phase response is substantially linear for frequencies below a threshold frequency associated with the corresponding all-pass filter, and Each phase response has a reduced interaural phase difference for frequencies above the associated threshold frequency of the corresponding all-pass filter; and outputting the second set of HRTFs. The audio processing device of claim 20, wherein outputting the second set of HRTFs involves storing the second set of HRTFs, transmitting the second set of HRTFs to a device configured to process audio data, providing the second set of HRTFs for further processing, or a combination thereof. The audio processing device of claim 20 or claim 21, wherein the computer unit is further configured to: define a set of basic filters based on the second set of HRTFs, wherein the set of basic filters has fewer components than the second set of HRTFs; obtain a bit stream of input audio data in an input audio format; combine the input audio data with one or more basic filters of the set of basic filters to generate left audio data and right audio data; and output the left audio data and the right audio data. An audio processor device as claimed in claim 22, wherein outputting the left audio data and the right audio data involves storing the left audio data and the right audio data, transmitting the left audio data and the right audio data, providing the left audio data and the right audio data to a set of speakers for playback by the control system, providing the left audio data and the right audio data for further processing, or a combination thereof. The audio processing device of claim 22, further comprising a storage device configured to store the first HRTFs, the second HRTFs, the left audio data, the right audio data, the input audio data, or a combination thereof. The audio processing device of claim 24, wherein the storage device comprises one or more of a random access memory, a read-only memory, and a non-transitory computer-readable medium. The audio processing device of claim 20 or claim 21, wherein the device corresponds to at least a portion of a codec of an immersive voice and audio service (IVAS).