HK1262874B - Binaural rendering for headphones using metadata processing - Google Patents

Description

Binaural rendering for headphones using metadata processing

This application is a divisional application based on the patent application with application number 201480060042.2, application date October 28, 2014, and invention name “Binaural presentation of headphones using metadata processing”.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application No. 61/898,365, filed October 31, 2013, the entire contents of which are hereby incorporated by reference.

Technical Field

One or more implementations relate generally to audio signal processing, and more particularly to binaural rendering of channel- and object-based audio for headphone playback.

Background Art

Virtual rendering of spatial audio through a pair of speakers typically involves the creation of stereo binaural signals representing the desired sounds arriving at the listener's left and right ears, synthesized to simulate a specific audio scene in a three-dimensional (3D) space that may contain numerous sources at different locations. For playback through headphones rather than speakers, binaural processing or rendering can be defined as a set of signal processing operations designed to reproduce the expected 3D positions of sound sources through headphones by emulating the natural spatial listening cues of a human subject. Typical core components of a binaural renderer are head-related filtering to reproduce direction-related cues and distance cue processing, which may involve modeling the effects of a real or virtual listening room or environment. One example of a current binaural renderer processes each of the 5 or 7 channels of a 5.1 or 7.1 surround sound channel-based audio presentation as 5 or 7 virtual sound sources in a 2D space surrounding the listener. Binaural reproduction is also commonly found in games or gaming audio hardware, where processing can be applied to individual audio objects in the game based on their individual 3D positions.

Traditionally, binaural rendering is a form of blind post-processing applied to multi-channel or object-based audio content. Some of the processing involved in binaural rendering can have undesirable and negative effects on the timbre of the content, such as smoothing of transients or excessive reverberation added to dialogue or some effects and musical elements. With the growing importance of headphone listening and the additional flexibility brought by object-based content (such as Atmos ^™ systems), there is a greater opportunity and need for mixers to create and encode this binaural rendering metadata at the time of content creation, for example, to instruct the renderer to process parts of the content with different algorithms or different settings. Current systems do not feature this capability, nor do they allow such metadata to be conveyed as an additional, specific headphone payload in the codec.

Current systems are also not optimized on the playback side of the pipeline as long as the content is not configured to be received on the device with additional metadata that can be provided immediately to the binaural renderer. While real-time head tracking has been previously implemented and shown to improve binaural rendering, this has generally prevented other features, such as automated continuous head size sensing and room sensing, and other customized features that improve the quality of binaural rendering from being effectively and efficiently implemented in headphone-based playback systems.

Therefore, there is a need for a binaural renderer running on a playback device that combines production metadata with metadata generated locally in real time to provide the end user with the best possible experience when listening to channel- and object-based audio through headphones. Furthermore, for channel-based content, it is generally required that artistic intent be preserved by incorporating audio segment analysis.

The subject matter discussed in the Background section should not be assumed to be prior art simply because it is mentioned in the Background section. Similarly, problems mentioned in the Background section or related to the subject matter in the Background section should not be assumed to have been previously recognized in the prior art. The subject matter in the Background section merely represents different approaches, which may themselves be inventions.

Summary of the Invention

Embodiments of systems and methods for virtually presenting object-based audio content and improving equalization in a headphone-based playback system are described. Embodiments include a method for presenting audio for playback through headphones, the method comprising: receiving digital audio content; receiving binaural rendering metadata generated by an authoring tool that processes the received digital audio content; receiving playback metadata generated by a playback device; and combining the binaural rendering metadata and the playback metadata to optimize playback of the digital audio content through the headphones. The digital audio content may include channel-based audio and object-based audio, the object-based audio including position information for reproducing the intended position of corresponding sound sources in three-dimensional space relative to a listener. The method also includes separating the digital audio content into one or more components based on content type, and wherein the content type is selected from the group consisting of: dialogue, music, sound effects, transient signals, and ambient signals. Binaural rendering metadata controls multiple channel and object characteristics, including: position, size, gain adjustment, and content-related settings or processing presets; playback metadata controls multiple listener-specific characteristics, including head position, head orientation, head size, listening room noise level, listening room properties, and the position of the playback device or screen relative to the listener. The method may also include receiving one or more user input commands to modify the binaural rendering metadata, these user input commands controlling one or more characteristics, including: boost emphasis, where boosted objects and channels can receive a gain increase; preferred 1D (one-dimensional) sound radius or 3D scaling factor for object or channel positioning; and processing mode enablement (e.g., to switch between traditional stereo or full processing of the content). The playback metadata may be generated in response to sensor data provided by an enabled headset housing multiple sensors, the enabled headset forming part of the playback device. The method may further comprise: separating the input audio into separate sub-signals, for example by content type, or demixing the (channel-based and object-based) input audio into constituent direct content and diffuse content, wherein the diffuse content includes reverberant or reflected sound elements; and independently performing binaural rendering on the separate sub-signals.

Embodiments also relate to a method for rendering audio for playback through headphones, comprising the steps of: receiving content-related metadata that determines how content elements are rendered through the headphones; receiving sensor data from at least one of a playback device coupled to the headphones and an enabled headset comprising the headphones; and utilizing the sensor data to modify the content-related metadata to optimize the rendered audio relative to one or more playback characteristics and user characteristics. The content-related metadata may be generated by an authoring tool operated by a content creator, and wherein the content-related metadata determines the rendering of an audio signal comprising audio channels and audio objects. The content-related metadata controls a plurality of channel and object characteristics selected from the group consisting of: position, size, gain adjustment, boost emphasis, stereo/full switching, 3D scaling factor, content-related settings, and other spatial and timbre properties of the rendered sound field. The method may further include formatting the sensor data into a metadata format compatible with the content-related metadata to generate playback metadata. The playback metadata controls a plurality of listener-specific characteristics selected from the group consisting of: head position, head orientation, head size, listening room noise level, listening room properties, and sound source device location. In an embodiment, the metadata format includes a container including one or more payload packets conforming to a defined syntax and encoding a digital audio definition of a corresponding audio content element. The method may further include encoding the combined playback metadata and content-related metadata along with the source audio content into a bitstream for processing in a rendering system; and decoding the encoded bitstream to extract one or more parameters derived from the content-related metadata and the playback metadata to generate a control signal for modifying the source audio content for playback through headphones.

The method may further include performing one or more post-processing functions on the source audio content prior to playback through the headphones; wherein the post-processing functions include at least one of: downmixing from multiple surround sound channels to one of a binaural mix or a stereo mix, level management, equalization, timbre correction, and noise cancellation.

Embodiments further relate to systems and articles of manufacture that perform or implement processing instructions to perform or implement the above-described methods and the like.

Incorporated by Reference

Each publication, patent, and/or patent application mentioned in this specification is herein incorporated by reference in its entirety, to the same extent as if each individual publication and/or patent application was specifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following figures, like reference numerals are used to refer to like elements. Although the following figures depict various examples, the one or more implementations are not limited to the examples depicted in these figures.

FIG. 1 illustrates an overall system of an embodiment of a combined content creation, presentation, and playback system, under some embodiments.

2A is a block diagram of an authoring tool used in an object-based headphone rendering system, under an embodiment.

2B is a block diagram of an authoring tool used in an object-based headphone rendering system, under an alternative embodiment.

3A is a block diagram of presentation components used in an object-based headphone presentation system, under an embodiment.

3B is a block diagram of presentation components used in an object-based headphone presentation system, under an alternative embodiment.

4 is a block diagram providing an overview of a two-end binaural presentation system, under an embodiment.

FIG5 illustrates an authoring tool GUI that may be used with embodiments of a headphone rendering system, under an embodiment.

6 illustrates an enabled headset including one or more sensors that sense playback conditions for encoding as metadata for use in a headphone rendering system, under an embodiment.

FIG7 illustrates a connection between a headset and a device including a headset sensor processor, under an embodiment.

8 is a block diagram illustrating different metadata components that may be used in a headphone rendering system, under an embodiment.

FIG9 illustrates functional components of a binaural rendering component for headphone processing, under an embodiment.

FIG10 illustrates a binaural rendering system for rendering audio objects in a headphone rendering system under an embodiment.

FIG11 shows a more detailed representation of the binaural rendering system of FIG10 , under an embodiment.

12 is a system diagram illustrating different tools used in an HRTF modeling system for use in a headphone rendering system, under an embodiment.

Figure 13 illustrates a data structure that enables delivery of metadata to a headphone presentation system, under an embodiment.

FIG. 14 shows an example case of three impulse response measurements for each ear in an embodiment of headphone equalization processing.

FIG15A shows a circuit for calculating free-field acoustic transmission, under an embodiment.

FIG15B illustrates a circuit for calculating headphone sound transmission, under an embodiment.

DETAILED DESCRIPTION

Systems and methods for virtually presenting object-based content through headphones and metadata delivery and processing systems for such virtual presentations are described, but the applications are not limited thereto. Aspects of one or more embodiments described herein can be implemented in an audio or audio-visual system that processes source audio information in a mixing, presentation, and playback system that includes one or more computers or processing devices that execute software instructions. Any of the embodiments described may be used alone or in any combination with each other. Although the various embodiments may be motivated by various deficiencies in the prior art that may be discussed or implied in one or more places in this specification, the embodiments do not necessarily address any of these deficiencies. In other words, different embodiments may address different deficiencies that may be discussed in this specification. Some embodiments may only partially address some of the deficiencies that may be discussed in this specification, or may only address one defect that may be discussed in this specification, and some embodiments may not address any of these deficiencies.

Embodiment is directed to optimizing the audio content generation and playback system of the presentation and playback of the audio based on object and/or sound channel by earphone.Fig. 1 illustrates the overall system of the embodiment of the content creation, presentation and playback system that merges under some embodiments.As shown in system 100, making tool 102 is used by the creator to produce audio content for playback by one or more devices 104, for the user to listen by earphone 116 or 118.Device 104 is generally a small computer or mobile telecommunication device or portable audio or music player that runs the application that allows playback audio content.Such device can be mobile phone or audio (for example, MP3) player 106, tablet computer (for example, Apple iPad or similar device) 108, music console 110, notebook computer 111 or any similar audio playback device.Audio can comprise music, dialogue, effect or any digital audio that can be expected to be listened to by earphone, and such audio can be wirelessly streamed from content source, played back locally from storage medium (for example, disk, flash drive etc.), or locally produced. In the following description, the term "headphones" generally and specifically refers to a close-coupled playback device, or in-ear listening device, worn by a user just above his/her ear; it may also generally refer to at least some of the processing performed to render a signal intended for playback on headphones, as an alternative to the terms "headphone processing" or "headphone rendering."

In embodiments, the audio processed by the system may include channel-based audio, object-based audio, or object- and channel-based audio (e.g., hybrid or adaptive audio). The audio includes, or is associated with, metadata that determines how the audio is rendered for playback on a specific endpoint device and listening environment. Channel-based audio generally refers to an audio signal plus metadata in which positions are encoded as channel identifiers, where the audio is formatted for playback through a predefined set of speaker zones (e.g., 5.1, 7.1, etc.) with associated nominal surround sound positions; object-based means one or more audio channels with parameterized source descriptions, such as apparent source positions (e.g., 3D coordinates), apparent source width, etc. The term "adaptive audio" may be used to refer to a channel-based and/or object-based audio signal plus metadata that uses an audio stream plus metadata in which positions are encoded as 3D positions in space to render the audio signal based on the playback environment. In general, a listening environment can be any open, partially enclosed, or fully enclosed area, such as a room, but the embodiments described herein are generally directed to playback through headphones or other closely located endpoint devices. Audio objects can be thought of as groups of sound elements that can be perceived as originating from a specific physical location or locations in the environment, and such objects can be static or dynamic. Audio objects are controlled by metadata that, among other things, details the location of the sound at a given point in time, and when played back, they are rendered according to the positional metadata. In hybrid audio systems, in addition to audio objects, channel-based content (e.g., "beds") can also be processed, where a bed is effectively a channel-based sub-mix or stem. These can be delivered for final playback (rendering) and can be created in different channel-based configurations (such as 5.1, 7.1).

As shown in FIG1 , the earphones utilized by the user may be legacy or passive earphones 118 that simply include a non-powered transducer that simply recreates the audio signal, or it may be enabled earphones 118 that include sensors and other components (powered or non-powered) that provide certain operating parameters back to the renderer for further processing and optimization of the audio content. The earphones 116 or 118 may be embodied in any suitable closed-ear device, such as open or closed-back earphones, on-ear or in-ear headphones, earbuds, earpads, noise canceling, isolating, or other types of earphone devices. Such earphones may be wired or wireless with respect to their connection to the sound source or device 104.

In embodiments, in addition to object-based audio, the audio content from the authoring tool 102 also includes stereo or channel-based audio (e.g., 5.1 or 7.1 surround sound). For the embodiment of FIG. 1 , the renderer 112 receives the audio content from the authoring tool and provides certain functions for optimizing the audio content for playback via the device 104 and headphones 116 or 118. In embodiments, the renderer 112 includes a pre-processing stage 113, a binaural rendering stage 114, and a post-processing stage 115. Among other functions, the pre-processing stage 113 generally performs certain segmentation operations on the input audio, such as segmenting it based on its content type; the binaural rendering stage 114 generally combines and processes metadata associated with the channel and object components of the audio and produces a binaural audio output or a multi-channel audio output with binaural audio and additional low-frequency output; and the post-processing component 115 generally performs downmixing, equalization, gain/loudness/dynamic range control, and other functions before sending the audio signal to the device 104. It should be noted that while the renderer will likely produce a two-channel signal in most cases, it can be configured to provide more than two channels of input to specific enabled headphones, for example to deliver a separate bass channel (similar to the LFE.1 channel in traditional surround sound). The enabled headphones may have specific sets of drivers that reproduce bass components separately from mid-range to higher frequency sounds.

It should be noted that the components of Figure 1 generally represent the main functional blocks of the audio generation, presentation and playback system, and certain functions can be incorporated as part of one or more other components. For example, one or more parts of the renderer 112 can be partially or entirely incorporated into the device 104. In this case, the audio player or tablet (or other device) can include a renderer component integrated within the device. Similarly, enabling headphones 116 can include at least some functions associated with the playback device and/or renderer. In such a case, a fully integrated headset can include an integrated playback device (e.g., a built-in content decoder, e.g., an MP3 player) and an integrated presentation component. In addition, one or more components of the renderer 112 (such as pre-processing component 113) can be implemented at least in part in a production tool, or as part of a separate pre-processing component.

Fig. 2 A is a block diagram of the production tool used in the object-based headphone presentation system under an embodiment.As shown in Figure 2A, input audio 202 from an audio source (e.g., a live source, a record, etc.) is input to a digital audio workstation (DAW) 204 for sound engineer processing. Input audio 201 is typically in digital form, and if analog audio is used, an A/D (analog-to-digital) conversion step (not shown) is required. The audio typically includes object-based and channel-based content such as that which can be used in an adaptive audio system (e.g., Dolby Atmos), and often includes several different types of content. The input audio can be segmented by (optional) audio segmentation pre-processing 204, which segments the audio based on the content type of the audio (or segments) so that different types of audio can be presented differently. For example, a conversation can be presented differently from a transient signal or an ambient signal. The DAW 204 can be implemented as a workstation for editing and processing segmented or unsegmented digital audio 202 and can include a mixing console, a control surface, audio converters, data storage, and other appropriate components. In an embodiment, the DAW is a processing platform that runs digital audio software that provides comprehensive editing capabilities and an interface for one or more plug-ins (such as a panner plug-in) in addition to other functions (such as equalizers, synthesizers, effects, etc.). The panner plug-in shown in the DAW 204 performs a panning function that is configured to assign each object signal to a specific speaker pair or position in 2D/3D space in a manner that conveys the desired position of each corresponding object signal to the listener.

In the authoring tool 102a, processed audio from the DAW 204 is input to a binaural rendering component 206. This component includes audio processing functionality that generates a binaural audio output 210, as well as binaural rendering metadata 208 and spatial media type metadata 212. The audio 210 and metadata components 208 and 212, together with a binaural metadata payload 214, form an encoded audio bitstream. Typically, the audio component 210 includes channel- and object-based audio, which is passed along with the metadata components 208 and 212 to the bitstream 214; however, it should be noted that the audio component 210 can be standard multi-channel audio, binaurally rendered audio, or a combination of both audio types. The binaural rendering component 206 also includes binaural metadata input functionality that directly generates a headphone output 216 for direct connection to headphones. For the embodiment of FIG. 2A , the metadata for binaural rendering is generated within the authoring tool 102a during mixing. In an alternative embodiment, as illustrated with reference to FIG. 2B , the metadata can be generated during encoding. As shown in Figure 2A, the mixer 203 uses an application or tool to create audio data and binaural and spatial metadata. The mixer 203 provides input to the DAW 204. Alternatively, it can also provide input directly to the binaural rendering process 206. In an embodiment, the mixer receives headphone audio output 216 so that the mixer can monitor the effect of the audio and metadata input. This effectively constitutes a feedback loop in which the mixer receives audio rendered by the headphones through the headphone output to determine whether any input changes are needed. The mixer 203 can be a human-operated device, such as a mixing console or computer, or it can be a remotely controlled or pre-programmed automated process.

FIG2B is a block diagram of a production tool used in an object-based headphone rendering system under an alternative embodiment. In this embodiment, metadata for binaural rendering is generated during encoding, and the encoder runs a content classifier and a metadata generator to generate additional metadata from legacy channel-based content. For the production tool 102b of FIG2B , legacy multi-channel content 220 that does not include any audio objects but only includes channel-based audio is input to the encoding tool and rendering headphone emulation component 226. Object-based content 222 is also input separately to this component. Legacy channel-based content 220 can first be input to an optional audio segmentation preprocessor 224 for separation into different content types for individual rendering. In the production tool 102b, the binaural rendering component 226 includes a headphone emulation function that generates a binaural audio output 230 as well as binaural rendering metadata 228 and spatial media type metadata 232. The audio 230 and metadata components 228 and 232, together with the binaural metadata payload 236, form a coded audio bitstream. As described above, the audio component 230 typically includes channel- and object-based audio that is passed to the bitstream 236 along with metadata components 228 and 232; however, it should be noted that the audio component 230 can be standard multi-channel audio, binaurally rendered audio, or a combination of both audio types. When legacy content is input, the output encoded audio bitstream can contain explicitly separated sub-component audio data or metadata that implicitly describes the content type that allows the receiving endpoint to perform segmentation and process each sub-component appropriately. The binaural rendering component 226 also includes a binaural metadata input function that directly generates a headphone output 234 for direct connection to headphones. As shown in Figure 2B, an optional mixer (person or process) 223 can be included to monitor the headphone output 234 to input and modify the audio data and metadata input that can be directly provided to the rendering process 226.

With respect to content types and the operation of content classifiers, audio is generally classified into one of a number of defined content types (such as dialogue, music, ambiance, special effects, etc.). An object can change content types throughout its duration, but at any given point in time, it is generally of only one content type. In embodiments, content type is expressed as the probability of an object being of a particular content type at any point in time. Thus, for example, a persistent dialogue object would be expressed as a dialogue object with a 100% probability, while an object that transitions from dialogue to music could be expressed as 50% dialogue/50% music. Objects with different content types can be processed by averaging their respective probabilities for each content type, selecting the content type probability of the most dominant object within a group of objects, or a single object over time, or some other logical combination of content type metrics. Content type can also be expressed as an n-dimensional vector (where n is the total number of different content types, e.g., four in the case of dialogue/music/ambiance/effects). Content type metadata can be embodied as a combined content type metadata definition, where the combination of content types reflects the probability distribution of the combined content types (e.g., a vector of probabilities of music, speech, etc.).

Regarding audio classification, in an embodiment, the process operates on a per-timeframe basis to analyze the signal, identify features of the signal, and compare the identified features with features of known classes to determine the degree to which the features of an object match those of a particular class. Based on the degree to which the features match those of a particular class, the classifier can determine the probability that the object belongs to that particular class. For example, if at time t=T, the features of an object match very well with the features of conversation, then the object will be classified as conversation with a high probability. If at time=T+N, the features of an object match very well with the features of music, then the object will be classified as music with a high probability. Finally, if at time T=T+2N, the features of an object do not match either conversation or music particularly well, then the object may be classified as 50% music and 50% conversation. Thus, in an embodiment, based on the content type probabilities, the audio content can be divided into different sub-signals corresponding to different content types. This is achieved, for example, by sending a percentage of the original signal to each sub-signal (on a broadband basis or per frequency sub-band basis) in proportions determined by the calculated media type probabilities.

Referring to FIG1 , output from the authoring tool 102 is input to the renderer 112 for rendering as audio output for playback via headphones or other endpoint devices. FIG3A is a block diagram of a rendering component 112a used in an object-based headphone rendering system under an embodiment. FIG3A illustrates in greater detail the pre-processing 113, binaural rendering 114, and post-processing 115 subcomponents of the renderer 112. From the authoring tool 102, metadata and audio are input to the processing or pre-processing components in the form of an encoded audio bitstream 301. Metadata 302 is input to a metadata processing component 306, and audio 304 is input to an optional audio segmentation pre-processor 308. As shown in FIG2A and FIG2B , audio segmentation can be performed by the authoring tool via pre-processors 202 or 224. If such audio segmentation is not performed by the authoring tool, the renderer can perform this task via pre-processor 308. The processed metadata and segmented audio are then input to a binaural rendering component 310. This component performs certain headphone-specific rendering functions, such as 3D positioning, distance control, head size handling, etc. The binaurally rendered audio is then input to an audio post-processor 314, which applies certain audio operations, such as level management, equalization, noise compensation or cancellation, etc. The post-processed audio is then output 312 for playback through headphones 116 or 118. For embodiments where the headphones or playback device 104 are equipped with sensors and/or microphones for providing feedback to the renderer, microphone and sensor data 316 is input back to at least one of the metadata processing component 306, the binaural rendering component 310, or the audio post-processing component 314. For standard headphones without sensors, head tracking can be replaced with a simpler pseudo-randomly generated head "jitter" that simulates small, continuously changing head movements. This allows any relevant environmental or operational data at the playback point to be used by the rendering system to further modify the audio to offset or enhance certain playback conditions.

As mentioned above, audio segmentation can be performed by either the production tool or the renderer. For embodiments in which the audio is pre-segmented, the renderer processes the audio directly. FIG3B is a block diagram of the rendering components used in the object-based headphone rendering system under this alternative embodiment. As shown for renderer 112 b, an encoded audio bitstream 321 from the production tool is provided as a component of metadata 322 that is input to a metadata processing component 326 and audio 324 that is input to a binaural rendering component 330. For the embodiment of FIG3B , the audio is pre-segmented in the appropriate production tool by audio pre-segmentation processing 202 or 224. The binaural rendering component 330 performs certain headphone-specific rendering functions, such as 3D positioning, distance control, head size processing, etc. The binaurally rendered audio is then input to an audio post-processor 334, which applies certain audio operations, such as level management, equalization, noise compensation or cancellation, etc. The post-processed audio is then output 332 for playback through headphones 116 or 118. For embodiments in which headphones or playback device 104 are equipped with sensors and/or microphones for providing feedback to a renderer, microphone and sensor data 336 are input back to at least one of metadata processing component 326, binaural rendering component 330, or audio post-processing component 334. The production and rendering system of Figures 2A, 2B, 3A, and 3B allows content producers to use production tool 102 to create specific binaural rendering metadata when creating content and to encode this specific binaural rendering metadata. This allows audio data to be used to instruct the renderer to utilize different algorithms or utilize different settings to process parts of the audio content. In an embodiment, production tool 102 represents a workstation or computer application that allows content creators (producers) to select or create audio content for playback and to define certain characteristics for each of the channels and/or objects that constitute the audio content. The production tool can include a graphical user interface (GUI) representation of a mixer-type console interface or a mixing console. Figure 5 illustrates an embodiment of a production tool GUI that can be used together with an embodiment of a headphone rendering system. As can be seen in GUI display 500, a number of different characteristics are allowed to be set by the producer, such as gain level, low-frequency characteristics, equalization, panning, object position and density, delay, fade, etc. For the illustrated embodiment, user input is facilitated by using virtual sliders for the producer to specify setting values, although other virtualized or direct input methods are also possible, such as direct text input, potentiometer settings, rotary dials, etc. At least some of the parameter settings entered by the user are encoded as metadata associated with the relevant channels or audio objects for transmission with the audio content. In embodiments, the metadata can be packaged as part of an additional, specific headphone payload in the codec (encoder/decoder) circuitry within the audio system. Using an enabling device, real-time metadata encoding certain operating and environmental conditions (e.g., head tracking, head size sensing, room sensing, ambient conditions, noise levels, etc.) can be provided to the binaural renderer in real time. The binaural renderer combines the produced metadata content with the real-time locally generated metadata to provide the user with an optimized listening experience. Generally, the object controls and user input interface provided by the authoring tool allow the user to control certain important headphone-specific parameters, such as binaural and stereo-bypass dynamic rendering modes, LFE (low-frequency element) gain and object gain, media intelligence, and content-dependent control. More specifically, the rendering mode can be selected based on content type, stereo (Lo/Ro), matrixed stereo (Lt/Rt), or object-based, using a combination of interaural time delay and stereo amplitude or intensity panning, or the entire binaural rendering (i.e., a combination of interaural time delay and level and frequency-dependent spectral cues). In addition, frequency crossover points can be specified to revert to stereo processing below a given frequency. Low-frequency gain can also be specified to attenuate low-frequency components or LFE content. As described in more detail below, low-frequency content can also be transmitted separately to enabled headphones. Other metadata can be specified on a per-content type or per-channel/object basis, such as a room model, typically described by direct/reverberant gain and frequency-dependent reverberation time, as well as interaural target cross-correlations. It may also include other more detailed modeling of the room (e.g., early reflection positions, gains, and late reverberation gains). It may also include directly specified filters that model specific room responses. Other metadata include a warp-to-screen flag (which controls how objects are remapped to fit the viewing angle and screen aspect ratio as a function of distance). Finally, a listener-relative flag (i.e., whether head tracking information is applied), a preferred scaling (specifying a default size/aspect ratio of the "virtual room" used to render the content, which is used to scale object positions and remap to the screen (according to the device screen size and distance to the device)), and a distance model exponent that controls the distance decay law (e.g., 1/(1+r ^α )) are also possible. Groups of parameters or "presets" that can be applied to different channels/objects or based on the type of content can also be signaled.

As shown with respect to the pre-segmentation components of the production tools and/or renderers, different types of content (e.g., dialogue, music, effects, etc.) can be treated differently based on the producer's intent and the optimal presentation configuration. Segmentation of content based on type or other salient characteristics can be achieved a priori during production (e.g., by manually saving segmented dialogue into their own set of tracks or objects), or a posteriori immediately prior to presentation in the receiving device. Additional media intelligence tools can be used during production to classify content according to different characteristics and generate additional channels or objects that can carry different sets of presentation metadata. For example, given the knowledge of the stems (music, dialogue, Foley, effects, etc.) and the associated surround (e.g., 5.1) mix, a media classifier can be trained for the content creation process to develop a model that recognizes different stem mix ratios. Associated source segmentation techniques can be utilized to extract approximate stems using a weighting function derived from the media classifier. From the extracted stems, binaural parameters, which will be encoded as metadata, can be applied during production. In an embodiment, a mirroring process is applied in the end-user device, whereby using the decoded metadata parameters will create a substantially similar experience as during content creation.

In an embodiment, extensions to existing studio production tools include binaural monitoring and metadata recording. Typical metadata captured during production includes: channel/object position/size information for each channel and audio object, channel/object gain adjustments, content-specific metadata (which can vary based on content type), a bypass flag indicating whether a setting (such as stereo/left/right rendering should be used instead of binaural), a crossover point and a level indicating that bass frequencies below the crossover point must be bypassed and/or attenuated), and room model information describing direct/reverberation gain and frequency-dependent reverberation time or other characteristics (such as early reflection and late reverberation gain). Other content-specific metadata can provide a warp-to-screen function that remaps the image to fit the screen aspect ratio or changes the viewing angle as a function of distance. Head tracking information can be used to provide a listener-relative experience. Metadata implementing a distance model exponent that controls distance according to a decay law (e.g., 1/(1+ ^rα )) can also be used. These represent only some of the characteristics that can be encoded in metadata, and others may also be encoded.

Fig. 4 is a block diagram of an overview of a two-end binaural presentation system provided under an embodiment. In an embodiment, system 400 provides content-related metadata and influences how different types of audio content will be presented. For example, the original audio content can include different audio elements, such as dialogue, music, effects, ambient sounds, transients, etc. Each of these elements can be best presented in different ways, rather than being limited to all being presented in a unique way. For an embodiment of system 400, audio input 401 includes a multi-channel signal, an object-based channel, or a mixed audio of a channel plus an object. The audio is input to encoder 402, which adds or modifies metadata associated with the audio object and the channel. As shown in system 400, the audio is input to headphone monitoring component 410, which uses user-adjustable parameterization tools to control headphone processing, equalization, downmixing, and other characteristics suitable for headphone playback. The user-optimized parameter set (M) is then embedded by encoder 402 as metadata or additional metadata to form a bitstream that is sent to decoder 404. The decoder 404 decodes metadata and parameter sets M of object- and channel-based audio for controlling a headphone processing and downmixing component 406, which generates a headphone-optimized and downmixed (e.g., 5.1 to stereo) audio output 408 for headphones. Although some content-dependent processing has been implemented in current systems and post-processing chains, it has generally not been applied to binaural renderings such as shown in the system 400 of FIG. 4 .

As shown in Figure 4, certain metadata can be provided by a headphone monitoring component 410, which provides specific user-adjustable parameterization tools for controlling headphone-specific playback. Such a component can be configured to provide a user with a certain degree of control over the headphone presentation of an old headphone 118 for passively playing back the transmitted audio content. Alternatively, the endpoint device can be an enabling headset 116, which includes sensors and/or a certain degree of processing power to generate metadata or signal data, which can be encoded as compatible metadata to further modify the produced metadata to optimize the audio content for presentation through the headphones. Therefore, at the receiving end of the content, the presentation is performed instantly and can take into account locally generated sensor array data, which can be generated by the headset or the actual mobile device 104 to which the headset is attached, and the metadata generated by such hardware can be further combined with the metadata created by the content creator at the time of production to enhance the binaural presentation experience.

As described above, in some embodiments, low-frequency content can be transmitted separately to enabled headphones that allow more than one stereo input (typically 3 or 4 audio inputs), or encoded and modulated into the higher frequencies of the primary stereo waveform carried to a headset with a single stereo input. This would allow further low-frequency processing to occur in the headphones (e.g., routing to specific drivers optimized for low frequencies). Such headphones may include low-frequency-specific drivers and/or filter-plus-crossover and amplification circuitry to optimize playback of low-frequency signals.

In an embodiment, a link is provided on the playback side from the headphones to the headphone processing component to enable manual identification of the headphones for automatic headphone preset loading or other configuration of the headphones. Such a link can be implemented, for example, as a wireless or wired link from the headphones to the headphone processing 406 in FIG. 4 . This identification can be used to configure the target headphones or to send specific content or specially rendered content to a specific group of headphones if multiple target headphones are being used. The headphone identifier can be embodied as any suitable alphanumeric or binary code that is processed by the presentation processing as part of the metadata or as a separate data processing operation.

FIG6 illustrates an embodiment of an enabled headset that includes one or more sensors for sensing playback conditions for encoding into metadata for use in a headphone rendering system. The various sensors can be arranged into a sensor array that can be used to provide real-time metadata to a renderer during rendering. For the example headset 600 of FIG6 , the sensors include, among other appropriate sensors, a range sensor (such as an infrared (IR) or time-of-flight (TOF) camera) 602, a strain/head size sensor 604, a gyroscope sensor 606, an external microphone (or pairs) 610, an ambient noise cancellation processor 608, and an internal microphone (or pairs) 612. As shown in FIG6 , the sensor array can include both audio sensors (i.e., microphones) and data sensors (e.g., orientation, size, strain/stress, and range sensors). Specifically, for use with headphones, orientation data can be used to “lock” or rotate spatial audio objects based on the listener’s head movement, and strain sensors or external microphones can be used to infer the listener’s head size (e.g., by monitoring audio cross-correlation at two external microphones located on the ear cups) and adjust relevant binaural rendering parameters (e.g., interaural time delay, shoulder reflex timing, etc.). The distance sensor 602 can be used to assess the distance to the display in the case of mobile A/V playback and correct the position of on-screen objects to account for distance-dependent viewing angles (i.e., objects are rendered wider as the screen gets closer to the listener) or to adjust global gain and room model to deliver appropriate distance rendering. Such sensor functionality is useful if the audio content is part of A/V content that can be played back on devices ranging from small mobile phones (e.g., 2-4" screen size) to tablets (e.g., 7-10" screen size) to laptop computers (e.g., 15-17" screen size). Additionally, the sensor can also be used to automatically detect and route the left and right audio outputs to the correct transducers without requiring a specific a priori orientation or explicit "left/right" markings on the headphones.

As shown in Figure 1, the audio or A/V content sent to the headset 116 or 118 can be provided by a handheld or portable device 104. In an embodiment, the device 104 itself may include one or more sensors. For example, if the device is a handheld game console or game controller, certain gyroscope sensors and accelerometers may be provided to track object movement and position. For this embodiment, the device 104 to which the headset is connected may also provide additional sensor data (such as orientation, head size, camera, etc.) as device metadata.

For this embodiment, certain headset-to-device communication methods are implemented. For example, the headset can be connected to the device via a wired or wireless digital link or an analog audio link (microphone input), in which case metadata will be frequency modulated and added to the analog microphone input. Figure 7 shows the connection between the headset and the device 104, including the headset sensor processor 702, under an embodiment. As shown in system 700, the headset 600 sends certain sensor, audio, and microphone data 701 to the headset sensor processor 702 via a wired or wireless link. The processed data from the processor 702 may include analog audio 704 with metadata or spatial audio output 706. As shown in Figure 7, each connection includes a bidirectional link between the headset, the processor, and the output. This allows sensor and microphone data to be sent between the headset and the device for creation or modification of appropriate metadata. In addition to the metadata generated by the hardware, user controls can also be provided to supplement or generate appropriate metadata if metadata is not available through the hardware sensor array. Example user controls may include: elevation emphasis, binaural on/off switching, preferred sound radius or size, and other similar features. Such user control may be provided through hardware or software interface elements associated with the headset processor assembly, playback device, and/or headset.

8 is a block diagram illustrating various metadata components that may be used in a headphone rendering system under an embodiment. As shown in diagram 800, the metadata processed by the headphone processor 806 includes authored metadata (such as metadata generated by the authoring tool 102 and the mixing console 500) and hardware-generated metadata 804. The hardware-generated metadata 804 may include user-entered metadata, device-side metadata provided by the device 808 or generated from data sent from the device 808, and/or headphone-side metadata provided by the headphone 810 or generated from data sent from the headphone 810.

In an embodiment, the produced 802 and/or hardware-generated 804 metadata is processed in the binaural rendering component 114 of the renderer 112. The metadata provides control over specific audio channels and/or objects to optimize playback through headphones 116 or 118. FIG9 illustrates the functional components of a binaural rendering component for headphone processing, under an embodiment. As shown in system 900, a decoder 902 outputs a multi-channel signal or a channel-plus-object track, along with a decoded parameter set M for controlling the headphone processing performed by a headphone processor 904. The headphone processor 904 also receives certain spatial parameter updates 906 from a camera-based or sensor-based tracking device 910. The tracking device 910 is a face tracking or head tracking device that measures certain angular and positional parameters (r, θ, φ) associated with the user's head. The spatial parameters may correspond to distance and certain heading angles, such as yaw, pitch, and roll. The original set of spatial parameters x may be updated as the sensor data 910 is processed. These spatial parameter updates Y are then passed to the headphone processor 904 for further modification of the parameter set M. The processed audio data is then sent to the post-processing stage 908, which performs certain audio processing, such as timbre correction, filtering, downmixing, and other related processing. The audio is then equalized by the equalizer 912 and sent to the headphones. In an embodiment, as described in more detail in the description below, the equalizer 912 may or may not use a pressure distribution ratio (PDR) transform to perform equalization.

Figure 10 illustrates the binaural rendering system for presenting audio objects in the headphone rendering system under the embodiment.Figure 10 illustrates some in signal components when they are processed by binaural headphone processor.As shown in Figure 1000, object audio component is input to unmixer 1002, and this unmixer 1002 separates the direct component and the diffusion (diffuse) component (for example, makes direct and reverberation path separate).Direct component is input to downmix component 1006, and this downmix component 1006 utilizes phase shift information that surround sound channel (for example, 5.1 surround) is downmixed to stereo.Direct component is also input to direct content binaural renderer 1008.Two two-channel components are then input to dynamic timbre equalizer 1012.For object-based audio input, object position and user control signal are input to virtualizer manipulator (virtualizer steerer) component 1004.This produces the object position of scaling, and the object position of scaling is input to binaural renderer 1008 together with direct component. The diffuse component of the audio is input to a separate binaural renderer 1010 and combined with the rendered direct content by a summer circuit before being output as two-channel output audio.

Figure 11 shows a more detailed representation of the binaural rendering system of Figure 10 under an embodiment. As shown in the diagram 1100 of Figure 11, multi-channel and object-based audio is input to a demixer 1102 for separation into direct components and diffuse components. Direct content is processed by a direct binaural renderer 1118, while diffuse content is processed by a diffuse binaural renderer 1120. After downmixing 1116 and timbre equalization 1124 of the direct content, the diffuse audio component and the direct audio component are then combined by an adder circuit for post-processing such as by a headphone equalizer 1122 and other possible circuits. As shown in Figure 11, certain user input and feedback data are used to modify the binaural rendering of the diffuse content in the diffuse binaural renderer 1120. For an embodiment of the system 1100, playback environment sensors 1106 provide data regarding the listening room properties and noise estimation (ambient sound level), head/face tracking sensors 1108 provide head position, orientation, and size data, device tracking sensors 1110 provide device position data, and user input 1112 provides playback radius data. This data can be provided by sensors located in the headphones 116 and/or the device 104. The various sensor data and user input data are combined with content metadata, which provides object position and room parameter information in the virtualizer manipulator component 1104. This component also receives direct and diffuse energy information from the demixer 1102. The virtualizer manipulator 1104 outputs data including object position, head position/orientation/size, room parameters, and other relevant information to the diffuse content binaural renderer 1120. In this way, the diffuse content of the input audio is adjusted to accommodate the sensor and user input data.

While the optimal properties of the virtualizer manipulator are achieved when sensor data, user input data, and content metadata are received, beneficial performance of the virtualizer manipulator can be achieved even in the absence of one or more of these inputs. For example, when processing legacy content (e.g., an encoded bitstream that does not include binaural rendering metadata) for playback through traditional headphones (e.g., headphones that do not include various sensors, microphones, etc.), beneficial results can be achieved by providing the direct energy output and diffuse energy output of the demixer 1102 to the virtualizer manipulator 1104 to generate control information for the diffuse content binaural renderer 1120, even in the absence of one or more other inputs to the virtualizer manipulator.

In an embodiment, the rendering system 1100 of Figure 11 allows binaural headphone renderer to provide personalized based on the sensing of interaural time difference (ITD) and interaural level difference (ILD) and head size efficiently.ILD and ITD are important clues about azimuth, and this azimuth is the angle of this audio signal relative to head when audio signal is generated in the horizontal plane.ITD is defined as the difference of the arrival time of sound between two ears, and the difference of the sound level that ILD effect uses to enter ear provides positioning clue. Generally accepted is that ITD is used to locate low-frequency sound, and ILD is used to locate high-frequency sound, and both are used to comprise high-frequency and low-frequency content.

The rendering system 1100 also allows for adaptive source distance control and room modeling. It further allows for direct extraction and processing of diffuse/reverberant (dry/wet) content, optimization of room reflections, and timbre matching.

HRTF model

In spatial audio reproduction, certain sound source cues are virtualized. For example, sounds intended to be heard from behind the listener can be produced by speakers physically located behind them, so that all listeners perceive these sounds as coming from behind. With virtual spatial presentation through headphones, on the other hand, the perception of audio from behind is controlled by the head-related transfer functions (HRTFs) used to generate binaural signals. In embodiments, the metadata-based headphone processing system 100 may include certain HRTF modeling mechanisms. The foundation of such systems is generally built on a structural model of the head and torso. This approach allows algorithms to be built on a core model in a modular approach. In this algorithm, the modular algorithm is referred to as a "tool." In addition to providing ITD and ILD cues, the model approach provides reference points relative to the position of the ears on the head (more broadly, the tools built on the model). The system can be tuned or modified based on the user's anthropometric characteristics. Another benefit of the modular approach is that it allows certain features to be emphasized to amplify specific spatial cues. For example, certain cues can be exaggerated beyond what the acoustic binaural filters would give an individual. FIG12 is a system diagram illustrating different tools used in an HRTF modeling system for use in a headphone rendering system, under an embodiment. As shown in FIG12 , after at least some input components are filtered 1202, certain inputs (including azimuth, elevation, fs, and range) are input to a modeling stage 1204. In an embodiment, the filter stage 1202 may include a snowman filter model consisting of a spherical head on top of a spherical body, and taking into account the contribution of the torso as well as the head to the HRTF. The modeling stage 1204 computes the pinna model and the torso model, and the left and right (l, r) components are post-processed 1206 for final output 1208.

Metadata structure

As described above, the audio content processed by the headphone playback system includes channels, objects, and associated metadata that provides the spatial and processing cues necessary to optimize the presentation of the audio through the headphones. Such metadata can be generated as authored metadata from an authoring tool and as hardware-generated metadata from one or more endpoint devices. Figure 13 illustrates a data structure that enables metadata delivery to the headphone presentation system, under an embodiment. In an embodiment, the metadata structure of Figure 13 is configured to supplement metadata delivered in other portions of a bitstream, which can be packaged according to known channel-based audio formats (such as Dolby Digital AC-3 or Enhanced AC-3 bitstream syntax). As shown in Figure 13, the data structure consists of a container 1300 containing one or more data payloads 1304. Each payload is identified within the container using a unique payload identifier value that provides an unambiguous indication of the type of data present in the payload. The order of the payloads within the container is undefined. The payloads may be stored in any order, and a parser must be able to parse the entire container to extract relevant payloads and ignore irrelevant or unsupported payloads. Protection data 1306, which can be used by a decoding device to verify that the container and the payload within the container are error-free, follows the last payload in the container. An initial portion 1302 containing synchronization, version, and key-ID information precedes the first payload in the container.

The data structure supports extensibility through the use of versioning and identifiers for specific payload types. Metadata payloads can be used to describe the nature or configuration of an audio program delivered in an AC-3 or enhanced AC-3 (or other type) bitstream, or can be used to control audio processing algorithms designed to further process the output of the decoding process.

Containers can be defined using different programming constructs based on implementation preferences. The following table shows an example syntax for a container under an embodiment.

Examples of possible syntaxes for the variable bits of the example container syntax provided above are shown in the following table.

Examples of possible syntax for payload configurations for the example container syntax provided above are shown in the following table.

The above syntax definitions are provided as example implementations and are not intended to be limiting, as many other different program structures can be used. In an embodiment, a method called variable bit is used to encode many fields within the payload data and container structure. This method enables efficient encoding of small field values with scalability to express arbitrarily large field values. When variable_bit encoding is used, the field consists of one or more groups of n bits, each group followed by a 1-bit read_more field. At a minimum, an n-bit encoding requires n+1 bits to be sent. All fields using variable_bit encoding are interpreted as unsigned integers. Various other different encoding aspects can be implemented according to practices and methods known to those of ordinary skill in the art. The above table and Figure 13 show example metadata structures, formats, and program content. It should be noted that these are intended to represent an example embodiment of metadata representation, and other metadata definitions and content are also possible.

Headphone EQ and Correction

As shown in Figure 1, certain post-processing functions 115 can be performed by the renderer 112. As shown in element 912 of Figure 9, one such post-processing function includes headphone equalization. In an embodiment, equalization can be performed by obtaining blocked ear canal impulse response measurements for different headphone placements for each ear. Figure 14 shows an example case of three impulse response measurements for each ear in an embodiment of headphone equalization processing. The equalization post-processing calculates the fast Fourier transform (FFT) of each response and performs RMS (root mean square) averaging of the resulting responses. The responses can be variable, octave smoothed, ERB smoothed, etc. The processing then calculates the inverse of the RMS average |F(ω)| by constraining the inverse amplitude response at mid- and high frequencies (+/- x dB). The processing then determines the time domain filter.

Post-processing can also include a closed-to-open conversion function. The Pressure Distribution Ratio (PDR) method involves matching the acoustic impedance between the eardrum and the free field for closed-back headphone designs by modifying how measurements are obtained for free-field sound transmission based on the direction of the earliest arriving sound. This indirectly enables matching the eardrum pressure signal between closed-back headphone and free-field equivalent conditions without the need for complex eardrum measurements.

FIG15A illustrates a circuit for calculating free-field acoustic transmission under an embodiment. Circuit 1500 is based on a free-field acoustic impedance model. In this model, _P1 (ω) is the Thevenin pressure measured at the entrance of an obstructed ear canal using a speaker positioned θ degrees around the median plane (e.g., approximately 30 degrees to the left and in front of the listener), which is used to extract the direct sound from the measured impulse response. The measurement of _P1 (ω) can be performed at the entrance of the ear canal or at a distance (x mm) from the entrance inside the ear canal (or at the eardrum) using the same speaker in the same placement used to measure _P1 (ω) (which is used to extract the direct sound from the measured impulse response).

For this model, the ratio P ₂ (ω)/P ₁ (ω) is calculated as follows:

FIG15B shows a circuit for calculating headphone sound transmission under an embodiment. Circuit 1510 is based on a headphone acoustic impedance simulation model. In this model, _P4 is measured at the entrance of an obstructed ear canal using a headphone (RMS averaged) steady-state measurement, and _P5 (ω) is measured at the entrance to the ear canal or inside the ear canal at a distance from the entrance (or at the eardrum) for the same headphone placement used to measure _P4 (ω).

For this model, the ratio P ₅ (ω)/P ₄ (ω) is calculated as follows:

The Pressure Distribution Ratio (PDR) can then be calculated using the following formula:

Aspects of the methods and systems described herein can be implemented in a suitable computer-based sound processing network environment for processing digital or digitized audio files. Portions of the adaptive audio system can include one or more networks comprising any desired number of individual machines, the machines comprising one or more routers (not shown) for buffering and routing data transmitted between the computers. Such a network can be built on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof. In embodiments where the network includes the Internet, one or more machines can be configured to access the Internet via a web browser program.

One or more of the components, blocks, processes, or other functional components may be implemented by a computer program executed by a processor-based computing device that controls the system. It should also be noted that the various functions disclosed herein, in terms of their behavior, register transfers, logic components, and/or other characteristics, may be described using any number of combinations of hardware, firmware, and/or as data and/or instructions contained in various machine-readable or computer-readable media. Computer-readable media in which such formatted data and/or instructions may be contained include, but are not limited to, various forms of physical (non-transitory) non-volatile storage media, such as optical, magnetic, or semiconductor storage media.

Unless the context clearly requires otherwise, throughout the description and claims, the words "including," "comprising," and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is, in the sense of "including but not limited to." Words using the singular or plural number also include the plural or singular number, respectively. In addition, the words "herein," "herein below," "above," "below," and similar words refer to this application as a whole and not to any particular parts of this application. When the word "or" is used in a list referring to two or more items, the word encompasses all of the following interpretations of the word: any item in that list, all items in that list, and any combination of items in that list.

Although one or more implementations have been described with respect to specific embodiments by way of example, it is to be understood that one or more implementations are not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements that will be apparent to those skilled in the art. Therefore, the scope of the claims hereto should be given the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims (12)

1. A method performed by an audio signal processing device for generating binaural presentation of digital audio content for playback via headphones, the method comprising: Receive encoded signals, the encoded signals including digital audio content and presentation metadata, wherein the digital audio content includes multiple audio object signals; Receive playback control metadata, which includes local setting information; Decoding the encoded signal to obtain the plurality of audio object signals; and The digital audio content is presented in a binaural manner in response to the plurality of audio object signals, presentation metadata, and playback control metadata. The presentation metadata indicates the location, gain, and how the audio object signal is remapped in response to screen size information for the audio object signal. The binaural presentation of the digital audio content includes: for the audio object signal, remapping the audio object signal based on the presentation metadata to fit the local screen size. 2. The method according to claim 1, wherein the playback control metadata further includes room model metadata. 3. The method according to claim 2, wherein the room model metadata includes frequency-related reverberation time. 4. The method of claim 2, wherein the room model metadata includes filters for modeling room responses. 5. The method of claim 1, wherein the presentation metadata further includes an indication of the apparent source width for each object audio signal. 6. The method according to any one of claims 1-5, wherein the digital audio content further comprises a plurality of channel audio signals. 7. The method of claim 6, wherein the presentation metadata for each channel audio signal further includes an indication of whether stereo presentation rather than binaural presentation is to be used for the channel audio signal. 8. The method of claim 6, wherein the presentation metadata further includes an indication of the position of the audio channel for each channel audio signal. 9. The method of claim 8, wherein the position of the channel audio signal is indicated by a channel identifier. 10. The method according to any one of claims 1-5, wherein generating the binaural presentation of the digital audio content includes generating separate direct binaural presentation and diffuse binaural presentation, and combining the direct binaural presentation and diffuse binaural presentation. 11. An audio signal processing apparatus for generating binaural presentation of digital audio content for playback via headphones, the audio signal processing apparatus comprising one or more processors, the one or more processors being configured to: Receive encoded signals, the encoded signals including digital audio content and presentation metadata, wherein the digital audio content includes multiple audio object signals; Receive playback control metadata, which includes local setting information; Decoding the encoded signal to obtain the plurality of audio object signals; and The digital audio content is presented in a binaural manner in response to the plurality of audio object signals, presentation metadata, and playback control metadata. The presentation metadata indicates the location, gain, and how the audio object signal is remapped in response to screen size information for the audio object signal. The binaural presentation of the digital audio content includes: for the audio object signal, remapping the audio object signal based on the presentation metadata to fit the local screen size. 12. A non-transitory computer-readable storage medium comprising a sequence of instructions that, when executed by an audio signal processing device, cause the audio signal processing device to perform the method according to any one of claims 1-5.