TWI843747B - Multi-range hdr video coding - Google Patents

Abstract

Although recently high dynamic range video codec technologies have been invented, at least some applications in the total HDR video market desire some more sophisticated methods, which can work defined around more than two different dynamic range re-graded images of a same original HDR scene. Thereto the inventors developed a number of technical ways to realize a high dynamic range video encoder (900), arranged to receive via an image input (920) an input high dynamic range image (MsterHDR) which has a first maximum pixel luminance (PB_C_H50) for which the encoder has a first metadata input (922), and arranged to receive via a second metadata input (921) a master luma mapping function (FL_50t1), which luma mapping function defines the relationship between normalized lumas of the input high dynamic range image and normalized lumas of a corresponding standard dynamic range image (Im_LDR) having a maximum pixel luminance of preferably 100 nit, characterized in that the encoder further comprises a third metadata input (923) to receive a second maximum pixel luminance (PB_CH), and the encoder further being characterized in that it comprises: - a HDR function generation unit (901) arranged to apply a standardized algorithm to transform the master luma mapping function (FL_50t1) into a adapted luma mapping function (F_H2hCI), which relates normalized lumas of the input high dynamic range image to normalized luminances of an intermediate dynamic range image (IDR) which is characterized by having a maximum possible luminance being equal to the second maximum pixel luminance (PB_CH); - an IDR image calculation unit (902) arranged to apply the adapted luma mapping function (F_H2hCI) to lumas of pixels of the input high dynamic range image (MsterHDR) to obtain lumas of pixels of the intermediate dynamic range image (IDR) which is output of this unit; and - an IDR mapping function generator (903) arranged to derive on the basis of the master luma mapping function (FL_50t1) and the adapted luma mapping function (F_H2hCI) a channel luma mapping function (F_I2sCI), which defines as output the respective normalized lumas of the standard dynamic range image (Im_LDR) when given as input the respective normalized lumas of the intermediate dynamic range image (IDR), which in turn correspond to respective lumas of the input high dynamic range image (MsterHDR); the encoder being further characterized to have: - an image output (930) to output the intermediate dynamic range image (IDR); - a first metadata output (932) to output the second maximum pixel luminance (PB_CH); - a second metadata output (931) to output the channel luma mapping function (F_I2sCI); and - a third metadata output (933) to output the first maximum pixel luminance (PB_C_H50), and a corresponding HDR video decoder.

Description

Multi-range HDR video encoding

The present invention relates to methods and apparatus for encoding high dynamic range images, and in particular for video being a temporal sequence of images which may be compressed according to a compression technique such as MPEG-HEVC (e.g. television broadcasting), in particular by using (multiple) image(s) of a second dynamic range (for communication to a decoder) to represent (multiple) (primary) image(s) of a first dynamic range, the dynamic range change involving changing the image pixel brightness (e.g. from a first one normalized to a value of 1.0 to a second one normalized to a value of 1.0) by applying a function which is generally communicated as metadata together with the (multiple) second dynamic range image(s).

About 5 years ago, the new technology of high dynamic range video encoding was introduced to the world (leading to special HDR Blu-ray discs to be viewed on high-end UltraHD Premium TVs at 1000 nits, for example).

This technically novel way of processing images contrasts technically with many of the ways in which all video for the past 50 years had been coded until then, which is now known as Standard Dynamic Range (SDR) video coding (also known as low dynamic range (LDR) video coding). To represent images, a digitally coded representation of the pixel colors is necessary, and the SDR luma code definition of Rec. 709 (also known as the Opto-electrical transfer function (OETF)) is capable of encoding (using 8 or 10-bit luma words) a luminance dynamic range of only about 1000:1 because of its approximately square root function shape (luminance: Y=sqrt(luminance L)). However, this was perfectly suited for encoding images to be displayed on monitors of that era, which had a typical brightness rendering capability (of all monitors at the time) of between about 0.1 and 100 nits, the latter value being the so-called peak brightness (PB), also known as maximum brightness.

See Rec. 709 brightness code definition function cannot mathematically represent the huge range of HDR scene image brightness (for example, between 0.001 nit and the desired image coding peak brightness PB_C of 10,000 nits). HDR researchers initially solved this problem by designing a new HDR code allocation that was more logarithmic in shape, allowing for encoding of a wider range of brightness (because the visual system requires lower accuracy, i.e., the code values for brighter areas are smaller than for darker areas, so it can be understood that 50 codes in each decimal brightness allocation 2^8=256 (where ^ represents the brightness function) can encode a dynamic range of 100,000:1). This is a simple, "natural" way to encode the colors of HDR images by using the SMPTE 2084-standardized (so-called perceptual quantizer (PQ)) function.

One might naively think that encoding and decoding HDR images is all there is to it, but things are not that simple, and therefore additional coding methods have emerged, particularly in relation to the applicant's previously developed methods for HDR video coding and processing.

To get a proper understanding of what is involved or required in HDR video encoding, Figure 1 summarizes some important aspects.

Assume on the left the luminance range of all possible (PQ decoded) HDR luminances up to PB_C=5000 nits. And assume for the moment that in order for this image to look as perfect as desired, all object pixels of what will be called the master HDR image are created on a computer (how to start from, for example, a broadcast camera is explained below with Figure 2). The problem with using native HDR codecs (which only provide the technology to encode brightness up to 10,000 nits, i.e. in this example also up to 5000 nits as desired) is that if the consumer also has an expensive monitor with a 5000 nit display peak brightness (PB_D) (and if he views the image under standardized viewing environment conditions), he can perfectly view the image as intended by the creator (e.g., movie director), but if he has a different monitor (e.g., PB_D=750 nits, or PB_D=100 nits), there is an unresolved and not simple problem: how to display an image with 5000 nits PB_C on a monitor with 750 nits PB_D? There seems to be no elegant simple solution to this. Applying an accurate brightness display will show all objects perfectly at a brightness of up to 750 nits, but will clip all brighter object pixels to the same PB_D=750 nits, causing many image objects to disappear into the white spot area, which is of course not a good look. One could argue that linear scaling of the content is a solution (dividing all HDR luminance by 5000/750, which is the so-called map content-white-on-display-white approach), but darker objects (like the person in the dark area of the cave in the example scene image ImSCN3 with HDR luminance (0.05 nits) which might already be too low for a smaller dynamic range display) would thus end up being imperceptibly dark on a 750 nit display (0.05*750/5000= 0.0075 nits).

FIG. 1 also teaches that different HDR images (of different typical HDR scenes) may have quite different requirements regarding how to squeeze various HDR luminances (possibly at “arbitrary” luminance positions along the HDR luminance range DR_2) into a smaller, e.g., LDR luminance dynamic range DR_1.

The range of real-world luminance can be, for example, when both indoor and outdoor objects are viewed simultaneously through illumination_contrast*object_reflection_contrast =(1:100)*(1:100), and while the luminance in the represented image need not or will not generally be identical to the original scene luminance, for a good HDR representation image one would expect pixel luminance to range up to at least 1000 nits, and to start with a minimum of at least 0.1 nit or less (so DR_im ＞= 10,000:1). Furthermore, good HDR images can be more about the intelligent distribution of the luminance of various objects along the luminance range than about physical dynamic range itself (not to mention misunderstanding the amount of bits it's directing, which by definition isn't true for non-linear luminance codes, and 10-bit luminance images can be only some HDR images as well as SDR images).

Figure 1 shows several typical illustrative examples of the many possible HDR scenes that a future HDR system (e.g. connected to a 1000 nit PB_D display) may need to be able to handle correctly, i.e. by displaying the appropriate brightness for all objects/pixels in the image.

For example, ImSCN1 is a sunny outdoor image from a western movie (it has mostly bright areas, brighter than the average for cloudy images, which should ideally appear brighter than on a 100 nit display to provide a sunny look than a rainy look, e.g., with an average brightness of, say, 400 nits). On the other hand, ImSCN2 is a very different type of image, namely a night image dominated by dark areas (and, e.g., good visibility thereof), however what makes this an HDR image and not a simple dark SDR image is that there are also bright pixels in spots below street lights, and possibly in lit windows of houses, and there are even very bright pixels (e.g., 3000 nits) on the lamp surface of the street lights.

What makes this ImSCN1 clear relative to the darkness of ImSCN2? Not necessarily the relative brightness, at least not in the SDR paradigm (it will be possible for both images to have pixel brightness spread over the range between 0.1 and 100 nits, although the spatial distribution of such brightness, and in particular the histogram, may be different). The thing that makes HDR image rendering different from how it was always done in the SDR era that ended only a few years ago, is that SDR has such a limited dynamic range (about PB=100 nits, and a minimum black level MB of about 0.1 to 1 nit) that only most of the intrinsic reflectivity of objects can be displayed in SDR (it will fall between 90% for a good white and 1% for a good black). Under technically controlled uniform illumination, it will be good for identifying objects (with some amount of brightness from their reflections and of course from their chromaticity), but not so good for the beautiful variations in lighting itself that can be present in natural scenes, and what kind of impact that can have on the viewer (sunlight coming out of a window, or plasma from wizard radiation). If the display allows it (and therefore the image encoding and processing technology should), walking in a forest really sees the sunlight through the trees, i.e., one wants to see bright and vivid sunlight-lit clothes as one walks from shadow into bright sunlight, rather than just a somewhat more yellowish impression as on an SDR display. And fire and explosions should also have optimal visual impact, at least when PB_D allows it.

In SDR, night images can be made somewhat darker than normally lit images, as can be perceived in the luminance histogram, but not too dark, otherwise it would just render an image that is too dark and ugly (maybe even mostly invisible) (this is where the common practice of making night images still relatively bright (but blue) comes in). Moreover, in 100 nit TV, or in encoding at 100 nits, there is just no room for anything that is too bright. So objects must be displayed independently of their lighting, and all the high-contrast lighting that can sometimes occur in a scene cannot be faithfully displayed at the same time. In practice, this means that a highly bright sunny scene must be rendered using almost the same display luminance (0 to 100 nits) as a dark rainy scene, and even a night scene.

In real life, human vision will also adapt to the amount of available light, but not that much (most people do perceive gradual darkening in real life, or they are in darker or quite bright environments). And one should not forget that the television displaying the image is not a simulation of the adapting eye, but a simulation of the real life environment, as good as it can be given the viewing environment and other technical limitations. So, one would like to display the image with all the spectacular local and also temporal lighting effects that can be artistically designed into the image, to get a more realistic rendering (at least if the end viewer has an HDR display available). Exactly what is the appropriate brightness for, say, a lightsaber in a dark room, will be left to the color grader who creates the master grading(s) (for simplicity of the teachings in this patent, it is assumed that various motion range images, at least two of the most extreme different motion ranges, are created by a human grader, but the images could similarly be created by automated software), and this application will focus on the technical components required to create and process such images for a variety of market players with potentially different needs.

On the left axis of Figure 1 are the luminances of objects that can be directly displayed on a 5000 nit PB_D (reference) display when one wants to see them with a (for example) 5000 nit PB_C master HDR grade (i.e. the image grader generated the image assuming that a typical high quality HDR TV at home would have a 5000 nit PB_D and that he could actually sit in such a home viewing room and in the representation of the grade on such a graded display). These pixel luminances must be specified and rendered bright enough, around for example 500 nits on average, if one wants to deliver not only an approximate illusion of the original HDR scene that was captured, but also an actual scene of a cowboy in a bright sunlit environment.

For night scenes, one wants mainly dimness, but the main character on the motorcycle should be well recognizable, i.e. not too dim (e.g. around 5 nits), and at the same time there can be quite high brightness pixels, e.g. street lights, e.g. around 3000 nits on a 5000 nit display, or around the peak brightness on any other HDR display (e.g. 1000 nit PB_D). The third example ImSCN3 shows what is now also possible on an HDR display: both many (semantically more related to not only lights, i.e. related to many in-area details, like a sunlit tree) very bright and many important but very dark pixels can be rendered at the same time. ImSCN3 shows what can be regarded as a typical and relatively difficult HDR scene image, with a dark cave with an opening through which the clear outside can be seen. For this scene, one might want to produce sunlit objects (like trees) less bright than a scene where one wants to give the impression of just a bright sunny scenery, e.g. around 400 nits, which should be more in tune with the essentially dark characters inside the cave. A color grader might want to best harmonize the brightness of all objects (already in the PB_HDR=5000 nit master HDR image) so that nothing looks unduly dark or bright and the contrast is good, e.g. a person standing in the dark in this cave could be encoded at around 0.05 nits in the master HDR graded image.

Once this master HDR image has been created, the artistic question (even before it is formulated in terms of implementation technology) is how this image should then be regraded to a different dynamic range, such as a conventional SDR image of at least 100 nits PB_C.

It helps to clarify when the relationship between luminances is given, so this will be done in this patent when convenient. In practical technical terms, luminance will be encoded as brightness via a brightness code allocation function (also known as a photoelectric transfer function (OETF)), and therefore all relationships between luminances can also be formulated, such as a function that calculates the output luminance L_out from the input L_in, and the same is true for the relationship between equivalent brightness.

Perhaps somewhat confusingly, it is also possible to formulate luminance in a normalized (e.g., using a maximum normalized luminance equal to 1.0) way, and define all actions on this normalized luminance. This has the advantage that the normalized HDR color gamut exactly overlaps the LDR color gamut (provided that the image pixel colors of both are defined in the same set of RGB primaries), and thus luminance variations can be displayed in this single normalized color gamut. Obviously, the relative position of a normalized LDR luminance that should be displayed with exactly the same absolute luminance as an HDR luminance defined in the HDR luminance luminance range with PB_C=5000 will have a different relative height (i.e., the luminance mapping required for a particular HDR pixel luminance when creating the corresponding LDR image pixel luminance as a function of relative/normalized variation of height in this normalized color gamut can then be displayed in this color gamut representation). The relationship between absolute and relative luminance is simple: L_norm=L_abs/PB_C, where PB_C is any maximum luminance encoded, e.g. 5000 nits for HDR encoding, and 100 nits for SDR as per standard protocol.

The last important thing to learn from Figure 1 (because all techniques must behave accordingly) is that depending on what kind of objects (i.e., their pixel brightness) are being processed in what type of HDR scene, there can be different high-level approaches on how to rescale the pixel brightness (i.e., the brightness transformation).

For example, an object in the dark (like a motorcyclist) can be rendered by equating the absolute luminance of all regraded images (which involves corresponding proportionally adjusted changes in normalized luminance), in particular the starting master HDR image on the left, the corresponding SDR image on the right, and any medium dynamic range (MDR) images in between, e.g., with a PB_C = PB_MDR = 800 nit display optimized (with correct object luminance) for direct display on an 800 nit PB_D display (e.g., for a consumer who has purchased such a display and gets a 5000 nit PB_C HDR image, e.g., from his cable TV provider, or via a satellite set-top box, or from the Internet, etc.). This makes sense since the creators of the content wanted to convey a dark atmosphere where only the motorcycle is visible, and for the sole reason that this display can do so (since it has a larger luminance range ending at a higher PB_D to show the luminance of all objects in the scene), making it appear brighter on a brighter display would be poor.

Objects (like the sun) would probably follow a completely different principle, namely the map white-on-white method, where the highest possible value in any image representation is always given, namely PB_C. Obviously, other types of objects may follow other types of rules, and could go on (for example, cowboys would follow the scaled middle gray principle), but the reader understands that it must be sufficient to have one of the techniques that allows an almost "arbitrary" distribution of brightness for all pixels, rather than, for example, a fixed one (as the simple technique would specify).

While Figure 1 simply summarizes the requirements for creating diverse HDR images (across such applications as movies, live sports broadcasting, etc., which are subject to different technical constraints), the problem for HDR technology developers remains how to encode HDR images, and also how to transform HDR images so that they can be optimally displayed on any display with a PB_D smaller than the encoded PB_C (i.e., the brightest pixel that may occur at least once in the video). Capturing HDR scene images, and equally important, artistically directing and lighting HDR scenes are also technical skills, but this application does not necessarily focus on that aspect.

The simplest thing to envision is to just encode HDR pixel brightness (ignoring the complexity of display adaptation (DA), i.e. how to map a PB_C1 image to an image for a less capable display). The problem is that the Rec. 709 OETF can only encode a brightness dynamic range of 1000:1, i.e. a new HDR OETF (or actually its inverse, the EOTF) had to be invented. The first HDR codec called HDR10 has been introduced on the market, which is used for example to create new black band HDR Blu-ray discs, and it uses a more logarithmically shaped function called Perceptual Quantizer (PQ) function standardized in SMPTE 2084 as OETF, and it allows to define brightness for luminances between 1/10,000 nits and 10,000 nits (enough for practical HDR video production). Furthermore, it has the nice property that the brightness codes it produces are adapted to how human vision works (the kinds of perceived gray values the brain uses to characterize different luminances in a scene, which is a nice property for efficiently re-grading certain gray value objects, and for efficiently representing luminance both (as the brain does)). After the calculation of the luminance, one has only a 10-bit pixel plane (or more precisely, also two chrominance planes, Cb and Cr 3-bit planes), which can be further processed classically below the line, "as if" they were mathematically SDR images, e.g. MPEG compressed (this is an important limitation, since it avoids the redesign and redeployment of several pre-existing technologies in the overall video pipeline).

A significant technical difficulty with HDR10 images remains how to properly display them on less capable displays (e.g. less than the 2000 PB_C for which the HDR content was made). For example, the most interesting (darker) parts of an image with PB_C = 1000 nits will typically look 10x too dark if one just linearly maps white on white (encoded image maximum white, aka encoding peak luminance PB_C to e.g. SDR display peak luminance PB_D), which would mean that night scenes ImSCN2 become unviewable. Due to the logarithmic nature of the PQ OETF, HDR10 images are visible (when only luminance is rendered, i.e. decoded with a wrong EOTF), but with an ugly deteriorated contrast that makes them look among other things washed out and brightness incorrect.

A simple system for creating HDR video content (e.g. in a broadcast scenario) is explained with reference to FIG2 . Again, to keep the explanation simple, the details of non-linear brightness or R'G'B' pixel color code assignment are not considered (the so-called Opto-optical approach: OOTF, using normal (absolute) luminance throughout the chain). Using the camera (201) exposure (EXP), it is possible to choose which object's luminance is faithfully recorded and at what relative value (because the camera functions as a relative luminometer for all spatial positions, or a relative colorimeter producing RGB triplets). Since both the camera sensor and the N-bit mathematical representation of the color components actually have an ultimate range (starting at a minimum and ending at a maximum), it is reasonable not to expose the details of the billion-nit sun, but at least to clip the luminance or RGB values to their maximum values. In a virtually infinite range, the exposure choice can be "corrected" by a later luminance remapping, but in any case this fact indicates to the reader that there is no "natural" obvious mapping of scene luminance to the luminance to be displayed (the reference to the latter luminance is known as display-dependent colorimetry, and is in fact the most important one). The linear luminance image LIN_HDR is generally first subjected to OOTF mapping (202). This already exists to some extent in the SDR era, and corrects for the fact that human vision in the generally darker viewing environment of a nighttime living room when watching TV requires higher contrast for a similar visual experience, so the OOTF is generally a soft gamma function. However, particularly when mapping scenes with a considerable dynamic range on a general display (205) with a smaller dynamic range (even when it is a high quality 4000 nit reference monitor), some kind of artistic optimization of the brightness of the pixels of various objects can be achieved by applying a possibly arbitrary curve (which will be referred to herein as a grading) by the grading unit 203. Particularly for offline high quality production, the grading effect may be appreciable in order to place a so-called creative look or look in the master HDR image MAST_HDR (which according to the invention still has to be further technically processed, e.g. advantageously encoded). The resulting image then looks optimal and can be sent via a certain image communication connection 204 to a display 205, where a human grader can check whether the image is as desired, or continue to fine-tune at least one brightness mapping function via a user interface control unit 206 (e.g. a grading console). This arbitrary grading forms the master look (not to be confused with an arbitrary re-grading (secondary look)) in order to obtain, e.g., the best possible corresponding SDR image, which can be called the master SDR image (e.g. when forming part of a video coding strategy as described below). Although only a simpler topology is explained to the reader, the reader will understand that in practice there may be different actual implementations depending on, for example, whether there is a real-life broadcast using only a single HDR camera or many mixed SDR and HDR cameras, or previously determined HDR images and corresponding re-graded SDR master images, which now need to be co-encoded according to coding principles (e.g., ETSI1 or ETSI2 principles seen below), etc.

The applicants understood that since there is a mathematical re-grading relationship between the various possible re-graded MDR images starting from the master HDR, if these functions can be technically and practically extracted, it is possible to actually encode the entire spectrum of different dynamic range functions (by sending only one of them) and at least one brightness mapping function to create another image from the one that has actually been sent (already illustrated with FIG. 1). The first introduction of this possibility and the subsequent technical coding concept was done in WO2011107905.

It has been found to be reasonable to define a brightness mapping function F_L for transforming a (e.g. 5000 nits PB_C) master HDR image luminance to an SDR image luminance, i.e., to allow the grader to define the desired regrading behavior between the most extreme image representations, and then recalculate the display-adapted brightness mapping function F_L_DA for calculating the intermediate MDR image pixel luminances corresponding to any possible 5000 nits PB_C M_HDR image luminance.

When the applicant subsequently standardizes, there are then two logical choices of the image to actually transmit (as a single image of the entire spectrum of rescalable images for different dynamic ranges, in particular the PB_C endpoint, since it can often be assumed that the lower endpoint MB is roughly fixed, e.g. 0.01 nit) to any receiver: the main HDR image or the corresponding SDR image (one should stop for a second to understand that in this case, since L_HDR_reconstructed = F_L_inverse[L_SDR], and in fact since the HDR image is still also passed to the F_L function, the actual normal SDR image is transmitted instead of the HDR image).

The second coding option, first standardized under ETSI TS 103 433-1 (note the -1 ; abbreviated to ETSI1 ), is quite useful when the technical limitation is that many legacy displays need to be served in an undisturbed way (in fact an old SDR display only gets an SDR image and does not need to know that this also encodes an HDR image, it can directly display the SDR image and immediately get a very good SDR rendition of the HDR scene, in fact the display can display this HDR scene as much as possible). Note that there are technical limitations to be able to reconstruct the original master HDR image with sufficient precision on any receiving side (like the need for reversibility of SDR image colors), which is part of the technical assumption that led to the standard encoding (de)coding method as defined.

ETSI TS 103 433-2 ( ETSI2 ) is a coding alternative that actually delivers a master HDR image to a receiver, and in which function(s) F_L (actually as will be shown below, although for illustration the system can be imagined as if there were a single global F_L function for all pixel luminances in the delivered image, for technical reasons a set of subsequently applied mapping functions is used) acts to compute the image for optimal display on a display with PB_D < PB_C_master (i.e. for so-called display adaptation). Various consumers can choose the system they wish to adopt, for example a cable operator delivering ETSI2 HDR will deploy STBs to its users that will decode and optimize for whatever display the user happens to have at home.

FIG3 first shows the components of a general single-image-plus-function HDR video communication system (encoder + decoder) at a bird's-eye view level, without limiting the general system to SDR communication type for the purpose of explaining the basic concepts.

The color converter 302 gets as input the MAST_HDR images from the image source 301 (e.g. when they are captured by a camera and graded by the system illustrated in FIG. 2 , and then passed through some professional video communication system to the broadcast station side encoder 321 , which will transmit a TV program over the air or via a TV cable network, for example). A set of color transformation functions F_ct is then applied (in this example, for example, as determined by graded automatic software, e.g., the applicant's automatic HDR to SDR conversion technique, which defines the F_ct function based on image characteristics (e.g., histogram, etc.); the specific details may be left out of the description of this application, as it is only required that such an optimization function exists for any image or set of time-sequential images), including at least a brightness mapping function F_L, to obtain the corresponding SDR brightness of the brightness of the pixels of the master HDR image (MAST_HDR). For ease of understanding, the reader may assume for simplicity that F_L is a 4th power luminance mapping function (L_out_SDR=power(L_in_HDR; ¼)) used to derive the 1.0 SDR output luminance of a pixel in the PB_C SDR output image Im_LDR (i.e., the luminance range on the right side of Figure 1) normalized to 100 nits.

Because there are now “normal” SDR images, they can be compressed using standard video compression techniques (e.g., MPEG standards (such as HEVC or MPEG2), or similar standards (such as AV1)), which are performed by the video compressor 303.

Since the receiver must be able to reconstruct the master HDR image from the corresponding compressed SDR image Im_COD received, in addition to the actual pixelated image to be transmitted, the color mapping function F_ct must also enter the video compressor. Without limitation, it can be assumed that the function is stored in metadata, for example, by means of a SEI (supplemental enhancement information) mechanism or similar techniques. Finally, the formatter 304 performs all the necessary steps to format (put data blocks, etc.) the video stream for a communication medium 305 of any technology, for example, for storage on a Blu-ray disc, or for DVB communication via satellite, etc. (this detail can be found by a person having ordinary knowledge in the respective technical field and is not relevant for understanding the concept of the invention).

After MPEG decompression in the video receiver 320 is performed by the video decompressor 307 (after having passed through the unformatter 306), the SDR image can be interpreted by the receiver by applying the standard Rec. 709 EOTF (to obtain an image for an SDR display), but the receiver can also decode the received Im_COD image differently to obtain a reconstructed HDR image Im_RHDR.

This is performed by a color converter 308, which is configured to convert an SDR image such as compressed Im_RLDR to an image of any non-SDR dynamic range (i.e., PB_C above 100 nits, and typically at least above 6x). For example, the original master image Im_RHDR at 5000 nits can be reconstructed by applying the inverse color transform IF_ct of the color transform F_ct used on the encoding side to generate Im_LDR from MAST_HDR (and which are received in the metadata and passed through the color converter 308). Alternatively, a display adaptation unit 309 may be included which converts the SDR image Im_RLDR to a different dynamic range, e.g., if the display 310 is a 3000 nit PB display, the optimally graded Im3000 nit, or a 1500 nit or 1000 nit PB image for a lower PB_D display, etc. It has been assumed without limitation that the video decoder and the color converter are in a single video receiver 320. A reader with ordinary knowledge will appreciate that many different topologies may be similarly designed with, for example, the decoding function separated in a set-top box to be connected to a display, the display acting only as a basic display (dumb display) for pre-optimized images as received, or it performing further image color conversion, etc.

FIG . 4 briefly summarizes the principles of the applicant's brightness and color mapping technique as standardized in ETSI2 (in fact it details a color converter 302 which is roughly introduced in FIG. 3 according to the ETSI2 decoding principles (or similarly the ETSI1 encoding principles)), because it must be understood in order to understand some of the more specific embodiment techniques of the present application.

The input should be PQ defined YCbCr pixel colors (i.e., luminance Y and chrominance Cb and Cr color components per pixel). First, the luminance is linearized to normal linear brightness L_in by the EOTF application unit 401 which must use the SMPTE 2084 PQ EOTF. The complete re-grading process from the input HDR pixel colors to the SDR output pixel colors can then be defined again using the normal (physical SI and CIE universally defined) brightness. After this, luminance processing may be performed by a luminance processor 401 which implements the overall F_L mapping as desired but through judiciously chosen sub-units (these units 402, 403, etc. are technically designed to facilitate the needs of various HDR applications, e.g., automatic grading, ease of human grading, complexity of IC design, etc.).

First, the brightness equalizer applies a fixed curve transformation whose shape depends only on the peak brightness PB_C_H of the input HDR image (PB_C_H = e.g., 5000 nits) by applying one of the PB-dependent curves defined as follows: Y’HP=log(1+(RHO-1)*power(L_in/PB_C_H; 1/(2.4)))/log(RHO) [Equation 1] Where RHO= 1+32*power(PB_C_H/10000;1/2.4)             [Equation 2]

This maps all luminances to a perceptually uniform grey brightness Y'HP. If PB_C_HDR = 10000 nits, this curve corresponds closely to the SMPTE 2084 PQ curve, which is known to be perceptually uniform. For input images with lower PB_C_HDR, the curve scales well (in fact, it represents a subcurve on the 10000 nit curve that ends at, for example, 3000 nits), resulting in a less steep log-gamma curve for the darkest colors in the normalized [0-1.0]/[0-1.0] input/output luminance axis representation. That is, the rest of the process is already well pre-normalized to begin with.

The black-white level shifter 403 may then apply an additive white level offset WLO and a black level offset BLO where desired.

The utility of white level shifting can be understood as follows. Assume that the content creator grades his images on a system set at PB_C=4000 nits (i.e., his reference grading monitor has a PB_D of 4000 nits, for example), however, throughout the entire video, he never actually produces the image using a pixel maximum brightness higher than, say, 1500 nits (the video maximum, which is a different thing from the encodable maximum PB_D). Then, because the SDR luminance dynamic range is small enough as it is, it makes sense to rescale the input HDR to drop those unused values by 1500 to 4000 nits (because a dynamically adjustable luminance mapping is being used, which can be optimized instantaneously per image/video anyway). 1500/4000 corresponds to a normalized (input) HDR luminance of 0.375, so this value can be mapped to the maximum value of the scaled HDR brightness Y’HPS by dividing by 2.6.

To be precise, according to the ETSI2 standard, the following calculation is performed: Y’HPS=(Y’HP-BLO)/(1-WLO-BLO) [Equation 3]

The WLO and BLO are communicated in metadata that is communicated together with or may be associated with the received video image.

Black level shifting is useful to get a higher contrast look for the SDR corresponding regraded image, but care should be taken that the ESTI1 received image should remain reversibly mapped to the HDR image, i.e. not too much black pixel detail should be lost (this is also the reason for the parallel gain limiter, not shown in Figure 4).

Basically, black level shift can be simply understood as putting a certain HDR "black" color at 0.0 in SDR, or more precisely, via a unit 403 that is prepared for HDR to SDR brightness mapping (i.e., using normalized brightness that is still in HDR, meaning having a relative distribution that can be used to get a good look on an HDR display, and a bad, still unoptimized look on an SDR display).

Then, a coarse dynamic range converter 404 applies the main brightness conversion to obtain SDR brightness (i.e., with a good first redistribution of the object brightness to get a reasonable look on an SDR display). For this, ETSI2 uses a curve consisting of a slope controllable linear section for the darkest HDR normalized luminance (the slope of this section is called shadow gain), another linear compression section for the brightest normalized HDR input luminance Y'HPS (with slope control parameter high brightness gain), and a controllable parabolic section that smooths it together by providing a good SDR look for the mid-tones (with control parameter mid-tone width, and the math is readable in the standard, and is only re-explained in this application (in a simple and digestible way as necessary) to the extent necessary to understand the new inventive embodiments according to the present insight). So the output brightness Y'CL of this coarse dynamic range converter 404 is defined for the first time in the SDR range, or in the SDR relative brightness distribution statistics.

The technical (and artistic) suggestion to the content creator of this unit 404 is that the grader can optimize well how bright he needs to be to produce the darkest pixel at the expense of intra-object contrast of other objects containing brighter pixels (because of the limited SDR luminance range), but he can co-tune, for example, high brightness gain. Shadow gain can be understood as for, for example, a person standing in a dark shadow area of a cave at 0.05 nit luminance. If it is displayed on an SDR display with a white-on-white criterion (i.e., a normalized mapping function, which is a constant function with a 45 degree slope of the diagonal line of the normalized luminance function graph), it will be found that its normalized luminance in HDR is 0.05/5000, which stays at the same normalized luminance due to the constant mapping used to roughly map the SDR luminance, i.e., after making it equiabsolute, these pixels should be displayed on the SDR display at (1/100000)*100, i.e., the minimum black ("0" drive signal) on the display and are invisible. Therefore, such brightness must be significantly improved, even in more log-uniform HDR and SDR relative gray value or brightness representations, to obtain SDR brightness that is sufficiently visible and results in object texture discernibility within individual figures (e.g., individual pixel brightness spanning 0.3 to 1.2 nits displayed on an SDR display). Thus, depending on how deep into the HDR luminance range that person happens to fall (which, as taught above, will depend on a combination of factors such as the HDR scene architecture, scene lighting, camera exposure, and the artistic master HDR grading chosen by the content creator), the encoder (e.g., one that makes a choice of the appropriate F_L portion of this first coarse luminance mapping to regrade the master HDR input to an optimal or appropriate human grading of corresponding SDL pixel luminance) will choose the appropriate shadow gain for processing those darkest pixels of this particular image (i.e., image content optimization). Note that in practice in ETSI, shadow gain SG is defined as an automatically scaled correction based on the ratio of the peak luminance (at least their brightness representation) of the input and output images. Under the isolumina principle, by starting from the normalized HDR luminance as: L_200=Y’200*L_HDR, it is reasonable to increase the luminance over the normalized luminance range corresponding to, for example, only 200 nits PB_C (or more precisely, according to the values of Equation 1 and Equation 2 above: Y’HP=Y’200=v(PB_C_H/200;RHO(200)), v is the pseudo-logarithmic equation of Equation 1 above). However, this generally gives an image that is too bright and low-contrast, so the grader may use an exposure gain correction: SG=expgain*Y’200, which will be a dimming factor that moves SG back towards the diagonal value of 1.0 and brings some darkness back to the SDR image (he would generally not choose expgain=1/Y’200, because the SDR normalized brightness would then be equal to the HDR normalized brightness and be too dark again; SG would, for example, fall between 1.0 and 1.8).

This curve acts like a non-linear luminance compression "spring" that squeezes much of the HDR luminance from a potentially much larger luminance dynamic range into the much smaller SDR DR. Because what is used is not a fixed curve which "should never be too unreasonable on average", but the encoder can apply a curve that has been optimized, the resulting SDR image will not be bad for many HDR scenes (not all HDR scenes are equally complex, for example, sometimes there is a slightly shadowed area right next to an evenly sunny area, and then although the simplest system will produce problems like clipping to white, a less complex intelligent HDR to SDR mapping (like the three-part curve of unit 404) will often do a good job of creating a suitable SDR regraded image of an HDR master image (for example, the one from the HDR camera of the content creator of the real life event capture).

However, several other scenarios may be more complex, and some content creators may also have a higher degree of professional needs in fine-tuning their artistic content (e.g., a Hollywood film director or DOP).

Thus, the next unit (customizable curve applicator 405) allows the content creator (again, whether human or intelligent automation with various rules encoded in its algorithm) to apply a customizable and possibly arbitrarily shaped fine-scale graded brightness mapping function F_L_CU to the Y'CL pre-graded brightness, producing the graded LDR brightness Y'GL (the only requirement on the function is that it is non-decreasing, and typically even monotonically increasing, and typically maps 1.0 input to 1.0 output, at least as chosen in ETSI2). In practice, the shape of this function F_L_CU can be communicated to the decoder as a set of shape-defining parameters (e.g., coefficients of a polynomial) or as one of a LUT or the like.

Such a fine grading may be necessary because visual systems have a complex way of determining the impression of the grey values of perceived image objects, and/or because squeezing a large span of HDR brightness into a limited SDR DR may sometimes require considerable understanding, and/or because content creators may clearly wish to put some additional artistic touch on this customized curve F_L_CU (which shape will then generally be determined by another color user interface computer hardware and software connected on the encoding side (not shown)). In fact, on the one hand, one could say that all MDR images should be some kind of compressed representation of all the information in the master HDR image (only), but on the other hand (since images with too little contrast could give a rather weak impression, e.g., as if seen through fog), other important requirements for content creation could be to make all images look like SDR images - as realistic as possible or at least as beautiful as possible, given their more limited DR capabilities. Human vision is highly non-linear and delicate, and can quickly perceive if it has been used to too simple functions. So in addition to the crude luminance squeezing function F_C, content creators may use the ability to understand the customizable function F_L_CU to do better when it is nearly impossible to produce an SDR image that still looks as good as possible to be used in an HDR scene, and preferably looks like an HDR scene (for example, lowering the brightness of a certain luminance subrange of pixels to produce just a little more contrast between objects, e.g. for the brightness of stained glass windows in church interiors, or indoor-outdoor visual contrast in SDR images, or optimizing the chromaticity of some objects in the scene with respect to luminance by choosing a particular local shape of the F_L_CU curve, etc.).

A single simple example of the "Shadow Man" image shown in FIG6 may be used to enlighten the reader and provide him with the minimum necessary understanding of a customizable brightness mapping function.

FIG6A shows geometrically what can be seen in the image, while FIG6B shows the functional relationship between L_HDR brightness and L_SDR brightness. The image shows a robot 602 moving through its dark space station (DRKSPST). At a certain image rendering time, it encounters a shadow man 601 (chromatically defined as a set of very bright HDR pixels), where there are some slight brightness differences between the various pixels that make up the shadow man's body. This occurs because he is standing behind a window in a strongly lit environment with a foggy atmosphere. The fog adds a component to the luminance originating from the shadow person's body (e.g., his clothes), giving the viewer a final luminance in the HDR image of, for example, L_pants = 20 nits + L_mist = 4500 nits = 4520 nits, L_shirt = 50 nits + L_mist = 4800 nits = 4850 nits, etc. The problem when using a progressive luminance mapping function with a slope that is too small for the brightest pixel is that the shadow person can become lacking in contrast and difficult to see in smaller dynamic range images (e.g., SDR images). One solution is to define the F_L_CU function so that it has a steeper slope locally in the input HDR luminance region 4500 to 5000 nits, resulting in a larger SDR luminance sub-range RS for the shadowed man, making him and his details (e.g. the tie he is wearing) more visible in the fog (even in the SDR image). It can be appreciated that there may be many situations where having a bit more additional regrading control than just the coarse mapping function F_C may be beneficial.

Returning to Figure 4, having defined the appropriate (uniform visual representation) SDR brightness, the linearizer 406 converts it to a (normalized) SDR brightness Ls. However, since the SDR brightness is now generated with an RHO corresponding to PB_C_S = 100 nits (which is input to unit 406) rather than the 5000 nits used for perceptual non-uniformity at the beginning of the brightness processing chain, the inverse of equation 1 above applies.

Color is of course not 1-dimensional (unless operating only with achromatic grey value images), which makes dynamic range conversion and encoding considerably more complex, but in any case it requires parallel processing tracks for the chrominance Cb and Cr of the pixel to obtain more appropriately corresponding SDR chrominance as the output color components Rs, Gs, and Bs, or indeed proper SDR RGB colors as ultimately shown in Figure 4.

The chroma processing track 450 of ETSI2 performs the following (explained again briefly only to the extent necessary). The input pixel chroma Cb and Cr are similarly multiplied by the value F_C[Y] by multiplier 452, producing the output chroma Cb*, Cr*. The difficulty is to always get the appropriate output chroma, and there are many difficulties: the irregular shape of the gamut of achievable colors (see explanation in Figure 5), the non-linearity of the mathematics, and the human visual system of the viewer. In addition, as will be shown in the embodiments of the present application below, the market has even more requirements, resulting in even more complex HDR processing systems.

ETSI2 uses a saturation processing determinator 451 which can load a LUT defining the output value to be sent to the multiplier depending on what brightness value Y the input pixel happens to have. Again, the content creator is free to define/optimize the shape of this brightness-dependent saturation multiplier definition function. Not least because, as will be seen below, sometimes inventing color mathematics requires defining this F_C[Y] LUT to the desired degree.

The matrix application unit 453 simply converts the Cb, Cr color specifications into corresponding normalized RGB representations (this mathematics is uninteresting for this application and can be found in ETSI2 and ETSI1 for interested readers).

The RGB triplet can be defined by multiplying the normalized R/Lh etc. values of "non-HDR brilliance" by the normalized Ls value calculated in the brilliance processing track 401. Note that the resulting RN, GN, and BN values are in fact still normalized brilliance, not absolute SDR brilliance (Rs etc.), but they are "SDR correct" normalized brilliance because they now take into account the brilliance that the SDR happens to get (Ls).

There is the possibility that this may be initially somewhat difficult to conceptualize for someone who is not generally knowledgeable in colorimetric techniques, so to speed up the reader it is explained what happens in the normalized (general, i.e., when normalized as explained above, the SDR and HDR color domains overlap well, but of course the HDR colors must be shifted to become proper SDR colors, even if the transformation is not highly sophisticated and optimized for the needs of the present HDR scene image, but is simply to equate the absolute SDR brightness to the input HDR absolute brightness) YCbCr color domain in FIG. 5 .

Pure brilliance transformations will occur in the vertical direction, so generally HDR brilliance or its corresponding brightness Y (i.e., brightness for ColHDR) will be moved upwards to an optimal new position (ColSDR), because for the HDR to SDR brilliance mapping, the F_L curve will always fall above the diagonal line on the normalized axis graph (i.e., an input HDR normalized brilliance or brightness with a certain x-coordinate also has a y-coordinate that is the height of the diagonal line at the position of the x-coordinate, and the normalized SDR output brilliance of the function that is always above the diagonal line will therefore always produce higher normalized output values). Which actual (absolute) SDR brightness corresponds to this normalized brightness value Y is found by first EOTFing the normalized brightness (performed by unit 406, since the processed brightness from Y'HP to Y'GL is defined by applying the corresponding EOTF of equation 1), and then simply multiplying those normalized brightness by 100 (e.g., 0.7*100= 70 nits) by multiplier 455. That is, the reader now sees that using this framework, anything required can be defined from the input HDR image color, and in particular from its PQ-defined luminance Y (e.g., as stored on an HDR Blu-ray disc), all the way to the absolute SDR brightness of the corresponding pixels to be displayed on an SDR display, in order to display the best corresponding SDR image to the HDR input image (and the resulting decode of the SDR image from the received HDR image).

By now the reader knows the basic starting point of HDR coding at least according to the applicant's ETSI standardized coding principles. For most consumers, the choice of either ETSI1 or ETSI2 (and everything that technically follows) will be sufficient for their purposes, namely to supply their market with beautiful HDR images (of course they will still need to produce such beautiful HDR images, including determining good shapes for at least the F_C function and preferably also for the F_L_CU function, or at least buy and use the applicant's automation that automatically produces quite good looking and subsequent codec function shapes for each HDR image category when optimizing these functions non-manually according to their own specific artistic needs). For example, a consumer who is going to do a full patching for future-proof high quality diverse HDR may deploy an ETSI2 system, and a market player that values their SDR images or one of the SDR consumers may instead deploy their HDR system as an ETSI1 system (this may also involve various discussions depending on where in the HDR video processing chain, e.g. content creators versus cable operators, and possibly involving transcoding, etc.).

However, there is another market need or there is a market proposal for consumers who do not like the deployment of an exact standardized ETSI1 or ETSI2. If the choice is to convey the HDR image as a single image representing the entire spectrum of the image needed for all kinds of PB_D displays, it is very reasonable to convey the (for example, 5000 nits PB_C) master HDR image itself, not only because these images are already available, but also because they are the best quality representation of the image HDR scene (in fact they are the "wealth" of the content creator, the images specifically created and approved by him, and often the starting point of a creative visual film, if it has not been the only thing he has created actively, if the rest of the re-grading works automatically by the chosen technology). However, especially in the coming years, there are market situations that can benefit from another additional approach. Unfortunately not all TVs (or video decoding or processing devices in general) on the market that are not basic conventional SDR displays (i.e. cannot do all the math involved in HDR decoding, display adaptation, etc.) will always be ETSI2 (or ETSI1) capable TVs out of the box. There are many TVs on the market that apply very different approaches to HDR encoding and display (like, for example, according to the recently standardized Hybrid Log-Gamma method). Or maybe some TVs can just decode PQ brightness encoded HDR images and nothing more. Maybe some TVs can just use that method, so it's likely that the best they can do is not process incoming ETSI2 HDR video at all. Similarly, there may be some TVs in the market that do not follow any standard principles, at least not with regard to display adaptation, i.e. re-grading, upon reception, e.g. a 2000 nit image into, e.g., a 900 nit image for a 900 nit PB_D display. Such a TV will need decoding capabilities to understand which pixel colors and especially the brightness the received image contains, but they can use their own (tone mapping) inspiration on how to produce the 900 nit image. The disadvantage, at least from the point of view of the content creator who wants every consumer to see a movie that is as good as it was when he originally created it, is that this variability will create a high degree of uncertainty over which any particular brand of TV will produce any received HDR image. For example, a simple display re-interpretation of an HDR image that has been performed recently is an absolute representation of the brightness of the HDR image. This means that all HDR image luminances up to 900 nits are displayed exactly as encoded in the image, but all higher luminances are clipped to the display's whitest possible white (i.e., PB_D). Using an example image, such as the space station in Figure 7, this may mean that some parts of the Earth will be clipped to ugly white spots (where the Sun over-illuminates the Earth to its right). While this TV will still be a somewhat beautiful HDR TV, since it will show a mostly very bright blue Earth seen through the top viewing port of the space station contrasting well with the dark interior, at least part of the image will look ugly (and some other scenes may show more severe errors on at least some TVs, such as, for example, clipping off every image detail outside of the cave or marketplace in Figure 1). Performing another simplified tone mapping reinterpretation (such as, for example, linear compression of brightness, like a white-on-white strategy) may create several other problems. So while such a system may work and produce some kind of HDR image for the end viewer (e.g., in our ETSI2 system, such a TV may only use the PQ function of 401, but ignore all other brightness mapping function metadata and the resulting sequential brightness mapping 402, 403, etc., which perform display adaptation functions in ETSI2), the result will not be the best visual quality nor predictable (it may be worse).

This leads to a new coding topology based on a second type of HDR image, in addition to the main HDR image, the so called intermediate dynamic range (IDR) images, first introduced in WO2016020189. The advantage is then that such secondary HDR images (IDR coded images, which will be delivered to the receiver instead of the main HDR image in the typical ETSI2 codec principles) can be defined at PB_C in the range of many TVs in the art (e.g. 1000 nits, or 750 nits; although one could also choose 500 nits, or possibly even 400 nits PB_IDR using the same technology). But in terms of desired artistic or practical technical limitations (e.g. available graded monitors), it is nevertheless desirable to produce a PB_MHDR main HDR. The idea is that whatever display re-interpretation (including tone mapping) technique any TV uses, it should be smooth, in the sense that if PB_D is close to PB_IDR (the peak brightness of the received IDR image), the processing should not deviate too much from the received image. For example, even a basic TV that just clips all pixel brightness above PB_D should not clip too much (e.g. not the clear outer side of an image of the entire Earth or a cave). And the content creator gets back some control, because even if he on the one hand wishes to produce beautiful super bright image areas, e.g. around an average of 4000 nits in a main image of a 5000 nit PB_C_H, he can control the way he regrades these areas in the IDR image so that they drop to, e.g., well below 1000 nits, so that even a basic TV of 800 nits should clip only the brightest and least visually disruptive pixels, e.g. clip only the light of the sun in the space station example of Figure 7. So some new technology is needed to cater to this new approach.

Figure 7 shows the codec principle of WO2016020189 catering for the channel adaptation approach (the channel conveyed image is an IDR image, whereby one can say that a particular communication channel is configured for sending, for example, a 1000 nit PB_CH image). Again an interesting example is chosen to illustrate some of the main concepts. One thing that should be understood is that while it can be useful if all the different PB_C images along the range are exactly or at least very close to what would be produced if the content creator graded each of them separately and without any technical system constraints, this requirement is not necessarily always the case, especially for IDR images. There may be some relaxation involved (on the other hand, there may also be some debate about when or why a certain image classification X for Y of the HDR scene class is best, and there seems to be enough bias for that bias; for example, one could imagine that the brightness of a pixel of a street light is less important than that of a face, especially if it should be seen as semi-hidden in darkness, but also because in real life any street light is likely to be brighter or less bright anyway).

WO2016020189 provides a way to define a function (different functions) from an IDR image as some midpoint (i.e., upwards towards the master HDR to be reconstructed from the IDR image as received by the receiver, and downwards for display adaptation for MDR displays with PB_D < PB_IDR). Using this technique, the master HDR range can be well chosen to always be fixed to the 10000 nit PB_C range, which is linked to the range of the PQ function.

See that again there may be different considerations involved in how to transform the various possible luminances, and these may advantageously be quite different on the left than on the right for the chosen IDR image. Because in fact, conceptually different things may be being done. On the left, a secondary ("smaller") HDR image is generated from the main HDR image. So one consideration may be that this IDR image must be "as good" as the main HDR image (despite the lower PB_IDR) (and then how to resolve this seeming contradiction elegantly?). On the right, one is heading towards even smaller PB_MDR compression (which is appreciable for some high complexity, meaning many key objects across the luminance range, and high PB_C_H images among other things), i.e., there seems to be a different displayed adapted image generation task. So, one can imagine that this could lead to a (quite) different technical treatment, in particular in that our image+brightness mapping function is shaped/designed differently in vision.

In this example, the dim space station brightness can (at least in principle) be displayed on every reasonable TV, since it is dimmer than 60 nits. But the brighter pixels must first be compressed quite gently to the IDR image, then less compression is done in the first part, more must be done towards the SDR image. And there may again be different criteria for the exemplary two bright objects, the bright blue Earth versus the much brighter but almost colorless Sun and its light. As the luminance sub-ranges on the main HDR image luminance range (BE) and the IDR luminance range for bright Earth objects (Be2) respectively indicate, ideally the content creator can expect the maximum brightness for the Earth to never be higher than, for example, 750 nits, regardless of the PB_C capabilities of any image or display (because otherwise the Earth might start to shine too brightly and look unrealistic). However, what must be done with the solar brilliance then becomes a function of several factors, not only artistic requirements but also the amount of brilliance left to encode solar objects above 750 nits in the chosen (800 nits PB_IDR) IDR image (of course in some cases the content provider could choose another higher PB_IDR value, but it is assumed here that any device connected to the receiving end of the communication channel always expects a PB_IDR of 800 nits for any video content, be it a Hollywood movie or a news show). The final chosen F_H2h luminance mapping function for building the IDR image luminance from the main HDR image luminance is shown with two arrows for all these brightest pixels as a subset: the solution was chosen to define an overall compression action for both objects together, which also reduces the luminance of some of the least bright Earth objects. This is an example of a situation where the ideal regrading needs of the content creator are not 100% fully met (because probably corresponds to some other technical difficulties), while the IDR image is close enough for most people. It is really not that important if the Earth pixels are just a little darker in the IDR image, and it might even be somewhat desirable for lower quality HDR images. But the point is that this IDR image still meets all the requirements of the original ETSI2 principles (while this additional codec step also meets the requirement of basic monitors close to 800 nits PB_D not to degrade the received IDR image too much before displaying it): by adopting the right luminance transform function, all MDR images up to the SDR master image can still be produced by the receiver as the content creator intended, and (even with darkened bright earth object pixels) the master HDR 2000 nits PB_C or 10,000 nits PB_C image can still be reconstructed by the inverse F_H2h function (which can also be optimized by itself for each image, a set of consecutive images of a specific shot of the encoded movie, according to its technical and/or artistic needs).

Two documents worth discussing regarding their irrelevance (because different technical aspects should not be confused) rather than their importance (but worth discussing due to potential confusion) are US20160307602 and EP2689392 (also known as WO2012127401), both of which consider so-called "display optimization" rather than video image coding framework design. The main difference is illustrated to those of ordinary skill using Figure 23, which shows a general example overall video processing chain. On the content creation side 2301, it is assumed that there is a live (or previously recorded) capture of an HDR scene by means of a camera 2302. The human grader (or colorist) determines, e.g., the main grade of the capture among other things (i.e., the relative position of the brightness of the various image object pixels on the brightness dynamic range of the main HDR image - which ends at a maximum representable value of e.g. PB_C_H50=5000 nits; and starts at some small black value, e.g. MB_C_H50=0.001 nit, which can be assumed equal to zero for the present discussion: e.g., for the space station he alters the original camera capture by image processing so that the sun pixels become 4500 nits, the bright blue of the Earth become, e.g., 200 nits, etc.). Secondly, the grader in our method generally wants to indicate with at least one luminance mapping function (in practice, such a luminance mapping function may be shaped differently for successive images of HDR video, and even in our ETSI standard it is explained how it may be technically quite convenient to define several functions even for a single instantaneous video image, but such further complexity is not necessary to illustrate the present innovative contribution to the art) which generally will be the function that specifies how the normalized main HDR luminance of 5000 nits PB_C_H50 must be regraded to a 100 nits LDR normalized luminance: FL_50t1.

The third important aspect is then the coding technique used for encoding of the main HDR image to be communicated (via at least one coding technique) to one or more receivers. At the beginning of HDR video research, and correspondingly in the simpler versions standardized by the applicants, this would be a relatively simple encoding of, say, an LDR 100 nit image, which would then be well retroactively compatible so that it could be displayed directly with good visual appearance on older LDR TVs that have no HDR understanding or processing capabilities. The WO2016020189 coding method and the present teachings are examples of more advanced second generation methods, which are more complex but can cater for additional needs in some specific HDR video communication or processing technology ecosystems. Grading, performed by for example a human grader 2304 (if this is not automated, e.g., as is common in real-life broadcast programmes), is done on a grading device 2303 (which will generally contain several tools to change pixel brightness, but for the present description can still be assumed to consist of components providing a user interface to specify the FL_50t1 shape, and communicating this function shape (e.g. as metadata containing some parameters defining the shape of the function)).

Although the video encoder 2305 (assuming, without limitation, that its input master HDR image is a set of luminances for all pixels, which will perform all techniques for the actual encoding of the master HDR image, i.e., for example, an 8-bit, 10-bit, or 12-bit pixelation matrix of generic YCbCr pixel color triplets together with metadata describing all further information (such as luminance mapping functions) in view of whichever encoding technique is chosen) may in principle be included in the grading device 2303, it is generally shown as a further connected device. This representation is sufficient to illustrate the simplification of the invention for the reader, which summarizes the various actual variations that occur in the outer broadcast track, such as capture (and possible classification), and perhaps encoding that occurs in some intermediate communication relays (for example, in inserting local advertisements in the signal, etc., which may also involve respective adjustments of various image contents, but which are details that do not require explanation). The key point is to understand what happens on the creation side (see, for example, the difference between contribution and distribution), which can be defined as the official end when the final encoded video signal is transmitted to some consumer, for example, by means of satellite antenna 2306 and communication satellite 2340 (or any equivalent video communication channel, for example, via the Internet, etc.).

On the receiving side, it is generally faced with consumer equipment in the end consumer's home, such as a satellite TV set connected to a local satellite dish 2351 on the input side and to an HDR display 2353 on the output side. box, or any equivalent decoding and final processing device 2352, the display may have various display capabilities, for example, 1000 nits, or 700 nits, or 2500 nits PB_D. While it may be sufficient for the set-top box to only decode the brightness values that need to be displayed again by means of the decoder 2381, which often performs the inverse operation of the encoder, this will generally only be useful in a limited number of cases. . Typically there will be a display optimization process by a display optimizer 2382 which again modifies the absolutely individually normalized brightness distribution (the received, e.g., LDR image or decoded master HDR, For example, any of the 5000 nit images), since the main image may have been encoded for, for example, 5000 nits PB_C_H50, i.e., may contain 2000 nit brightness pixels, a particular consumer's HDR display may still only, for example, Displays up to 700 nits (which can display the whitest white).

So, on one hand, there is a major technical difference between the apparatus on one side (and their technical design principles etc.), e.g. the authoring/encoding/transmitting side will only have a video encoder 2370 to encode the master HDR video (MsterHDR) into some channel coded intermediate dynamic range image IDR, while the receiving side can also display an optimized reconstructed 5000 nit HDR image (RecHDR) in, e.g., a 700 nit PB_C image ImDA optimized for a display, optimally suited for a connected 700 nit PB_D display. The technical difference between the two can be seen in that one can perform display optimization as an (optional) post-processing, while the encoding/decoding is only an image reconstruction technique, generally not requiring any teachings related to display optimization. The equipment (and operating procedures, etc.) on both sides are generally handled by quite different technical personnel. Content creation equipment may be designed by professional video equipment manufacturers and operated by broadcast engineers, etc. Set-top boxes and TVs are generally manufactured by Asian consumer electronics manufacturers.

US20160307602 is the applicant's first display optimization patent. In summary, the idea here is that content creators can come up with rules and algorithms for guided re-rating behavior for various (at least two) regions that may exist in an image (region is a concept that is both a set of pixels in an image, and the re-rating behavior required for those pixels when there are various available displays with various dynamic ranges). While this first achieves the connection between the needs of the content creator and the actual display at the final consumption location, the actual controlled behavior of the display adaptation must occur at this end location. And ideally, the manufacturer of a set-top box or TV (if at least display adaptation occurs in that TV) would mostly follow what the content creator specifies as a good bet for various area objects in the video image (e.g. someone fading in from a dark area to be neither too visible nor too invisible on any display capability, even a 100 nit PB_D LDR display), because it is required by the content, rather than blindly doing anything himself. But this is clearly the ultimate behavior that occurs on the consumer side, and is completely orthogonal to any specific video codec principles of how the video communications technology vendors want to develop, and how any implementation creator wants to deploy respectively. Nor should it be confused with any ad hoc tone mapping techniques, which are targeted at the fact that such a mapping will generally be irreversible, should it have the properties of being encoded via lower dynamic range IDR images.

WO2012127401 is also an early HDR era technology for specifying display optimization behavior, which can be accomplished by various implementations of a DATGRAD structure that specifies how various image content should be regraded for different luminance dynamic range capabilities. This DATGRAD structure will be used to generate any required medium dynamic range image (MDR) for an MDR display PB_D between the main HDR encodeable peak luminance PB_C (i.e., PB_C_H50 in this notation) and the 100 nits LDR PB_C of the minimum necessary regrading specification (p. 16). The derivation of the MDR image is best done by using not only the image regrading requirements as encoded in the DATGRAD data structure, but also specific display-side viewing conditions such as, for example, the viewing environment brightness or the final viewer preference settings (see p. 5).

It should be clear from the outset that such teachings do not bring the person of ordinary skill anything regarding codec redesign without further quite specific insights.

Besides the difference in generating specific functions, more importantly than what can already be found in the prior art, the innovative codec structure/framework itself, it should also be mentioned that the actual communication of the second PB_C value (the highest one of the main content, in addition to the lower one of the actually communicated IDR image) is also different from the (optional) kind of characterizer that can be used in WO2016020189. Apart from the fact that the two are literally not the same, the enumerator can play a different role, and especially if the details of the framework compared to the present teaching are considered. Such a characterizer of '189 can be useful in the case where there are, for example, two upward re-scaled brightness mapping functions. It may then be useful to choose which one to use in order to obtain anything like a close reconstruction of the main HDR image on the authoring side. But such information is neither strictly needed nor necessary in the prior art. Instead of a 5000 nit reconstructed image, an upscaling function from the main HDR image can be used to obtain a 4000 or 6000 nit reconstructed image. There are two sides of the intermediate image, the downscaling function is usually one of critical image content (especially content that must be bright enough to be displayed faithfully on all PB_D displays), but the upscaling function will specifically differ in the way it specifies the regrading behavior for the very brightest objects (like headlights, sunlight reflections, etc.). These are general HDR impact purposes/effects, which however cannot be reproduced correctly in any way, because they are in the upper area of the brightness range that changes the most on displays of various capabilities. Thus, a 4000 nit PB_C reconstructed image produced from, for example, a 600 nit IDR image may have some headlights that are slightly too dim compared to their ideal brightness value (even though such a value is representable on the 4000 nit brightness range), but it will still be a fairly good looking HDR image if one simply applies, for example, a multi-linear normalized rescaling function on a [0-1]/[0-1] axis system, where the horizontal axis represents the PB_C normalized brightness of the IDR image and the vertical axis corresponds to whatever was chosen to compute the reconstructed HDR image PB_C that is not unreasonably far from the master HDR PB_C (which may not be known but only assumed). In our technique, the PB_C_H50 brightness value itself is actively communicated in the metadata, since it is also used in the decoder's algorithm.

The inventors of this patent application want to limit the generic IDR approach in many ways, especially around the ETSI2 coding principles and systems (ICs, TVs, set-top boxes) that are already deployed today.

Many technical considerations were generated by the inventors. On the one hand, they wanted their system to be compatible with deployed ETSI2 decoders. Therefore, if an IDR image of, for example, 1500 nits is communicated (the ETSI2 decoder does not know anything about the principles of the IDR architecture, so it assumes that this is just an original HDR master image of the HDR scene), a F_L_IDR brightness mapping function (and all other color mapping information according to ETSI2) which is the F_I2s function of Figure 7 that does the correct display adaptation should be communicated together. Therefore, an ETSI2 (also called SLHDR2) decoder should be able to properly build all MDR images up to SDR images, regardless of whether an IDR additional technique was used, and they should (ideally) look as the content creator intended. Ideally, any new decoder according to the present principles (which will be called a SLHDR2PLUS decoder ) should also produce exactly or at least approximately the same look for all images between IDR and SDR (i.e. at least one of the IDR and SDR images should preferably not deviate too much from the MDR image which would result in how the color grader or generally content creator would want or at least accept to see it). On the other hand, a very important criterion is that the main HDR can be reconstructed almost perfectly (but there may be some minor trimming errors that are introduced unnoticed, e.g. when DCTing IDR images in the MPEG compression phase for transmission which would result in very minor unobtrusive reconstruction errors of the main HDR on the receiving side). Of course, there may be some systems that have some relaxation on the reconstruction quality of the master HDR (some content providers may consider IDR images to be more important, at least in situations involving some temporary aspect, like broadcast or even single investment to a limited audience, rather than, for example, for storage on e.g. Blu-ray disc), but usually at least one key party involved in the video processing chain will find perfect reconstruction of the master HDR image important (as distinguished from blind attempts to create some higher dynamic range appearance starting from IDR).

Finally, although it is seen that to serve without the need to redesign and redeploy the ETSI2 decoder, the F_I2s functions must be communicated together (i.e., preferably re-using the encoding (de)coding circuitry of the SLHDR2 system as much as possible, but at least the video signals including their brightness and color mapping functions should still comply with the standardized definition so that, in particular, traditional SLHDR2 systems know what they are getting, except for some metadata that they do not need and can be ignored), the content grader may generally want to specify his brightness (and color) mapping functions between a master HDR that he has created and some corresponding SDR (i.e., 100 nits PB_C) version of it (which he may have created, for example, using one of the systems shown in Figure 4). The F_Mt1 function (see FIG. 10 ) is neither the F_H2h nor the F_I2s function of FIG. 7 , but rather a function that is global in the regrading effect between the primary HDR and primary SDR (i.e., this F_Mt1 defines the regrading requirements of an HDR scene image between the most different dynamic range representations of the HDR scene). Therefore, a technique is needed to elegantly relate these two cases, particularly in or around the ETSI2 framework principles (e.g., the SLHDR2PLUS decoding method produces the same MDR image look as an ETSI2 receiver that displays adapted to the received IDR image and the D_I2s function; for each instant in time, one or more functions partially perform the regrading between the input dynamic range image at that instant and the desired output dynamic range image at that instant).

As will be seen below, this can be done in several ways according to the different insights of the various inventors, depending on exactly which kind of system is desired, and which desired constraints are relaxed more and which are relaxed less (also taking into account such specific practical technical factors as, for example, how many loops or transistors will be needed for various choices, which may make some choices more desirable than others, but it is not necessary to delve into those details for the purpose of this patent application).

There are some basic fundamental principles that all methods use. At least two solutions can be summarized with Figure 8. The receiver receives IDR images (which should be reconstructed somehow into the Master HDR image, in this example with PB_C = 4000 nits), and they also have the function F_I2s. But they must somehow find the function F_?? for each IDR brightness to calculate the required corresponding normalized and therefore absolute Master HDR brightness (which reconstructs the Master HDR exactly as it was, still by no means conveying its image(s)). One could construct a novel SLHDR2PLUS decoder color conversion system that can determine the required functionality (it still has a brightness processing track and its details (using at least some of the sub-units), and a chrominance processing track, based at least on its processing IC or software core as shown in FIG. 4 ), or one could try to put all the intelligence into the encoder so that the standard ETSI2 encoded color conversion method can be used as is (except for its novelty of being newly planned to reconstruct the original HDR of 4000 nits by receiving meta-information of this second desired peak brightness PB_C_H50, which generally involves the loading of appropriate full or partial brightness and chrominance processing LUTs), setting it to interpolate rather than display adapted images for PB_D smaller than the PB_IDR value. Both methods and their implementations will require some common new technical components, although falling under the common SLHDR2PLUS new coding principles.

From the basic architecture of the SLHDR2PLUS codec 900 kind of the generic IDR codec shown in Figure 9 (which will be explained in more detail below), one sees the difference to normal HDR coding, and in particular ETSI2 HDR video coding: there are now two peak luminances encoded together in the metadata, namely first the "normal" peak luminance (which shall be called channel peak luminance PB_CH, i.e. the peak luminance of the received IDR image, using whatever technique is used for it, i.e. whatever peak luminance level looks best to the content creator, owner, or transcoder, and whatever mathematical technique is used to calculate IDR pixel luminance), which indicates the maximum encodable luminance of the transmitted and later received video (i.e. the IDR image(s)) [this is what a normal ETSI2 decoder will see, ignoring all other novel methods]. Secondly, but now also the peak brightness of the original master HDR image(s), which is the content peak brightness PB_C_H50 (e.g., 5000 nits). In some embodiments, the PB_C_H50 of the second party may have been specified when the master HDR image is created (e.g., based on camera capture action, in the computer, etc.), many months before the IDR image is created, and the PB_CH may be set as an external input to the decoder 900 during channel encoding in a number of different possible ways (e.g., a cable TV operator may have a fixed value set in memory that may be upgraded on an annual basis to reflect the current average state of his consumers' HDR displays, or once optimized the PB_CH may be calculated taking into account certain brightness or other image details of at least one image of the video, or its associated metadata, and may even specifically include metadata used to guide subsequent IDR re-encoding). Having a (single) peak brightness communicated together is useful for HDR coding, at least for the systems of ETSI2 (which at the time had been considered to be the only thing they needed, namely "the" peak brightness of the received image), but given the full transparency availability of conventional ETSI2 displays, they would have to be PB_CH as described (otherwise they could not do their normal display adaptation calculations). On the other hand, PB_C_H50 must be fully able to calculate the F_?? function of Figure 8, and using this function, the final desired main HDR reconstructed image comes from the received IDR image.

Thus, the difference between a conventional ETSI2 video coded data stream is immediately apparent and conventional ETSI2 decoders will be unaware of this additional metadata and simply ignore it since ETSI2 decoders do not have to judge any image using a higher PB_C than the PB_C_H they receive in the metadata indicating the brightest possible brightness in the image they receive (since according to a pure ETSI2 principle the image received is always the best quality image and in fact it is the highest quality master HDR image created by the content creator). But as shown in Figure 11, a generic SLHDR2PLUS decoder will not only receive and read the PB_C_H50 value, but also use it to reconstruct the REC_M_HDR image, which is a near-perfect reconstruction of the master HDR image created by the content creator (in fact, such a decoder will use the PB_C_H50 value to calculate the required F_?? function(s) from the received F_I2sCI function(s)). This decoder can also advantageously output lower PB_C images, such as, for example, 400 nits PB_C MDR_300 images, but a standard ETSI2 compute core may be chosen for such images with a PB_C lower than PB_CH, or the calculation may be performed in an embodiment of the new SLHDR2PLUS compute core (but new insights are certainly needed for accurate reconstruction of images with higher PB_C than PB_CH, as this cannot be done in detail using ETSI2 techniques).

Therefore, the tasks set to be solved by the new technology are implemented by a high dynamic range video encoder (900), which is configured to receive an input high dynamic range image (MsterHDR) via an image input (920), the input high dynamic range image having a first maximum pixel brightness (PB_C_H50), the encoder having a first meta-data input (922) for the first maximum pixel brightness, and the high dynamic range video encoder is configured to receive a main brightness via a second meta-data input (921). A brightness mapping function (FL_50t1) defining the relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR), the low dynamic range image preferably having an LDR maximum pixel brightness equal to 100 nits, characterized in that the encoder further comprises a third meta-data input (923) for receiving a second maximum pixel brightness (PB_CH), and the encoder is further characterized in that it comprises: -                    an HDR function generation unit (901) configured to apply a normalized algorithm to transform the master brightness mapping function (FL_50t1) into an adapted brightness mapping function (F_H2hCI), the adapted brightness mapping function relating the normalized brightness of the input high dynamic range image to the normalized brightness of an intermediate dynamic range image (IDR), the intermediate dynamic range image being characterized by having a maximum possible brightness equal to the second maximum pixel brightness (PB_CH); -                    an IDR image calculation unit (902) configured to apply the adapted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR) which is the output of this unit; and an IDR mapping function generator (903) configured to derive a channel brightness mapping function (F_I2sCI) based on the master brightness mapping function (FL_50t1) and the adapted brightness mapping function (F_H2hCI), the channel brightness mapping function defining as output the respective normalized brightnesses of the low dynamic range image (Im_LDR) when the respective normalized brightnesses of the intermediate dynamic range image (IDR) are given as input, the respective brightnesses in turn corresponding to the respective brightnesses of the input high dynamic range image (MsterHDR); the encoder is further characterized by having: -                  an image output (930) to output the intermediate dynamic range image (IDR); -                    a first meta-data output (932) to output the second maximum pixel brightness (PB_CH); -                  a second meta-data output (931) to output the channel brightness mapping function (F_I2sCI); and -                  a third meta-data output (933) to output the first maximum pixel brightness (PB_C_H50).

First note that although conceptually a separate input is shown for each required data item for the present encoder, in practice the reader of ordinary skill understands that one or more of these inputs (and similarly for the outputs) may be the same, depending on what a video input technology can handle (e.g., some older HDMI video inputs cannot handle the dynamically changing - i.e., master brightness mapping function (FL_50t1) which may be different for each temporally consecutive video image, in which case the data may be communicated in a synchronized manner via, for example, a Wi-Fi connection). How the various input data are input may also depend on where they originate, i.e., which other system the encoder is connected to or into (which may depend on whether it is desired to encode in real time at the same time as the camera captures the event, or to encode later for some video communications system, such as, for example, a cable television distribution system which receives all the data from the original content creator at any later time in order to best distribute it given the limitations or requirements of this particular cable television system, etc.).

Without intending to limit, it may be assumed that the MasterHDR image has previously been graded by a human color grader using color grading software on a computer, and that he has defined the FL_50t1 function which defines a corresponding lower dynamic range image, typically a 100 nit SDR image (although the current lowest end of the regraded image spectrum is a 100 nit PB_C image according to standard protocol, so this does not appear likely to change, and the lowest of the three, i.e. the LDR image, may in future embodiments have an LDR maximum luminance that is not exactly 100 nits, but perhaps 1 00 nits, for example preferably k is at most 3x, i.e. the maximum brightness of the LDR in the embodiment of the present system is 300 nits), corresponding to the MasterHDR image (taking into account the considerable lower brightness dynamic range, the SDR image preferably looks as similar to the MasterHDR image as possible), which generally conveys the desired look for best visually telling (e.g., a movie story) as needed (different video applications may also have different requirements, such as different color criteria, which may involve different technical limitations on the FL_50t1 function).

The PB_CH value is somewhat different from other metadata in that it is in fact a setting for the intermediate dynamic range encoding. Therefore, it may or may not come from the grader. It may be, for example, a fixed value for a specific video coding system (such as, for example, a satellite broadcasting system), which may be extracted from, for example, a fixed memory attached to or in the encoder. In Internet-based delivery, this PB_CH value may be communicated on demand to an end consumer for whom the IDR image is produced. For example, a consumer with a poor quality mobile display may only request a 500 nit PB_IDR image to be calculated by a server on the other side of the Internet, such as a video on demand company's server, while some other consumer may request a 1000 nit PB_IDR version, and in this case the requested PB_CH=PB_IDR would be input into the encoder.

So, while on the encoding side there is a highest quality (in fact, highest PB_C) MsterHDR image, this is not the image the receiver (complementary decoder) will receive, but the IDR image (and they will need to closely reconstruct the MsterHDR image by computing the REC_M_HDR image). These techniques are best implemented by formulating everything to be normalized in 0.0 to 1.0 luminance. In fact, when we talk about a brightness mapping function, this is actually also equivalent to a brightness mapping function (because of the relationship between brightness and their corresponding brightness (e.g., the brightness generally to be displayed)), but technically speaking, our calculations work better using a brightness mapping function, and are preferably defined in terms of visually psychologically uniform brightness, as calculated by the Philips v function (see Equation 1 and Equation 2).

As explained above, our approach to processing HDR video (specifically encoding not only a single, or two different re-graded images in different dynamic ranges of a particular peak brightness, but also encoding the entire spectrum of the corresponding different DR re-grades) is to correlate various possible normalized brightnesses that a pixel of at least two correlated images may have, for example, 0.2 in image_1 corresponding to 0.4 in image_2, etc. This is defined by the brightness mapping function, between one case (i.e., one re-grade) and any other chosen different case.

Using a standardized algorithm means that there must be some fixed way to relate a first set of possible functions (which can have many different shapes and definitions) to a second set of corresponding functions. That is, it simply means that in some communications technologies (or even all technologies), the designers of the encoder and the decoder have defined a method that uniquely specifies how to transform the shape of any input function (usually on the axis normalized to 1.0) to the shape of the output function. There may be a variety of such algorithms, so in principle the codec designer can decide the order number of any such algorithm he may want to communicate to the decoder etc - for example, agreeing on algorithm number 3, but normally there is no need to be so complicated as our method will work perfectly and most simply by pre-agreeing on a fixed normalized function transformation algorithm, such as the one in the supporting mathematics below.

To give the reader a quick overview, here is a simple example of this algorithm. Assuming the input function is the power function: power(x_in; P), the algorithm can derive the corresponding function power(x_in; P-1). By inversion, when receiving the corresponding functions (via the +1 algorithm), the original functions can also be re-derived again.

It should not be misunderstood that the standardized algorithm itself is generally not communicated to the receiver, only the resulting output correspondence function. This is why it is standardized, or pre-agreed, i.e. must be fixed, so that the decoder can know what has happened on the encoding side. The way this is agreed upon is not so relevant to understanding the patented technology. For example, there may be 5 different fixed algorithms, and a cable TV operator may decide to encode with algorithm 3, and supply his consumer set-top boxes with corresponding settings to decode fixed algorithm 3 (even though the STB may in certain cases be reset to, for example, algorithm 4 for some other video communication; but algorithm changes will generally not be necessary, although changes on the PB_CH used for different cable TV channels, for example, may be of interest).

Careful attention should also be paid to the fact that the corresponding adapted brightness mapping function F_H2hCI is not generally communicated to the receivers, but then another channel brightness mapping function (F_I2sCI) is communicated which can be further derived, and the decoder also needs to invert this double derivation in some way. In fact, the total re-rating map is split into two parts, so if the first part is standardized, the second part can also be defined, so the inversion of this IDR encoding by the decoder can be considered to be probably possible (although difficult) (making the architecture and correct functioning of the new SLHDR2PLUS codec possible).

This concept of a normalized encodable peak brightness dependent function change algorithm has been further illustrated with FIG . 24 . Thus on the left one sees the occurrence of various FL_50t1 functions designed automatically by the grader or content author side regrading function decision. For example, FL_50t1_1 may have been decided for an HDR scene with significant action taking place in it with fairly deep blacks, which must be noticeably brightened to still be visible enough on a low dynamic range display, but where the brightest parts are not so critical - like e.g. street lights - and can be represented by a single nearly maximum white luminance on any display (or so the image is calculated, i.e. containing exactly those absolute luminances as it is waiting to be rendered on the display, or more precisely, the luminance codes that typically encode those luminances). In contrast, FL_50t1_2 is established for a shot of an image or succession of images containing another type of HDR scene, where there is both a significant lower brightness object (or more precisely, area) and a higher brightness object, which has resulted in specially tuned regrading curve shapes on either side of the "middle". FL_50t1_3 is yet another possible input function applied to the normalization algorithm by the HDR function generation unit 901, which may occur, for example, for a day scene (and a main image with no excessively high PB_C) with very bright content and some local darker areas, like, for example, entering a temple in an outdoor scene in India.

Unit 901 will determine an output function for any of these three cases, and all the other million cases. The nature of the algorithm is that the function will be similarly shaped, but closer to the diagonal (because if the original function represented a re-grading between, say, an X nits PB_C image corresponding to a (reasonably similar to the extent capabilities allow) Y nits PB_C2 image (let's say an image of 100 nits), then re-grading from an X to a Z nits PB_C image with, say, Z halfway between X and Y will involve a similar re-grading, but to a lesser extent; if mapping from X to X, there will be a constant transformation corresponding to the diagonal).

There are several ways in which one can define such a normalization algorithm to uniquely obtain the output F_H2hCI_1, F_H2hCI_2, and F_H2hCI_3 brightness mapping functions corresponding to the respective input functions, and the details of which do not really form an essential element of the invention, except for the fact that there must be some normalized algorithm that behaves in this way available. For example, some metric can generally be defined (quantifying the PB_C_CH dependency at the maximum brightness that can be encoded for the selected PB_C_CH IDR image) that can be used to shift the points y(x) of the input function towards the diagonal for any normalized input brightness in some way (e.g., large equal steps, or non-uniformly, etc.). Although vertical offsets are also possible, the preferred embodiment described below, which works quite well, offsets such function points on a trajectory orthogonal to the diagonal line from [0,0] to [1,1].

An advantageous embodiment of the high dynamic range video encoder (900) is characterized in that the standardized algorithm of the HDR function generation unit (901) applies a compression towards the diagonal of the main brightness mapping function (FL_50t1) to obtain the adapted brightness mapping function (F_H2hCI), the compression involving scaling all output brightness values of the function by a scaling factor, the scaling factor being dependent on the first maximum pixel luminance (PB_C_H50) and the second maximum pixel luminance (PB_CH).

There may be various defined F_L50t1 functions (the para definition below is one example), and they may be scaled in various ways by the standardized algorithm, but generally there will be scaling involved, and this scaling depends on the starting PB_C_H50 and the target value PB_CH=PB_IDR. This may be done by different metrics, but the applicants have found over the years that it is easy to define the scaling factor based on a ratio of the peak luminances that are visually psycho-uniform values and by sending them through the v function, i.e. define a scaling factor based on the v function brightness output corresponding to the two peak luminances (and possibly the third PB_C for the SDR image).

An advantageous embodiment of the high dynamic range video encoder (900) comprises: a limiter (1804) configured to redefine a slope of the channel brightness mapping function (F_I2sCI) for a sub-range of the normalized luminances including the brightest normalized luminance equal to 1.0. This is not essential for many embodiments, but is in particular a useful way of handling the particular choice of the coding HG_COD of the high brightness gains of para standardized in ETSI2 to fully comply with the useful proprietor of this particular embodiment.

A corresponding mirroring technique for the encoder (in fact, eliminating all encoding processing by being able to re-derive all required information (even if such data is not actually communicated)) is a high dynamic range video decoder (1100) having an image input (1110) for receiving an intermediate dynamic range image (IDR) having a second maximum pixel brightness (PB_CH) that is smaller than a first maximum pixel brightness (PB_C_H50) of a main high dynamic range image (MsterHDR) by a multiplication factor preferably of 0.8 or less, the second maximum pixel brightness (PB_CH) being obtained by a second meta-data input (1112) receiving, the decoder having a first meta-data input (1111) receiving a brightness mapping function (F_I2sCI), the brightness mapping function defining the transformation of all possible normalized brightnesses of the intermediate dynamic range image (IDR) to the corresponding normalized brightness of an LDR maximum pixel brightness low dynamic range image (Im_LDR), the decoder is characterized in that it has a third meta-data input (1113) receiving the first maximum pixel brightness (PB_C_H50), and the decoder comprises: -                    a brightness function determination unit (1104) configured to apply a normalized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I), the decoded brightness mapping function specifies as output a corresponding normalized HDR brightness of the master high dynamic range image (MsterHDR) for any possible input normalized brightness of a pixel of the intermediate dynamic range image (IDR), the normalized algorithm using the values of the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH); and -                    a color converter (1102) configured to continuously apply the decoded brightness mapping function (F_ENCINV_H2I) to the input normalized brightness of the intermediate dynamic range image (IDR) to obtain normalized reconstructed brightness (L_RHDR) of pixels of a reconstructed master HDR image (REC_M_HDR); the decoder further having an image output (1120) to output the reconstructed master HDR image (REC_M_HDR). The LDR maximum brightness is again preferably the standardized 100 nit SDR brightness, although future variations are envisioned where similar operation is deployed in which the low (i.e., lowest) dynamic range (i.e., maximum brightness) image of the regraded image spectrum and its communication is, for example, a 200 nit image.

So, the MsterHDR image is not actually received as an image, but it is still uniquely defined by the data received (so, although formally, this MsterHDR image exists in the corresponding matching decoder location with the corresponding host image, and the decoder just reconstructs a nearly identical REC_M_HDR image from the IDR image it receives, various functions do define the MsterHDR image properties even at any decoding location). Different consumers may choose various values for both PB_C_H50 and PB_IDR. The first may be chosen by the content creator for various reasons, e.g. because he buys a 4000 nits rated monitor, or because he likes to give a certain optimal quality to his main content (e.g. everything is established/defined with a PB_C of no less than 10,000 nits), or because, at least according to the creator, certain types of images require a certain quality, namely PB_C_H50 (e.g. a spectacular firework show or light show or a pop concert comparable to, for example, a reasonably evenly lit tennis match or news reading deserves a higher PB_C_H50).

The PB_IDR value may be chosen based on various technical considerations, such as the pricing of a typical consumer to a video communications company, and as mentioned, the communications company may typically be different than the creative company.

In general, it is not reasonable to regrade IDR content that results in a difference in PB_C of less than at least 20% (i.e., a factor of 0.8, although in principle the value of PB_C could be closer, such as 0.9), but it is more common to have a multiplication factor of 2 or more between PB_Cs (e.g., 2000 nit main material sent at a PB_CH that is lower than 1000 nits (e.g., 800, 700, or 600 nits) and generally higher than 500 nits). The PB_C_H50 at the decoding location is generally similar to other metadata, and in particular the PB_CH value, so it is generally received as metadata associated with the video data, such as a non-restrictive SEI message, or a special packet on a video communication protocol, etc. (whether in one logical data structure or several structures, according to what is best suited to each standardized or non-standard video communication protocol, which is a minor detail of the new technology). Since the decoder uses a standardized algorithm to ultimately come to an IDR image and its ETSI2 compliant metadata, a corresponding standardized algorithm can be designed for or in the decoder which ultimately determines the F_ENCINV_H2I brightness mapping function needed for the reconstruction of the REC_M_HDR image pixel brightness (and then further doing anything with this image, displaying it is a typical application, but storing on hard disk memory for example is another application).

An interesting embodiment of the high dynamic range video decoder (1100) is characterized in that the standardized algorithm of the brightness function decision unit (1104) calculates a scaling factor which depends on the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH). As mentioned, this can be done in various ways accordingly, but a visually psychologically uniform v-function-based scaling factor is practically quite easy to use for well-controlled HDR image processing and meets various, even critical, artistic requirements, while keeping the technical complexity under control.

A useful embodiment of the high dynamic range video decoder (1100) has the brightness map function (F_I2sCI) defined by a brightness map consisting of a first linear segment with a first slope (SG_gr) for a dark normalized brightness range, a second linear segment with a second slope (HG_gr) for a bright normalized brightness range, and a parabolic segment for brightnesses between the two ranges. The corresponding mathematics involves, among other things, solving second order equations to obtain the channel-adapted high brightness gains required for the reconstruction. This is a useful first order HDR re-grading method that is applicable to markets with non-highest pixel color control requirements, such as, for example, real life television broadcasts (as opposed to, for example, detailed color control sometimes involved in, for example, blockbuster movies). As mentioned below, this in some further partitioned embodiments may be a separate component that fully defines the F_L50t1 function and all derivable functions (e.g., the function communicated with the IDR image: F_I2S), but it may also be a partial definition of the rescaling function, e.g., defining the entire rescaling together with a customizable function as illustrated in FIG. 4 .

A useful embodiment of the high dynamic range video decoder (1100) has a color converter (1102) thereof, which is configured to calculate the pixel brightness of a medium dynamic range image (MDR_300) having a maximum pixel brightness (PB_MDR), the maximum pixel brightness being not equal to the equivalent value of 100 nits, the first maximum pixel brightness (PB_C_H50), and the second maximum pixel brightness (PB_CH), and the decoder has an image output (1122) for outputting the medium dynamic range image (MDR_300). While the reconstruction of a REC_M_HDR image may be all that is needed for some devices in some submarkets (with all kinds of other transformations possible applied to the reconstructed image), it would be advantageous if our SLHDR2PLUS decoder could, in addition to reconstructing only the main HDR image, also use other PB_Cs to calculate the corresponding image (e.g. an MDR image that can be directly displayed on some display with any PB_C). This would also use the mathematical principles of the invention, for example in the manner illustrated in FIG. 16 or in any equivalent manner.

Another useful embodiment of the high dynamic range video decoder (1100) has a meta-data output (1121) for outputting a brightness mapping function (F_L_subsq) which defines, for all normalized brightnesses of the reconstructed main HDR image (REC_M_HDR) or, alternatively, the medium dynamic range image (MDR_300), a corresponding brightness of an image having another maximum pixel brightness, which is preferably 100 nits, or a value higher or lower than the maximum brightness value of the respective reconstructed main HDR image (REC_M_HDR) or, alternatively, the medium dynamic range image (MDR_300). It is possible to reconstruct the received IDR image into a REC_M_HDR image that is not directly displayed on a basic monitoring display, but sent to some system that performs further colorimetric calculations. It is then useful that the decoder embodiment may also output a suitable brightness mapping function, i.e., generally meaning a brightness mapping function associated with the image it is being output, such as a REC_M_HDR image (generally associated in the sense that the input normalized brightness of the function being defined is the normalized brightness of the image being output together, and the output of the function is the normalized brightness of a reference image (usually an SDR image) which, when normalized to have PB_C = 100 nits, is generally the minimum quality that would be desired in the HDR era, without excluding the fact that one might want to apply the present teachings with a PB_C for defining the output vertical coordinate of the commonly transmitted function to be, for example, 80 or 50 nits).

Anything specified for a device (or a portion or collection of devices) may be equivalently specified as a signal, a memory product containing an image (e.g., a Blu-ray disc), a method, etc., for example:

A method for high dynamic range video encoding of a received input high dynamic range image (MsterHDR) having a first maximum pixel brightness (PB_C_H50), comprising receiving a master brightness mapping function (FL_50t1), the brightness mapping function defining a relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR), the low dynamic range image having an LDR maximum pixel brightness preferably having a value equal to 100 nits, the method being characterized in that the encoding further comprises receiving a second maximum pixel brightness (PB_CH), and the encoding comprises: -                    Applying a normalized algorithm to transform the master brightness mapping function (FL_50t1) into an adapted brightness mapping function (F_H2hCI), the adapted brightness mapping function causing the normalized brightness of the input high dynamic range image to be related to the normalized brightness of an intermediate dynamic range image (IDR), the intermediate dynamic range image being characterized by having a maximum possible brightness equal to the second maximum pixel brightness (PB_CH); -                  Applying the adapted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR); -                 A channel brightness mapping function (F_I2sCI) is derived based on the master brightness mapping function (FL_50t1) and the adapted brightness mapping function (F_H2hCI), the channel brightness mapping function defining as output the respective normalized brightnesses of the low dynamic range image (Im_LDR) when the respective normalized brightnesses of the intermediate dynamic range image (IDR) are given as inputs, which brightnesses in turn correspond to the respective brightnesses of the input high dynamic range image (MsterHDR); -                  output the intermediate dynamic range image (IDR); and -                  output the second maximum pixel brightness (PB_CH), the channel brightness mapping function (F_I2sCI), and the first maximum pixel brightness (PB_C_H50).

Alternatively, a method for decoding high dynamic range video upon receiving an intermediate dynamic range image (IDR), the image having a second maximum pixel brightness (PB_CH) lower than a first maximum pixel brightness (PB_C_H50) of a master high dynamic range image (MsterHDR) by a multiplication factor preferably 0.8 or less, the second maximum pixel brightness (PB_CH) being received as metadata of the intermediate dynamic range image, the decoding method also receiving the metadata a brightness mapping function (F_I2sCI) in which the brightness mapping function defines the transformation of all possible normalized brightnesses of the intermediate dynamic range image (IDR) to the corresponding normalized brightness of an LDR maximum pixel brightness low dynamic range image (Im_LDR), and the decoding method is characterized in that it receives the first maximum pixel brightness (PB_C_H50), and the decoding method is characterized in that it comprises: -                    Applying a normalized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I), the decoded brightness mapping function specifies as output a corresponding normalized HDR brightness of the master high dynamic range image (MsterHDR) for any possible input normalized brightness of a pixel of the intermediate dynamic range image (IDR), the normalized algorithm using the values of the first maximum pixel luminance (PB_C_H50) and the second maximum pixel luminance (PB_CH); -                    Applying the decoded brightness mapping function (F_ENCINV_H2I) to the normalized brightness of the intermediate dynamic range image (IDR) to obtain the normalized reconstructed brightness (L_RHDR) of the pixels of a reconstructed master HDR image (REC_M_HDR); and -                  outputting the reconstructed master HDR image (REC_M_HDR).

FIG. 25 illustrates the SLHDR2PLUS encoded HDR image signal (2501) of the present case, i.e., for example, a 4kx2k pixel color matrix 2502 (YCbCr, or any color representation that can be calculated into the required YCbCr representation by a pre-processor [not shown] using known chromaticity equations), and the necessary metadata: the luminance mapping function F_I2sCI, and the two PB_C values. If this HDR image signal is communicated to and received by a standard SLHDR2 decoder 2510, since F_I2sCI is a normal function, the decoder will be expected to rescale from its received image peak brightness (PB_CH, in this example 800 nits to any lower peak brightness, which can be optimized for display for, e.g., a connected 350 nit medium dynamic range display by display optimization calculation Im350 (which, as mentioned, is not a critical aspect of this application, it only makes possible as one of the starting design criteria to the novel codec framework, and can be used for display optimization, e.g., to reveal 20160307602, or a similar one). But it now becomes possible that anyone with a SLHDR2PLUS decoder 2520 (e.g., newly deployed by a cable TV operator who decides to introduce this service, and the like) can produce other images above the PB_CH value of PB_C, such as a reconstruction of the exemplary master HDR image at 5000 nits (i.e., the output image Im5000), or any display-conditioned image between PB_C_H50 and PB_CH, or possibly even above PB_C_H50 (such as the Im1250 display-conditioned image), etc.

Fig. 9 schematically shows the new SLHDR2PLUS encoder 900. When input, it gets a master HDR image (e.g., a 5000 nit PB_C image MsterHDR), which, without losing the commonality that the reader may assume has been generated at or around the time of encoding by a human color grader using color grading software, e.g. by starting with an HDR image captured from an unprocessed camera (the MsterHDR image is optimized for, e.g., a TV viewing environment that is typically dark at night (i.e., it has average ambient luminance, etc.); the present technique may also work with other or variable environments, but it is more a matter of display adaptation rather than a new method of creating or encoding HDR images). The grader has also established at least one good brightness degradation function FL_50t1 to convert the 5000 nit master HDR image into a corresponding good looking SDR image that he has checked on his SDR reference monitor (i.e., typically a 100 nit PB_C image), and he has done this by filling in some of the partial regraded states of 403, 404, and 405 according to the chroma processing unit 451, and some good color adjustments F_C[Y]) (for example, in real life event broadcasting, other methods may calculate the applicable function shape on the fly, and then there may be some director who roughly views the result, or even none, but the principle is that a good function FL_50t1 emerges, whether it is from just one of the partial units or the total function of all units together, etc.).

This function FL_50t1 must also be input as the start information of the novel encoder 900. The peak brightness static (for overall movie or broadcast) metadata PB_C_H50 is also input, as it will be used, but is also output by the encoder as the total IDR video coded signal (IDR+F_I2sCI+PB_CH+PB_C_H50, where the image is generally compressed or decompressed according to some suitable video communication standard (e.g., HEVC), and other metadata can be communicated according to any available or configurable metadata communication mechanism, ranging from MPEG SEI messages to dedicated Internet packets, etc.).

The HDR function generation unit 901 will calculate the HDR to IDR brightness mapping function F_H2hCI, which is required to calculate the IDR image from the Mster HDR image, and it will require the selection of the PB_CH for the IDR, assuming it is obtained from some other input (for example, this may have been selected by the cable TV operator and placed somewhere in memory to be loaded by the configuration software); PB_CH will be assumed to be equal to 1000 nits (for illustrative purposes only; generally speaking, this value will be several times higher than the SDR PB_C (for example 4x higher), and the technical aspects are somewhat different in implementation details based on the selected value).

FIG . 10 illustrates how the HDR function generating unit 901 may function.

Assuming that the grader has defined a certain function (here in the illustrated example a linear-parabolic-linear function (abbreviated to para ), the applicant uses it according to the ETSI standardized codec principles to perform a first and mostly good rebalancing of the brightness of the dominant image areas (i.e. it provides sufficient visibility for dark colors in SDR images, for example at the cost of co-controlled compression of the brightest brightness areas).

This function relates the input luminance of the darkest sub-region of luminance (L＜Ld) (in a psycho-visually equivalent representation by transforming pixel luminance according to equations 1 and 2 above) to the desired output luminance by a linear relationship with a controlled slope SG_gr optimally chosen by the grader for this HDR image: Ln_XDR= SG_gr *Ln_Mster_HDR    if(Ln_Mster_HDR＜Ld)  [Equation 4] (where Ln_Mstr_HDR and Ln_XDR are the brightness of the input master HDR image when pre-graded by the grader as the best starting image (i.e., the visually uniform representation of the corresponding pixel brightness), and Ln_XDR is the summary of several output brightnesses of images with different dynamic ranges and especially peak brightness PB_C, all displayed on the same normalized vertical axis for explaining the concepts behind the present invention and its embodiments). In particular, when the grader begins to regrade the corresponding best SDR image for his best graded Master_HDR image, the XDR will be that kind of SDR, and the corresponding brightness mapping function shape will be shown as F_Mt1 [the shorthand symbol xty is used to indicate that the function maps brightness from the starting PB_C x to the ending PB_C y, and either x and y may roughly indicate the PB_C of the image, such as M indicating Master, or numerically indicating an instance value with two zeros dropped, for example, 50 means 5000 and 1 means 100 nits].

Similarly, for input brightness Ln_Mster_HDR higher than Lb, there is again a controllable linear relationship: Ln_SDR=HG_gr* Ln_Mster_HDR +(1-HG_gr) if(Ln_Mster_HDR＞Lb) [Equation 5]

The parabolic portion of para extending between Ld=mx-WP and Lb=mx+WP has the functional definition of L_XDR=a*x^2+b*x+c, whose coefficients a, b, and c can be calculated by calculating the point where the line intersects the curve from its pole and its abscissa mx (as defined in the ETSI1 standard; mx=(1-HG)/(SG-HG)).

The general idea underlying the invention is as follows (and it can be explained in a multiplicative view). Any master HDR luminance can be transformed into itself by applying a constant transformation (diagonal). If at the end of the regraded image's spectrum (i.e. to create the corresponding SDR luminance (XDR=SDR)) an output luminance L_SDR=F_Mt1(Ln_M) has to be obtained, where Ln_M is some specific value of the Ln_Mstr_HDR luminance, this can also be viewed as a multiplicative boost of the input luminance L_SDR=b_SDR(Ln_M)*Ln_M. If now some intermediate function F_Mt1_ca can be defined, the final processing is the successive application of two functions F_IDRt1(F_Mt1_ca(Ln_Mster_HDR)), where F_IDRt1 performs the final brightness mapping starting from the calculated IDR pixel brightness (derived from the main HDR brightness) towards the SDR brightness of any pixel (or object). In the form of multiplication terms, it can be said that L_SDR=b_IDR*b_ca*Ln_M, where the two boosts correspond to the intermediate function (or channel adaptation function) and the rest of the related function (which happens to be the function that is passed along with the IDR image to create ETSI2 compliant HDR video coding). Note that these boost factors are themselves functions of Ln_Mster_HDR (or in fact any intermediate brightness associated with it).

Now this is convenient if there is no need to pass any additional functions (which might for example be lost if metadata management is not perfect etc).

It may therefore be useful if the SLHDR2PLUS principle uses a predetermined fixed way to transform the grader's F_Mt1 function (i.e. a mechanism for whatever function shape he wishes to use) into a channel adaptation function corresponding to PB_IDR (which value is generally also communicated to the receiver as PB_CH according to the ETSI2 coding method). It can be shown that the up-grading function F_H2h does not need to be communicated together in the metadata associated with the IDR picture, since it is fixed and known to the decoder, so the inverse function F_?? may be calculated from the received F_I2s function, as will indeed be shown (if PB_C_H50 is also communicated to the receiver). The novelty of the decoder is this new way of deriving pictures for which PB_C > PB_IDR. In theory, any fixed method of deriving the F_Mt1_ca function from the main F_Mt1 could be done, provided that it is mathematically reversible or at least decodable on demand, but it is desirable to choose a method for performing HDR to IDR regrading (i.e. deriving the F_Mt1_ca shape) such that its use in deriving further deformations of the MDR image is compatible with what would result from ETSI2 (in theory, just normalize ETSI2 images between PB_C and 100 nits, so one can do the same for PB_IDR and 1 00 nits, but one may also try to impose technical constraints on the solution to be obtained, that is, the image upscaled from the received IDR towards the master HDR image (i.e. using F_?? to be calculated by the SLHDR2PLUS decoder) has the same look as would be obtained by display adaptation of ETSI2, which is used to receive, for example, 5000 nit PB_C Master_HDR images, and the overall brightness remapping function F_Mt1.

It is first explained how this better channel adaptation (i.e., the calculation of F_Mt1_ca, or F_H2hCI calculated in Figure 9; and the corresponding (multiple) IDR image) can be designed to be useful for several methods/embodiments of SLHDR2PLUS.

FIG. 12 a shows the white level offset WLO_gr, and if available, the black level offset (BLO_gr), optimally chosen by the grader (or automatically); corresponding to cell 403 in FIG. 4 .

It can be assumed at this point that this is the only dynamic range adjustment, i.e. a luminance mapping operation to obtain an SDR image from the Mster_HDR starting image (this white-on-white and black-on-black is more precisely a basic type of dynamic range conversion giving poor quality LDR images which have neither the correct average brightness nor average visual contrast, let alone the desired higher image quality descriptors of the resulting image, but it is a good step as a first step in the re-grading chain according to the applicant's method and we must first explain this step and its channel adaptation). The idea is that if (despite the possibility of code brightnesses up to PB_HDR = 5000 nits) there are actually no pixel luminances higher than the value MXH in the current image to be mapped (or in the image shots in the video of the same scene if one decides to use the same function for all such time-series images), then it is reasonable to map the highest MXH to the maximum luminance code in SDR (i.e., e.g. 1024, corresponding to a brightness of 100 nits). Any other mapping method (e.g. HDR-white-on-SDR-white mapping) would make all actually current luminances even darker, and is not optimal, given that the SDR luminance range is already small enough, and a large range of correspondence simulations is still needed to optimally contain HDR luminances.

The question then is whether this WLO value should be adjusted for the IDR image (as can be seen in Figure 12b, the brightest luminance in the intermediate image may have dropped to be closer to PB_IDR, which will still be useful for the final offset of the SDR regraded image to map on 1.0; the mapping can also be equivalently shown on the normalized luminance range of the HDR 5000 nit image, as indicated by ON). In the first approach, it is not needed (because there is some freedom in how to design the algorithm for calculating the F_Mt1_ca function), but if it is adjusted proportionally, the following method can be used.

The scaling factor used for such horizontal scaling needs to be determined in order to be able to scale the brightness mapping function, which in this case is its parameter WLO_ca, and similarly scale BLO_gr (symbol BLO_ca). If it is desired that this parameter scales linearly with PB_IDR, then the limitation is that this action is done exactly, i.e. when PB_IDR=PB_SDR, the offset has its maximum extent BLO_gr. On the other hand, for HDR images, BLO or WLO should be zero, since there is a constant transformation for mapping 5000 nits Mster_HDR to Mster_HDR, so there is nothing to correct for.

Therefore, the following definition of parameters can be made: WLO_ca=scaleHor*WLO_gr    (0＜=ScaleHor＜=1) BLO_ca=scaleHor*BLO_gr      [Equation 6]

The question then is how to define ScaleHor.

Fig . 12b shows the spectrum of different dynamic ranges, more specifically, different PB_C images organized along the horizontal axis. They are located along the perceived position of the peak brightness PB_C of each image. Therefore, they are placed on the horizontal axis position v(PB_C), whereby v is the function of equation 1, with the value PB_C for the parameter L_in, and with the value RHO of equation 2 calculated for the peak brightness of the graded Master_HDR image (i.e., for example, RHO is 25 for a 5000 nit PB_C Master_HDR image). If the vertical axis also has its brightness L parameterized according to the v function (on the vertical axis), with the same RHO=25, then PB_C follows a straight line nicely and can be defined and calculated in this framework. For example, the brightness of the peak brightness PB_C of any intermediate image can be projected onto the primary (5000 nits) brightness axis. The notation used is "P_I1oI2", meaning the value of brightness corresponding to the application of the v-function of the peak brightness of image I1 (which is normal brightness) when represented over the brightness range of image I2. So, for example, P_IoH is the brightness of the peak brightness of the selected IDR image over the Master_HDR brightness range, and P_SoH is the brightness of 100 nits (note that 1.0 on this range corresponds to PB_C of the Master_HDR image, so that, for example, the position of 100 nits (e.g., 0.5) will change depending on the Master-HDR image representation chosen, which is why Equations 1 and 2 are a family of curves parameterized by RHO).

Then the appropriate function for ScaleHor would be to start with 1-P_IoH. The more PB_IDR decreases, i.e. the further to the right one is chosen, the more this function will actually increase our IDR image representation of the MasterHDR image. And if P_IoH=1, it will produce 0, which happens when a 5000 nits IDR image is chosen (purely for theoretical explanation of the scaleHor equation, as it is not technically sound). However, when IDR=SDR, this equation is not equal to 1.0, so it needs to be scaled using a factor k.

If k = 1-P_SoH (which is a fixed value that is compared to the variable P_IoH corresponding to various IDR positions), it can be proved that the regularization is correct, so: ScaleHor = (1-P_IoH)/(1-P_SoH)         [Equation 7]

The determination of the correct para for the channel conversion (FIG. 4, unit 404) is more complex and is illustrated in FIG . 13 .

In this case, the inventors decided to perform the function transformation in a diagonal direction orthogonal to the constant diagonal ([0,0]-[1,1]). This must be transformed with equivalent parameterization in the normal Master_HDR/XDR coordinate system representation of all functions rescaled.

The basic scaling is defined in an axis system that is a 45 degree rotation that changes the diagonal to the horizontal axis (Fig. 13a). This is seen as a function Fx of the rotated para, for example. It is reasonable to scale any value dY at a point on the rotated diagonal (i.e., the new x-axis) by a factor La/K (this dX corresponds to some horizontal coordinate, i.e., L_Mster_HDR brightness in the original axis system), whereby K is the full motion of the function, i.e., the full dY value, and the scaled dY_ca value in this rotated system will be (La/K)*dY.

Define sc_r=La/K, where La= 1/P_IoH and K=1/P_SoH (note that the values of I2 brightness on the I1 axis can be re-scaled to the values of I1 brightness on the I2 axis, in particular, for example 1/P_IoH=P_HoI; for example, if P_IoH=0.7, this means that PB_Mstr_HDR will be pegged 1/0.7 above PB_IDR).

Now we need to calculate the equivalent vertical scaling sc* of the diagonal sc_r.

This can be done by applying the reverse rotation math (actually by first defining K and La to be 1.0 instead of 1.4) to bring the representation of Figure 13a to the diagonal of Figure 13b. This is produced by a matrix rotation (rotating to any x_r, y_r in the diagonal system of the main representation, such as 1, dY): [x1,y1]=[cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoI= 1/La] [x2,y2]=[cos(pi/4) -sin(pi/4); sin(pi/4) cos(pi/4)]*[1, P_HoS= 1/K] [Equation 8]

Note that because the diagonals are scaled, both the x- and y-coordinates will change, but in any case SG and HG (and any other scaled point changes) are defined as slopes, not angles.

The rotation of the line from (0,0) to the square representing the diagonally scaled point of the brightness mapping function in Figure 13b to the line from (0,0) to the circle representing the original brightness mapping function point (or vice versa) can be found by dividing the slope by any fixed abscissa value (e.g., a) (with an angular change corresponding to a vertical change in the normalized scale factor sc*): sc* = (y2/x2)/(y1/x1)=[(1+1/K)/(1-1/K)]/[(1+1/La)/(1-1/La)]=[(K+1)/(K-1)]/[(La+1)/(La-1)] =[(La-1)*(K+1)]/[(La+1)*(K-1)]         [Equation 8]

Then, the actual vertical coordinate distance n corresponding to the full vertical scaling (sc*=1) must be calculated, and this can be done by taking the distance Fd from the midpoint of the scaled mip (the intersection of the two linear segments (mx, my) below it to the diagonal and above it to para) due to the 45 degree angle involved in the diagonal. Therefore, n=Fd is equal to half the differential slope SG-1 at mx, that is, mx*(SG-1)/2.

Then, the offset intersection point (mxca, myca) must be calculated as follows: mxca=mx+d= mx+ [mx*(SG-1)/2]*(1-sc*) myca=my-d=SG*mx-(mxca-mx)= -mxca+ mx*(SG+1)     [Equation 9]

Using the position of the new point, the channel-adjusted shadow gain (SG_ca, see Figure 10) and channel-adjusted highlight gain HG_ca can finally be calculated: SG_ca = myca/mxca HG_ca=(myca-1)/(mxca-1) [Equation 10]

Finally, there are several methods/implementations for the middle section of the parabola.

In one approach that produces quite good visual results in practice, WP_ca=WP_gr is taken, where WP_gr is the original width of the parabola segment when optimized by the content creator's grader or automatically for the master HDR and master SDR images, and WP_ca is the width of the channel-adapted para function. Another approach is to define WP_ca=v(abs(sc*), 100)*WP_gr, where the v function is again defined by Equations 1 and 2 above.

Making it an available technology, it can be used to define appropriate IDR definitions for SLHDR2PLUS.

Returning to FIG. 10 , the above equations define how a function F_Mt1_ca may be uniquely defined for a selected 1000 nit PB_IDR starting from a 5000 nit master HDR image, for example. If this function is determined by the HDR function generation unit 901, it may be output as F_H2hCI and sent as input to the IDR image calculation unit 902. This unit will apply this function to all pixel luminances of the MasterHDR image received as image input [L_IDR=F_H2hCI(L_MsterHDR)= F_Mt1_ca(L_MsterHDR)] to obtain the corresponding IDR image pixel luminances, and will output the IDR image.

Now the question remains which brightness mapping function to add to the IDR metadata so that it appears as a normal ETSI2 image (i.e., so that any conventional ETSI2 decoder can decode it normally, producing an SDR image or any MDR image that looks like it should).

This secondary IDR brightness mapping function F_I2sCI (which will also be para) can be defined as follows (and will be calculated by the IDR mapping function generator 903). The shadow gain for the IDR image SG_IDR can be considered as the remaining multiplication (or slope) after going from Master_HDR to the IDR image (i.e., the remaining relative brightening starting from the IDR image to obtain the SDR image): Y_out (x_in)=SG_gr*x_in; = F_I2sCI(L_IDR=SG_ca*x_in)

It is also known that the same para linear segment mapping used for the darkest pixel is applied to the new IDR brightness input: Y_out=SG_IDR*L_IDR Thus: SG_gr= SG_IDR*SG_ca [Equation 11] (For example, take the input x_in=L_Mster_HDR=0.2, which is mapped diagonally to L_IDR=0.3=(0.3/0.2)*x_in, which is finally mapped to Y_out=0.4=k*0.3, where k=0.4/0.3; Y_out=SG_gr*0.2=(0.4)*0.2=(0.4/0.3)*(0.3/0.2)*0.2).

Therefore, from Equation 11, the required SG_IDR is calculated in this way (given that the fixed method is used to determine SG_ca as described above): SG_IDR=SG_gr/SG_ca   [Equation 12] Similarly: HG_IDR=HG_gr/HG_ca [Equation 13]

Wherein, HG_gr is again the optimal high brightness gain determined by the content creator relating the master SDR image sample to the master HDR image sample (i.e., its brightness distribution), and HG_ca is the channel-adapted high brightness gain corresponding to the original high brightness gain HG_gr.

Note that the basic shadow gain adjustment can be determined relative to the expected simple shadow gain from the difference in peak brightness between the SDR and IDR images as: ShadBst = SG_IDR / P_IoS. As described, when expressed on the normalized brightness axis of the SDR image, P_IoS is the maximum encodable brightness of the IDR image, i.e., 7.0, for example.

Note that there are some practical embodiments where the high brightness gain cannot be larger than a predefined number (codified in the ETSI standard as high brightness gain), in which case a further recalculation of the high brightness gain is required, see below, but this is not necessary for all embodiments. For example, this can be implemented as:

If HG_IDR＞KLIM then HG_IDR_adj=KLIM [Equation 14], where KLIM is preferably equal to 0.5.

Indeed, assuming that the grader has brought HG_gr close to a maximum value of 0.5, and the corresponding HG_ca (which as a softer mapping should have a HG_ca closer to the diagonal, i.e. larger than HG_gr) is, for example, 0.75, the division is found to be 0.67, which is above the maximum value that can be conveyed from a pure ETSI2 HDR video signal when normalized. One solution is, for example, to redefine a smaller HG_gr so that HG_IDR will not be higher than 0.5 (the normalized maximum value). This again requires considerable calculations taking all re-grading aspects into account, as will be shown below. Another option is, for example, to make the IDR+metadata signal conform by limiting HG_IDR to 0.5, while being conveyed as additional metadata (completely unrestricted HG_IDR). HG_gr will generally depend on the PB_C of the Master_HDR image, but will also depend on what kind of image objects are in the image (e.g. bright, colorful objects that are important enough not to have their brightness compressed too much, an extreme example being images of bright planets close to a powerful Sun, which use many very high L_Master_HDR brightness values and very few dim ones). HG_ca will generally depend on, among other things, how close the chosen PB_IDR is to the PB_Master_HDR.

In addition, assume WP_IDR = WP_gr [Equation 15]

As stated, other embodiments are possible, but to illustrate the principles in an easier way, this assumption is made now.

The appropriate channel-adjusted values for the black level offset and white level offset (if any such offsets are defined by the content creator) are calculated using Equation 6. Now all that remains is how to calculate (by the IDR video encoder) the corresponding values for BLO_IDR and WLO_IDR.

First, the coding value glim is calculated in a better way: glim= {log[1+(rhoSDR-1)*power((0.1/100);1/2.4)]/log(rhoSDR)}/{log[1+(rhoHDR-1)*power(1/PB_Mster_HDR;1/2.4)]/log(rhoHDR)} [Equation 16] where rhoSDR=1+32*power(100/10000;1/2.4), and rhoHDR=1+32*power(PB_Mster_HDR/10000; 1/ 2.4)

This will lead to a simple way of adapting the BLO, since in practice in the ETSI1 and ETSI2 standard methods of HDR coding there are also linear gain limiters in parallel with the brightness processing chain (units 402 to 406 in Fig. 4 and 1502 to 1506 in Fig. 15) that apply a linear curve with the angle glim to the perceived Y'HP and compare it with the Y'GL value calculated by the explained unit and take the maximum of the two values calculated in parallel (this is important among other things for the reversibility of ETSI1 to allow reconstruction of the dimmest HDR brightnesses), the figures are only used to easily understand the partially sequential re-grading steps of the described inventor's method.

It can now be shown that due to the action of this limiter, the BLO value can be easily adapted to the channel with the following equation: BLO_IDR=BLO_gr*glim          [Equation 17] glim depends on the specific choice of PB_Mster_HDR as shown above and can be, for example, 0.6.

This is illustrated in Figure 17. Figure 17b shows a zoom-in of the darkest brightness of the full range brightness map shown in Figure 17a. The various functions are again shown on a normalized graph, corresponding to various input PB_C and output PB_C.

FL_gr is a function created by the content creator to map, for example, 4000 nits Master_HDR to SDR. The dotted curve FL_ca is a channel adaptation used to produce, for example, 500 nits IDR from Master_HDR. The dashed curve FL_IDR is the curve that maps IDR brightness to SDR brightness. In the zoomed-in graph of Figure 17b, the FL_gr curve is seen to have a sharp twist at about 0.03 of the input, which is where the parallel gain limiter comes into play (i.e., its linear output y=glim*Y’HP is chosen as the function output for lower brightness inputs, rather than the Y’GL value from the action of all units in the chain shown in Figure 4 (see ETSI1 standard, Figure 4 for the full circuit description)).

The BLO value for any curve is the intersection with the horizontal axis that would occur if there was no gain limiting, i.e., BLO_gr by extending the local slope to above 0.3 of the FL_gr curve as shown by the dotted line, for example.

For this application, it is sufficient to know that the FL_IDR curve can also be extended to obtain the BLO_IDR value (note that there is a glim_IDR value, which is what the ETSI2 standard will use, which is different from glim_gr), and that this lower BLO_IDR value can be found as glim*BLO_gr (note that this glim (the only glim that needs to be calculated for SLHDR2PLUS) is the one shown as glim_gr in Figure 17b).

Subsequently, the following calculation is performed to obtain WLO_IDR.

What is also shown in Figure 17a is that there are three different WLOs, namely the WLO_gr that was originally generated by the grader as his main HDR to SDR mapping strategy (also ON in Figure 12b), the channel-adapted WLO_ca where the FL_ca curve crosses the upper horizontal line, and the one that is the mapping of WLO_gr brightness to the IDR brightness axis (which can be imagined using a representation like Figure 12, where MXH is projected onto MXI, and finally there is also WLO_IDR, which is left to convert IDR luminance down-luminance mapping to SDR's WLO (which differs from the scaled WLO_ca due to the changed normalized luminance coordinate definition when starting from the associated PB_C=5000 for WLO_gr and WLO_ca (because the input image for regrading using these functions is 5000 nits Mster_HDR) to the IDR-related defined PB_C=1000 nits required for regrading (because in an ETSI2 compliant viewpoint the received starting image from which the other images are derived is, for example, a 1000 nits PB_C IDR image).

Figure 17c zooms in on the upper corner of the function graph (close to [1,1]). As shown by the circular projection from the (normalized) ordinate position to the abscissa position, the WLO_IDR value follows the WLO_gr value that will be sent through the FL_ca curve as input. It can be seen in Figure 12b that the MXI position is indeed the normalized position on the IDR brightness axis that is mapped to an SDR brightness of 1.0, so it is as required by the definition of WLO_IDR.

It can be assumed superficially that if the curve through which the WLO value is subsequently mapped on the encoding side is para (see FIG. 4 , unit 404 is mapped after unit 403 ), it will generally involve an upper linear section of para.

However, due to how para is defined, any part of it can be involved (there are even settings where only special values of SG of para define a very high intersection point that theoretically moves above 1.0, so the behavior in cases up to the brightest luminances is determined only by the shadow gain slope, resulting in a linear curve that is useful for regrading to SDR mostly HDR images containing very bright luminances, like for example a desert planet illuminated by 5 suns in a sci-fi movie). So this becomes a bit of a calculation involving testing which of the three sub-parts of para are applicable, the best mathematical implementation is: WLO_co=255*WLO_ca/510 BLO_co=255*BLO_ca/2040 Xh=(1-HG_ca)/(SG_ca-HG_ca)+WP_ca WW=(1-WLO_gr*255/510-BLO_co)/(1-WLO_co-BLO-co) IF WW＞=Xh THEN WLO_IDR=HG_ca*(1-WW)*510/255 [the upper linear segment] ELSE { Xs=(1-HG_ca)/(SG_ca-HG_ca)-WP_ca IF WW＞Xs { [Input, i.e. WLO_gr must be mapped through the parabolic sub-portion of the channel-adapted para] A= -0.5*(SG_ca-HG_ca/(2*WP_ca)) B=(1-HG_ca)/(2*WP_ca) + (SG_ca+HG_ca)/2 C= -[(SG_ca-HG_ca)*(2*WP_ca)-2*(1-HG_ca)]^2 / (8*(SG_ca-HG_ca)* 2*WP_ca) WLO_IDR=(1-(A*WW*WW+B*WW+C))*510/255 } ELSE [In the special case of the shadow gain sub-portion of para applied] WLO_IDR =(1-SG_ca*WW)*510/255 }

These parameters SG_IDR, HG_IDR, WP_IDR, BLO_IDR, WLO_IDR (and similar additional parameters for customizable curves if desired) are the parameters that characterize and are therefore the output of the function F_I2sCI (whether these parameters characterizing the shape of this desired curve for display adaptation are actually output, or whether a LUT characterizing the function is output is merely an implementation choice; the main thing is that the correct brightness mapping function shape F_I2sCI in an axis system normalized to 1.0 is conveyed as meta-data together with the IDR image(s)).

The encoder is now characterized according to the novel SLHDR2PLUS method. The question then is how the decoder should be designed. It must be understood that the decoder will now only get the F_I2sCI function, so it must somewhat compute the function F_?? needed to reconstruct the original Master_HDR image from the received IDR image. In the SLHDR2PLUS encoding method this will be the inverse of the F_H2hCI function used in the encoder to generate the IDR luminance, but such a function should still be computable.

As generally illustrated in FIG . 11 , the SLHDR2PLUS video decoder 1100, the brightness function determination unit 1104 must calculate the F_?? function based only on the information it receives, namely F_I2sCI and the two peak brightnesses PB_CH and PB_C_H50. Once the function is determined, it can be applied to reconstruct the original Master_HDR brightness by applying it (in the color converter 1102) to the received IDR brightness: L_REC_M_HDR=F_??(L_IDR), and the corresponding HDR brightness can be calculated from the brightness by applying the inverse functions of Equations 1 and 2 to the L_REC_M_HDR brightness. Finally, the reconstructed master HDR image (REC_M_HDR) can be output by the color converter 1102 in any format as desired, such as a PQ-based YCbCr color recipe, etc. The decoder 1100 can also in a preferred embodiment be configured to calculate any displayed adapted image, for example MDR_300 if a 300 nit PB_D connected display is to be supplied with the best equivalent of the received HDR image, and this can be done by SLHDR2PLUS mathematics, or just by regular ETSI2 decoding, since the appropriate image (IDR) and brightness mapping function (F_I2sCI) are already available as inputs in the color converter 1102).

Figure 14 shows what is involved in para to reconstruct a REC_M_HDR image from a received IDR image (similar calculations will be done for WLO and BLO, and customizable curve shape points where applicable (it should be noted that as discussed below, some embodiments will not be between Mster_HDR and IDR, but simply apply a customizable curve principle between IDR and SDR as in the SDR downgrade technique).

Now, the new master HDR reconstructed shadow gain (SG_REC) and reconstructed high brightness gain (HG_REC) need to be calculated, and the inverse parabolic equation of the parabolic segment must be calculated to complete the required reconstructed para luminance mapping function shape F_L_RHDR (note that for illustrative purposes only, the inverse SDR to Master_HDR luminance mapping function has also been shown as a dotted line on this normalized graph; it should be noted that due to the inverse function nature of the SDR to HDR mapping, the shadow gain of this curve SG_RM is equal to 1/SG_gr, etc.).

Figure 15 first illustrates some aspects of the core computational topology of a general decoder 1502. As can be seen, it is roughly the same structure as the encoder, although it performs rescaling in the opposite direction (reconstructing REC_M_HDR from IDR), which is convenient as this computational topology can be easily reconfigured as required. If the luminance mapper 1501 gets the overall LUT (of all partial successive rescaling actions), it will indeed operate in a similar way (like a decoder).

Of course, some differences need to be configured to enable the decoder to do the correct HDR reconstruction re-grading. First, L_in will now be the IDR normalized luminance, and the output luminance Lh will be the normalized luminance correctly scaled for, e.g., a 5000 nit PB_D display presentation. Also see that the final multiplier that produces the REC_M_HDR image pixel color (Rs, Gs, Bs) is now multiplied by the PB_C_H50 value received in the metadata. In fact, the sensing outer calculation loops executed by the sensor 1502 and the linearizer 1506 apply the PB_CH and PB_C_H50 values in Equations 1 and 2 and the inverse of these equations, respectively. It should also be noted that now the order of the various partial regradings in their presence is reversed: first the perceived IDR brightness Y'IP is finely graded by the inverse customizable curve in the fine grading unit 1503 which generates the regraded IDR brightness Y'IPG. Afterwards, a first mapping to the HDR brightness axis, i.e. the corresponding redistributed brightness for the corresponding correct HDR sample (in fact, a 5000 nit PB_C_H50 Mster_HDR sample), is performed by the coarse brightness mapping unit 1504 which applies the inverse para of FIG. 14 , which still needs to be calculated correctly, and which will generate the initial HDR brightness Y'HC. Finally, the anti-black and white offset 1505 will establish the correct normalized REC_M_HDR brightness (Y'HR) for further calculations with the chrominance to arrive at the full three-dimensional color of each pixel. As explained, unit 1504 will generally obtain the calculated SG_REC, etc. (or a LUT version of the brightness mapping function to be applied to these three values accordingly). It should be noted that if the various PW values are kept the same, then WP_REC is again WP_gr. Unit 1505 will similarly obtain the black and white offsets (WLO_REC, BLO_REC) used for the reconstruction of Mster_HDR. The lower part of the core unit performing chroma processing (chroma processor 1550) will be similar to the encoder topology of Figure 4, except that the correct C_LUT F_C[Y] is loaded in the chroma processing decision unit 1551 (see its calculation explained below).

The question now is whether and how to calculate the parameters of the function applied in the decoder programmed to reconstruct Master_HDR from IDR (which is not the case before HDR video decoding).

For example, one can see the method used for shadow gain.

Before calculating SG_REC, it can be asked whether the total shadow gain SG_RM from SDR to Master_HDR can be determined, and then SG_REC can be determined from it by division in Equation 12. So SG_IDR = SG_gr / SG_ca It can also be shown that SG_ca = (mx / mxca) * (SG_gr + 1) -1

This can be seen because myca=SG_ca*mxca (by definition of the lower linear section of the channel-adapted para), and it can also be seen that myca=my-d = mx*SG_gr+(mx-mxca).

The second relationship of mxca/mx follows by dividing the upper equation of Equation 9 by mx.

Since SG_ca can be written in terms of SG_gr by filling the first relation into the second (removing the mx/mxca part), the final relation can now be formed between SG_IDR and SG_gr: SG_ca= (SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1 From this: SG_IDR=SG_gr/{(SG_gr+1)/[(SG_gr-1)*(1-sc*)/2+1]-1}         [Equation 18]

Given the known (received) SG_IDR (and sc* has been calculated only from the peak brightness (which is also known) since both PB_CH (i.e. PB_IDR) and PB_C_H50 are received and PB_SDR is typically 100 nits, but can be built into the signal's metadata if this is not the case), this equation can now be solved for the unknown SG_gr.

Using SG_IDR = y and SG_gr =x to simplify notation, then: y=[(x-1)*(1-sc*)*x/2+x]/[x-(x-1)*(1-sc*)/2] Thus: x^2+x*(y-1)*[(sc*+1)/(sc*-1)]-y=0             [Equation 19] [These coefficients (hereinafter referred to as A’, B’, C’) which are functions of y and sc* will be used hereinafter to solve the quadratic equation in the overall system of equations for reconstructing the brightness of the Master_HDR image].

To determine all parameters given the shape of the reconstructed luminance mapping function, the following equations can generally be done in one of the embodiments (this reconstruction is the inverse of the function used to generate the IDR image at the encoder side). First, determine the correct para, from which the black and white offsets can then be calculated.

Calculate rhoSDR again as above, and calculate rhoCH as: rhoCH=1+32*power(PB_CH/10000; 1/ 2.4) mu= log[1+(rhoSDR-1)*power(PB_CH/PB_SDR ; 1/2.4)]/log(rhoSDR) K, La and sc* are calculated as above, where K=P_HoS and La=P_HoI A’=1 B’=(SG_IDR-1)*(sc*+1)/(sc*-1) C’=-SG_IDR

Once it has been possible to determine the necessary parameters of all required functions on the decoder side (note: from other received available parameters SG_IDR etc.), the rest of the decoding will cancel the action of the IDR encoding para as shown in FIG. 10 due to the reversibility of the inverse curve(s) of the just applied coding (e.g. like para in FIG. 14 (suitably shaped by calculating its appropriately defined parameters 1/SG_REC etc.) i.e. the re-decoding of the defined IDR to Master_HDR brightness etc.).

From now on, follow SG_gr=[-B’+SQRT(B’ ^2-4*A’*C’)]/2*A’

Where ^2 indicates square. SG_REC = SG_gr/SG_IDR       [Equation 20]

Therefore, the inverse channel adaptation shadow gain (1/SG_REC) is already known.

Similarly, the required high brightness gain can be calculated. A’’= (SG_REC*HG_IDR -SG_gr)*(SG_gr+1)/(SG_REC+1) B’’=SG_gr-HG_IDR-(SG_REC*HG_IDR-1)*(SG_gr+1)/(SG_REC+1) C’’=HG_IDR-1 MxRec=[-B’’+SQRT(B’’ ^2-4*A’’*C’’)]/2*A’’ IF MxRec =1 THEN HG_REC= 0 ELSE = HG_REC= max[0,(MxRec *SG_gr-1)/(MxRec -1)]

When a para function is defined from its parameters, once its parameters are computed, the required para is defined.

To obtain BLO_REC and WLO_REC, execute the following equations: mx=(1-HG_gr)/(SG_gr-HG_gr) mxca=mx*(SG_gr-1)*(1-sc*)/2+mx myca=mx*(SG_gr+1)-mxca SG_ca=myca/mxca IF mxca=1 THEN HG_ca=0 ELSE HG_ca=max[0, (myca-1)/(mxca-1)] ScaleHor=(1-1/La)/(1-1/K) RHO=1+32*power(PB_C_H50/10000; 1/2,4) glim = {log[1 + (rhoSDR-1) * (0.1/100)^(1/2.4)] / log(rhoSDR)}/{log[1 + (RHO-1) * (1/PB_C_H50)^(1/2.4)] / log(RHO)}; [As before; because Im_PB_C_1 in the ETSI method <> This fixed parallel bypass of the Im_PB_C_2 mechanism, the glim is the same as used by the encoder, the two images are defined as re-graded starting from the same PB_C_1, and in this particular SLHDR2PLUS method are the Master_HDR and IDR images respectively] BLO_gr=BLO_IDR/glim[The inverse of Eq. 17, so this is relatively easy to determine without higher order equations, and then only a fixed per-channel adaptation mechanism needs to be applied to obtain the required WLO_REC, which is equal to the WLO_ca used by the encoder, but now it will be inverted, and addition becomes subtraction] BLO_REC=BLO_ca=BLO_REC*ScaleHor

WLO_REC is then calculated by projecting it through para (as in the coding principle) to be inverted afterwards. IF HG_ca=0 WLO_REC=0 ELSE { BLO_co=255*BLO_ca/2040 Xh=(1-HG_REC)/(SG_REC-HG_REC)+WP_REC Xh_REC=HG_REC*Xh+1-HG_REC WW_REC=1-WLO_IDR*255/510 IF WW_REC＞=Xh_REC THEN WCA=1-(1-WW_REC)/HG_REC ELSE Xs=(1-HG_REC)/(SG_REC-HG_REC)-WP_REC Xsca=SG_REC*Xs IF WW_REC＞Xsca { A’’’=-0.5*(SG_REC-HG_REC)/(2*WP_REC) B’’’=(1-HG_REC)/(2*WP_REC)+(SG_REC+HG_REC)/2 C’’’= - [(SG_REC-HG_REC)*(2*WP_REC)-2*(1-HG_REC)]^2 / [8*(SG_REC-HG_REC)*(2*WP_RE)] WCA=(-B’’’+SQRT(B’’’^2-4*A’’’*{C’’’-WW_REC})/(2*A’’’) WCA=min (WCA,1) } ELSE WCA=WW_REC/SG_REC WLO_REC=(1-WCA)*(1-BLO_co)/[(1-WCA*ScaleHor)*(510/255)]

Note that while BLO is effectively a purely additive contribution with respect to mapping, WLO translates into a multiplicative scaling to the maximum value (e.g., in Figure 4): Y’HPS = (Y’HP-BLO)/(1-BLO-WLO)         [Equation 21]

All this information can generally be populated into a single brightness processing LUT that relates Y'IP (e.g. in the perceptual domain) to Y'HR (or better still an overall LUT that defines Lh for each L_in value). This will reconstruct the REC_M_HDR image.

As mentioned above, it is also useful if the decoder can directly output display adapted images, e.g., MDR_300.

The subsequent technique may be used as illustrated in FIG . 16 (where two partial LUTs are used, it is most useful in practice to load only one LUT, called P_LUT, since the luminance calculation is on track in a better core computing unit, e.g., each pixel color processor of a dedicated decoding IC, which is typically simply implemented as a LUT. The Y_IDR brightness values are input (e.g., typically PQ-based YCbCr encoding), and they are converted by the linearizer 1601 into normalized luminance L_ in. The sensor 1602 operates as explained above (Eq. 1 and Eq. 2) and uses the RHO value for the IDR peak brightness PB_IDR, e.g. 1000 nits. This produces a sensed IDR brightness Y'IP. The luminance mapping unit 1603 reconstructs the master HDR image as explained above, i.e. it obtains all calculated parameters defining the IDR to MasterHDR reconstruction luminance mapping function F_L_REC, or generally obtains the function of the form This produces a reconstructed Master_HDR brightness Y'HPR. This image forms a good basis for calculating the low dynamic range/peak brightness PB_C image. Basically, this operates like the ETSI2 mechanism if the right functions are applied. These functions can be scaled from the F_L_IDR commonly communicated as metadata, or calculated from a reconstructed F_50t1 function that the content creator defines on his side as the best function to get from Master_ The HDR image calculates a reconstruction of the function of the main SDR image. This F_50t1 function can then be calculated into an appropriate display adaptation function F_L_DA for, for example, 300 nits PB_D according to the principles defined in the ETSI2 standard (the reader is referred to the standard for this detail). This is loaded into the HDR to MDR brightness mapper 1604 (assuming there is one). In practice, a single P_LUT will contain all the actions of F_L_REC and subsequently F_L_DA.

Finally, the obtained MDR relative brightness is sent to the first multiplier 454 of Figure 4 for the same processing (also using the correct accompanying F_C[Y]).

Finally, a suitable C_LUT needs to be calculated (F_C[Y] in Figure 4 or Figure 15 respectively) which gives the appropriate chromaticity to the brightness regraded output colors (to have a look as close as possible to the Master_HDR image, i.e. the chromaticity of the output image pixels and the Master_HDR image should be almost the same, possibly giving a different smaller dynamic range).

The C_LUT used for Master_HDR reconstruction is as follows (other re-grading C-LUT operations follow similar principles, e.g. taking ETSI2 teachings into account).

First the CP-LUT is calculated, which is the inverse of the P_LUT mentioned above applied at the encoder to map the Master_HDR image to the IDR image (so in the decoder this inverse chrominance correction will be used to inversely convert from the received IDR image chrominance Cb and Cr to the Master_HDR reconstructed chrominance).

Then the C_LUT for Master_HDR reconstruction can be calculated as follows: XH=v(PB_M_HDR; 10000) XS=v(PB_SDR=100; 10000) XD=v(PB_D; 10000) XC=v(PB_CH; 10000)

where v is again the function v(x,RHO) defined in Equations 1 and 2 above. CfactCH=1-(XC-XS)/(XH-XS) CfactCA=1-(XD-XS)/(XH-XS) C_LUT[Y}=[1+CfactCA*power(CP_LUT[Y] ;2.4)]/[Y*{1+CfactCH* power(CP_LUT[Y] ;2.4)}]        [Equation 22]

The display target PB_D may be set to PB_Mster_HDR for reconstruction, in which case only the divider remains as the C_LUT determiner. In a practical embodiment, the 2.4th power may also be included in the LUT as, for example, CPP_LUT = power(CP_LUT[Y]; 2.4), which may save some operations in some embodiments.

It was mentioned above that some practical embodiments of the SLHDR2PLUS encoder (for current ETSI2 metadata definition compliance) recalculate HG_gr for use in conforming HG_IDR values. This can be done as described below.

For example, the metadata may reserve an 8-bit code for the HG of para, i.e. in this case, since the IDR picture + its metadata should be ETSI2 compliant signaling, the question is whether the required HG_IDR will fit in the allocated code. The standard generally uses a code allocation function to transform the actual required HG_IDR into a certain HG_COD: HG_COD = F_COD[HG_IDR] in [0,255]. For example, FCOD may be 128*(4*HG_IDR), which means that a maximum value of 255 corresponds to a maximum HG_IDR of 0.5.

It is desirable to ensure that the IDR image is generated such that HG_IDR fits exactly within the code range, i.e., a practical embodiment may achieve this by adapting the grader's HG_gr a little (so that the IDR meta-data decision with fixed channel adaptation and based thereon just avoids overflow).

The calculations used for this (optional) embodiment may be, for example: Set HG_IDR=(254*2)/(255*4); Exposure=shadow/4+0.5 [where shadow is the ETSI2 encoding of the shadow gain SG_gr] SG_gr=K*exposure A= SG_gr*(HG_IDR-1)-0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+SG_gr) B=SG_gr-HG_IDR+1+0.5*(SG_gr-1)*(1-sc*)*(HG_IDR+1) C=HG_IDR-1 MxLM=[-B+sqrt(B*B-4*A*C)]/(2*A) IF MxLM= 1 THEN HG_gr_LM = 0 ELSE HG_gr_LM=max[0, (MxLM*SG_gr-1)/(MxLM-1)] Where HG_gr_LM is the adjusted HG_gr value. The rest of the algorithm then works as described above, as if the grader had chosen the optimal HG_gr_LM value from the beginning.

This article details one approach to addressing the design issues of the new SLHDR2PLUS codec. Depending on the technical choices made, especially the aspects that are found to be of critical importance versus other aspects that can be relaxed, there are alternative approaches.

The above mathematics define a completely new way of implementing an HDR decoder, which has at least the core calculation method consistent with the ETSI1 and ETSI2 methods: in particular, although the P-LUT and C_LUT functions of different shapes will be calculated as they are equivalent to those described above (although Figures 4 and 15 detail the technical physics behind how and why our HDR encoding method works, which is actually equivalent to the overall luminance processing of the luminance processing in the luminance processing tracks 401 and 1501 respectively [in a one-dimensional color state, These two are image dependent functional transforms via non-linear correlation] which are performed by loading the correct overall P_LUT brightness mapping function shape, and similarly for the C-LUT called F_C[Y] in units 451 and 1551 respectively), the computational topology can be reused, which is a highly useful property for customers (who have to buy the IC once, e.g. in a STB, and which can be reconfigured to various new encoding strategies by reprogramming the metadata handling but maintaining the per-pixel color transform engine).

It is also possible to design an IDR coding technique that deeply reuses the same ETSI2 decoding mathematics (i.e., partially reclassifying the chain 1503 to 1505), by simply instructing the ETSI2 decoder to appropriately interpolate instead of its normal operation of degrading the received image, and the display adapts it to a display with PB_D < PB_IDR. It should be emphasized that this is not a "blind" extrapolation method that gives "just any" higher dynamic range image that corresponds to the look of an IDR image (i.e. in particular the statistical distribution of the relative brightness or absolute luminance of the IDR pixels), but is actually generated "automatically" in this way by encoding an HDR output image that looks as close as possible like the original Master_HDR image on the content creator's side (which in such embodiments is still not actually received, nor its metadata, e.g. SG_gr). This automaticity is of course not so simple and involves the right approach on the content encoding side. For the decoder in an embodiment of this principle, the received PB_C_H50 secondary peak brightness is equivalently run in the core per-pixel decoder programming as if it were the desired display brightness PB_D (which is then, for example, 5x higher than PB_IDR).

Figure 18 illustrates this approach (block diagram of how the codec math conceptually works). Again, for simplicity, it will be assumed (although it is not necessary to link these choices as needed for this example) that the choice of fixed channel adaptation algorithm is free to be chosen to perform only the para transform linking Master_HDR and IDR, leaving any BLO and WLO (if already applicable to the current image or image lens) and customizable curves to the secondary transform, i.e. IDR to SDR re-scaling, and conveying the metadata pertaining to the ETSI2 compliant IDR signal to the receiver (whether a conventional ETSI2 receiver or a SLHDR2PLUS decoding receiver).

First some introductory definitions:

The inverse curve of the para curve shown in Figure 10 (i.e., with the ETSI standardized shape definition as formulated in Equations 4 and 5 above and the parabolic middle part defined by a*x^2+b*x+c) shall be referred to as the abcara curve in this document for simplicity. According to ETSI1 Section 7 (HDR Signal Reconstruction), it is defined as: L_out = 1/SG * L_in (if 0＜= L_in ＜= xS) L_out = -b/2a+ sqrt(b^2-4*a*(c-L_in))/2a (if xS＜ L_in ＜ xH) L_out = 1/HG *(L_in-1)+1 (if xH＜= L_in) [Equation 23]

where xS and xH are the points where the linear segment changes to the middle segment of the parabola, consistent with how para is defined for encoding (or any other purpose).

The basic principle of what the video encoder embodiment of Figure 18 attempts to achieve is shown in Figure 20 (in this example, an example of a PB_C IDR of 500 nits has been chosen to illustrate, without wanting to say that this approach is somehow limited to or more suitable for lower PB_IDRs).

If there is a fixed mechanism (in an ETSI2 compliant or ETSI2 legacy decoder) to extrapolate from IDR to a PB_C higher than PB_IDR (using such PB_C setting as if it were the display peak luminance), then the encoder can also be designed to invert this process, i.e. by using the inverse function F_ENCINV_H2I (from the IDR signal (i.e., I The IDR image is created by receiving a receiver (DR image + F_I2S function adaptation compliant with ETSI2 specification) and then adding the correct metadata, which as described will be F_I2S, which is derived from the total brightness mapping function F_H2S (e.g., F_50t1) established by the content creator (e.g., human grader) or by automation in any intermediate real-time encoding process.

This relationship can also be formulated from a multiplication point of view: L_SDR=m_F_I2S*m_F_ENCINV_H2I*L_HDR= m_F_I2S* L_IDR L_HDR=m_F_E_I2S*L_IDR

where m_F_I2S or more precisely m_F_I2S(L_HDR) is the corresponding multiplier needed to achieve the brightness re-grading for each any chosen L_HDR value, corresponding to the F_I2S brightness mapping function shape, and similarly for the other multipliers.

So it must be solved that the inverse of para from HDR to IDR (i.e. abcara running from IDR to HDR) has the same effect as extrapolating (starting at any L_IDR) to some para of PB_HDR.

To understand this a bit better, use Figure 21. In normal interpolation mode from a higher input image PB_C (i.e. operating on any normalized input brightness L_in_X corresponding to actual brightness via PB_Ch above the normalized output image brightness PB_D) to a lower PB_D, the original grader's para F_H2S (as received in the metadata via a standard ESTI2 encoded video link) will be scaled diagonally following the arrows towards the diagonal [0,0]-[1,1], yielding F_ENCIV_H2I (which now corresponds to PB_IDR/PB_HDR vs. PB_SDR/PB_HDR (i.e., for example v(100/5000)/v(500;5000)=0.54/0.72 [where v(x;y) is the function of Eq. 1 with abscissa x and RHO is the visually averaged virtual log distance ratio via Eq. 2 for y]). One can imagine that continuing the regrading behavior from any higher to lower PB_D case by the constant process of mapping PB_HDR to PB_HDR will produce a curve that becomes a steeply descending curve, in fact for the para kind of brightness mapping curve, which becomes mathematically abcara. In practice, the required function for extrapolating any received IDR image (based on the starting luminance mapping function F_H2S received in the metadata, by using the ETSI2 Chapter 7.3 display adaptation mechanism) F_E_I2S will be the mirroring function obtained by mirroring around the diagonal of F_ENCINV_H2I (and vice versa).

Therefore, given the desire to reuse the standard ETSI2 algorithm to implement the SLHDR2PLUS functionality, all that remains is to define the corresponding codec, as illustrated in Figure 18 .

For example, the SG of F_ENCINV_H2I is defined in abcara as 1/SG * L_in_X.

According to SG_COD (i.e., the ETSI defined encoding of the above physical mathematical shadow gain SG), we get (SG_COD=SGC*255/2 and ETSI1 equation C23 exposure=SGC/4 +0.5 and C24 expgain=v(PB_HDR=5000/PB_target=500; PB_target) and equation C27 SG=expgain*exposure): 1/[(SGC/4+0.5)*v(5000/500;500)]=(X/4+0.5)* v(500/5000;500) [Equation 24]

The unknown para shadow gain control X is to be solved (ie, X is the SG of F_ENCINV_H2I).

That is, the decoder defines for any grader's F_H2S choice how the F_E_I2S shape will look like (using the ETSI2 7.3 algorithm), but it needs to be interpreted as an ETSI1 abcara so that the abcara can be related to the corresponding required inverse para F_ENCINV_H2I to ultimately use the corresponding para in the new SLHDR2PLUS encoder to calculate the IDR image brightness (in the first best embodiment of this particular class of methods of the generic SLHDR2PLUS method, i.e., calculated using the derivative of the brightness mapping function of the second peak brightness; white and black offsets will be ignored in this class (at least in the HDR＜＞IDR sub-range) as they will be applied to the HDR＜＞SDR sub-range of different PB_C image spectra, as shown in Figure 7).

The encoder now operates in practice in the other order (but obeying the same relationships to keep the system ETSI2 compliant). Channel adapter 1801 calculates (from the received F50t1 function shape) the required para to convert L_HDR brightness to, for example, 500 nits PB_C L_IDR brightness (the channel adaptation mathematics of the previous embodiment described above can be used by just applying para, but then ignoring the WLO and BLO adaptations, i.e. para only operates between two 0 to 1.0 brightness representations without any offset involved). Inverter 1802 calculates the corresponding abcara using the inverse of equation 24 (i.e., with 1/X on the left calculated in view of the known SGC on the right side of the equation). This is a mapping that will reconstruct the L_HDR pixel brightness from the received L_IDR brightness. Assuming for example that WP is kept constant on the codec definition chain, the inverter 1802 will therefore calculate the shadow gain SG_abc and the high brightness gain HG_abc of abcara. The down-track for meta data management will eventually require the calculation of F_L_IDR (=F_I2S), so the adapter 1803 determines the required mapping function F_I2S (especially its SG_IDR and HG_IDR) by applying the algorithm of ETSI2 7.3 in the reverse direction (if partial brightness re-grading has been done for the IDR image brightness by using F_ENCINV_H2I, the residual transform F_I2S of the total transform F_H2S is achieved).

As already mentioned above, in certain scenarios it can happen that the HG_IDR value drops above what can be ETSI2 compliantly encoded as HG_COD. What can be done in this scenario is to limit the value of HG_IDR to its maximum value and chain back to the intended one, in particular to which the F_H2S function of a different raw classifier will correspond. All calculations can then be restarted from this situation and are performed in one continuous processing line with the optional units shown in dashed lines.

FIG22 illustrates what the limiter 1804 performs when the brightness mapping curve is reshaped. The starting F_H2S is shown as a dotted line, and how the F_ENCINV_H2I function can be derived therefrom using a fixed channel adaptation algorithm, and how the (original) residual re-scaling function F_I2S_or can be derived (assuming no additional specific restrictions in the current formulation of ETSI2, the original F_IDR calls the specific embodiment method described in more detail now explained). In view of this completely new approach to HDR video coding, the HG_IDR_or of this function may not fit into the HG_COD definition, i.e. requires values higher than its 8-bit maximum value of 255 that can be conveyed in an ETSI2 compliant HDR video coded signal. Therefore, HG_IDR_or must be reduced to at most the limited still codable value HG_IDR_LIM (which is 2.0 in the current implementation of ETSI2, but this is not a fundamental limitation of the method). This creates a para with a high-luminance linear segment somewhat closer to the upper horizontal margin (L_out_X=1.0), which corresponds to a brighter IDR picture, but no fundamental problem (as mentioned above, there is some slack possibility in the system to design various variations). It will mean that the highest brightness areas in the HDR scene imagery get a smaller contrast IDR representation (although the original master HDR is fully recoverable, and the SDR look and all MDR re-grades will look good too), but because it's graded from the higher PB_C HDR master imagery there's no real problem, and this corresponds to, for example, things in the 3000 to 5000 nit range, which are typically lamps and the like, which may suffer a bit of degradation (since some degradation mapping is always necessary anyway, and is somewhat expected for such super bright areas). The second channel adapter 1805 will then apply all of the above mathematics again, but now in the case of finite HG_IDR (so first the equivalent F_H2S can be calculated, as described, in this class of embodiments it can be performed by extrapolating the finite F_I2S_LIM to the case PB_D = PB_Mster_HDR, and then channel adaptation can be applied again).

This resulting F_H2I_LIM (i.e. mapping L_HDR luminances to L_IDR luminances) can now be applied by the image pixel brightness mapper 1806 to determine all IDR luminances (or in fact, all IDR YCbCr colors, also using the chrominance processing of ETSI2, i.e. the defined C_LUT corresponding to the shape of the F_H2I_LIM luminance mapping function) on a pixel-by-pixel basis. Finally the IDR metadata determiner 1807 calculates the complete metadata set for implementing an ETSI2 compliant metadata-based regrading to reduce the PB_C image (for any display PB_D) to below the PB_IDR to which it belongs (or by extrapolation above the PB_IDR). So, SG_IDR, HG_IDR, and WP_IDR are again determined according to any of the possible combinations forming the embodiments explained above. BLO_IDR and WLO_IDR are now also determined (as explained above, a specific brightness on the Master_HDR brightness axis can be mapped to 1.0 on the SDR brightness axis, and this can be reformulated as a mapping of appropriately scaled IDR brightness, i.e., WLO_IDR is defined, and similarly for BLO_IDR).

Finally, the customizable curve can be optimized for the new IDR metadata case by the customizable curve optimizer 1808 (if the customizable curve is used, since some sub-market codec implementation variations (e.g., real life broadcasting) may have chosen not to use the customizable curve at all, and then apply the former's para+offset math).

Fig. 19 illustrates how the adaptation of a customized curve works. It always consists of two conceptual components (regardless of whether it is applied directly in a single direction (or inversely)). The first component can be understood by focusing on the object: suppose that one of the control points of the following polylinear segment customizable curve corresponds to a pair of pants (so a particular L_in_S normalized brightness xo1I is, for example, the average brightness of all pants pixels). A transformation is used to classify the pants according to a human (or automated software), for example, to brighten the pants pixels (either around or specifically at one of the control points) to output a normalized brightness that is better for the pants. Also see in Figure 4 that in the ETSI approach this happens as the last (optional) fine level encoding step in the decoder (unit 405), and correspondingly the first step in the decoder. So in practice this brightness transform is actually defined in the SDR brightness domain (after a coarse HDR to SDR brightness mapping of para+offset (if any)).

Therefore, it follows that any brightness needs to be transformed (for that object!), which can be written multiplicatively as L_out=m(L_in_SDR)*L_in_SDR.

The required multiplicative brightness change (in percentage) may be different on any other image (e.g., an IDR image), but one thing that should be relied upon is that the correction of the fine grading corresponds to a specific "object" that needs to be re-graded (even if another benefit of using customizable curves for this, in addition to specific object fine grading, e.g., an improvement in the shape of the coarse-graded brightness mapping curve, can still be physically interpreted as this object-based improvement by defining a set of virtual objects corresponding to some brightness sub-range). So, if tracking such objects to another DR brightness range, the normalized x-coordinate values may change, but not the core nature of the object (e.g., a person on a motorcycle has a different normalized brightness in HDR (i.e., 5/5000) than in SDR (i.e., 5/100)). So, the function for the new normalized luminance position distribution must be recalculated (this can be done for any amount of re-scaling of the luminance mapping function in the middle, even no matter how complex the various upward and downward tracks one would want to design an HDR video encoding implementation). So, Figure 19a shows this roughly: the original SDR object luminance (e.g., the linear segment endpoint of the linear segment of the customizable curve) xo1I is moved to xo1N (this would happen by applying, for example, an abcara which is the inverse of F_I2S of Figure 20). The same happens to other points, e.g. the pentagonal segment points (which can generally be assumed to have a sufficiently good diffusion of segment points (e.g. 16), if the grader, for example, applies a roughly linear customized regrading to relatively large sub-areas of darker brightness, which can be set automatically by the grading software (e.g. 10)). So, having offset all these points, an intermediate curve CC_XRM can now be defined from the original CC_gr curve of the master content metadata grader (F_H2S with CC over the SDR luminance range) by applying the original CC_gr offset (i.e., L_out_SDR = CC_gr[L_in_S]), where the L_in_S values are the original values xo1I etc. (but now with the L_out values applied to the xo1N remapped IDR luminance positions (producing the imaginary curve). Of course, this will not be a proper HDR to IDR (or more accurately, IDR to IDR) mapping multiplier, so the correction is performed in step 2, as shown in Figure 19b.

As can be seen again in FIG. 19b , the multiplicative fine correction can be interpreted as a scalable process varying between no correction (by definition, the Mster_HDR pixel brightness is already correct since this image was optimally graded by the content creator to start with) to full correction for the PB_C image that differs most extremely in the spectrum of the regraded image (from Mster_HDR), which in the applicant's approach is typically a 100 nit SDR image (where the full correction for a particular pixel is, for example, mso1, which can be written as an absolute offset, but can also be written as a multiplicative correction yio1=mso1*xso1 (any brightness mapping curve shape yio1=F_L(xso1) can be formulated as a curve of brightness-dependent multiplicative values).

Since the multiplicative correction viewpoint can be formulated as an offset from the diagonal, where yio1=xso1, a vertical scale factor can be introduced: ScaleVer= max [(1-La)/(1-K); 0]         [Equation 25] Using La and K as defined above.

The required adjusted value of the customizable curve is then found to be: yiDA=Min[(yio1-xso1)*ScaleVer+xio1;1]         [Equation 26] and this is calculated for all values of xso1.

FIG. 27 gives another way to determine the segment endpoints of a customizable fine-gradation curve in a technically elegant manner for a decoder. It has been described how the parameters of the coarse-graded para curve can be recalculated (and if there are black and/or white offsets, but the explanation will be simplified by focusing on para). Assume that para is coarsely graded from any starting dynamic range to a final dynamic range (e.g., an LDR dynamic range). The black and white offsets can take into account the normalized range differences (if any are necessary), so the customizable curve is just about relocating the relative brightness of specific regions along the normalized axes. Therefore, the curve will start at (0,0) and end at (1,1), respectively, and in the example have some segment connector points between 2 curve shape decision points (e.g., (x1,y1)). It is also reasonable that the number of linear segments and points is equal in any representation and its regrading, because the nature of the regions does not change (the darkest region, e.g. indoor colors may end up at a different (generally perceptually uniform) normalized brightness in e.g. a 200 nit PB_C image than in a 1500 nit PB_C image), but the fact that there are two regions (indoor and outdoor) does not change in the regrading.

Therefore, for the reshape determination of the multilinear regrading function, it is only necessary to find the corresponding endpoints (xnew, ynew).

Another property to be met (ideally) can be exploited, namely that regardless of whether the master HDR image is regraded directly using the total stride function FL_50t1 (which in this case will consist of two consecutive functions to be applied: total para 2710 and total multilinear function 2711), or is regraded in two steps, first from a 5000 nit master image to a 700 nit IDR (again by using two functions: IDR generation para 2701 and IDR generation multilinear function 2702), and then downgraded from it to a 100 nit LDR image (using channel para 2703 and channel multilinear function 2704), the result must be the same: the same LDR image, because it is the LDR image always generated for the master HDR image, i.e. the one that the content creator has encoded and delivered (with the downgraded luminance mapping function shape). That is, choosing any of all possible input HDR normalized luminances x1_MH, the final LDR output luminance should be the same. Therefore, this will also be true for input luminances that map (via previous mapping) to the x-coordinate of the channel multilinear: x1_CH_L. This can be used to recalculate the line segments, since there is equality on the longitudinal coordinate y, and only x_new needs to be calculated for specific segments of the corresponding multilinear customized curve over the other dynamic range.

So on the encoding side, the channel-adapted Y_CHA can be calculated for any x1_MH input by applying the scaled normalization algorithm. This value Y_CHA will form the corresponding input x-coordinate of the next block, which enters channel PB_C after determination para (the equation given above). The yi_CH value is already known, since it is equal to the y1_L value for the total re-grading from 5000 nits to 100 nits, which is of course directly known on the encoding side (as opposed to the decoding side) (e.g. generated by a human grader). Doing this for all points of the multilinear function, we obtain all its characterization parameters to be written into the video signal (as part of F_I2sCI).

On the decoder side, the same principle can be used again to obtain a somewhat different algorithm, since now some unknown parameters must be calculated. So now the x1_ML values corresponding to the received and therefore known x1_CH_L values must be calculated, since the first step is to recover the total re-grading function(s). There is generally a numerical accuracy of the function, for example 256 quantized x values (i.e. not specific, for example, points between two or three segments, but all points, so also points on the line in between), so it is simple to construct the LUT table for all the points of the customizable curve when it is customized, i.e. y1_L of the known curve, corresponding to the required x1_ML for x1_CH_L.

Mapping from LDR to IDR brightness, for any yi_CH we get x1_CH, and this value can be demapped via para 2703. If we know para 2701 and multilinearity 2702, we can also determine which of all possible x1_MH values maps to this Y_CHA value. From above we know how to calculate para 2701 from the function metadata received on the decoder side, as explained above. Multilinearity 2702 is not (still) known, but it is not needed for now. Because we know that the customized curve 2702 also follows the vertical scaling equation of the standardized algorithm. Any tested X1_MH can be converted to a corresponding X_CHA, and the corresponding (and required) Y_CHA value follows: Y_CHA=(y1_L - x1_ML)*scaleVer + X_CHA, and x1_ML can be calculated from x1_MH by applying total para 2710.

Thus, exactly one corresponding to the x1_MH and x1_ML values, respectively, will be found which will recover the total multilinear function 2711. Since the total reclassification and the channel partial reclassifications are then known, the remaining reclassifications can also be determined (i.e., between 5000 nits main image and 700 nits IDR), so decoding everything, i.e. determining the function, and processing of all IDR image pixel colors can begin, as explained using FIG.

To ensure a better understanding of the reader, FIG. 26 again summarizes conceptually all that a reader with general knowledge can already find in the detailed explanation above. The upper track grid is about the meta-data recalculation, that is, the various steps of the determination of the various brightness mapping functions (the lower unit 2650 etc. is the one that performs the actual pixel color processing). Now we can see the two-step calculation, which corresponds to the standardized algorithm application in the HDR function unit 901 in the encoder (but now from the SLHDR2PLUS video decoder side) and the function determination of the IDR mapping function generator 903. As explained, the decoder gets a function F_I2sCI that specifies the luminance re-grading behavior between the received selected IDR image with its channel peak luminance PB_CH and a grading of 100 nits. But it needs to decide the larger span function (between 100 nits and the main HDR peak luminance of, for example, PB_C_H50=6000 nits) that the original function calculator 2601 will perform, namely the FL_50t1 function (or more precisely, the inverse shaping function of that used on the encoding side). But it is still not reached, and it is desired to decode the IDR normalized luminance (or more precisely, the perceptually uniform normalized pixel brightness in our general decoding topology) into the main HDR reconstructed luminance. Therefore, the originally received F_I2sCI, and function FL_50t1, cannot determine the re-gradation between the PB_C_H 50 nit main image and 100 nit (not the PB_CH nit IDR image or either of the other two), so it is necessary to determine the function F_IDRt50 applied to the received IDR pixel brightness to obtain the (perceptually uniform) reconstructed main HDR image pixel brightness YpMstr, which is what the reconstruction function determiner 2602 will do. The display adaptation possibility has been shown as a dashed display optimization function calculation unit 2603, because as mentioned, although it will generally also be implemented in our full-featured SLHDR2PLUS decoding IC, it is in principle optional for SLHDR2PLUS decoding. The channel peak brightness PB_CH will be used to convert the normally encoded (e.g., 10-bit YCbCr) IDR pixel brightness to the perceptually uniform IDR pixel brightness YpIDR, on which our reconstructed brightness mapping will generally be performed in our preferred SLHDR2PLUS IC (although one of ordinary skill will understand how the principles of the invention can be implemented in alternative circuits or software that do not apply perceptual equalization, or other methods, etc.). The perceptual equalizer 2650 applies Equations 1 and 2 to it with PB_C_H = PB_CH. The brightness upmapper 2651 reconstructs the main HDR image brightness by simply applying the determined function, i.e., YpMstr = F_IDRt50(YpIDR). If display adaptation is required to create e.g. a 350 nit PB_C image, the display optimizer 2652 simply applies the determined display optimization function to it, resulting in a display optimized pixel brightness: Yglim = F_DO(YpMstr). These can be converted to actual normalized pixel brightness L by a linearizer 2653, which applies the inverse equations 1 and 2, but now using the display optimized e.g. 350 nit PB_C_DO instead of PB_CH. Finally, there is typically optionally a further brightness code generator 2654, which applies the perceptual quantizer EOTF of SMPTE 2084 to give an output brightness YPQ in the popular HDR10 format.

While some embodiments/teachings are presented to illustrate some aspects that may vary individually or in combination, it is understood that several further variations may be formed along the same basic principle: re-derive the brightness mapping equation from different intermediate dynamic range image metadata received in accordance with ETSI2 HDR video communication or similar to reconstruct the master HDR image that has been optimally graded at the content creation site. The algorithm components disclosed herein may (in whole or in part) be implemented in practice as hardware (e.g., as part of a specific application IC), or as software running on a special digital signal processor or a general purpose processor, etc.

A person of ordinary skill in the art should be able to understand from our presentation that these components may be optional improvements and may be implemented in combination with other components, and how the (optional) steps of the method correspond to individual components of the apparatus, and vice versa. The word "apparatus" in this application is used in its broadest sense, i.e. a group of components that allows the implementation of a specific purpose, and may thus be, for example, an IC (a small circuit component), or a dedicated appliance (e.g., an appliance with a display), or a part of a network system, etc. "Arrangement" is also intended to be used in the broadest sense, so it may include, among other things, a single device, a part of a device, a collection of (parts of) cooperating devices, etc.

The computer program product designation should be understood to cover any physical implementation of a general or special purpose processor, a set of commands that, after a series of loading steps (which may include intermediate conversion steps, such as translation into an intermediate language and a final processor language) to input the commands into the processor, performs any of the characteristic functions of the invention. In particular, a computer program product may be implemented as data on a carrier (such as, for example, a disk or tape), data stored in a memory, data traveling via a network connection (wired or wireless), or program code on paper. In addition to program code, characteristic data required by the program may also be embodied as a computer program product.

Some steps required for the operation of the method may already exist within the functionality of the processor rather than being described in the computer program product, for example, data input and output steps.

It should be noted that the embodiments mentioned above illustrate rather than limit the present invention. While a person of ordinary skill can easily implement a mapping of the presented embodiments to other areas of the claimed scope, we do not mention all such options in depth for the sake of brevity. In addition to the combination of elements of the present invention combined in the claimed scope, other combinations of elements are possible. Any combination of elements can be implemented in a single dedicated element.

Any reference signs between parentheses in the claims are not intended to limit the claims. The word "comprising" does not exclude the presence of elements or aspects not listed in the claims. The word "a" or "an" before an element does not exclude the presence of a plurality of such elements. [Terms and abbreviations used]:

PB_C : Maximum encodable brightness of the image, usually indicated for any case, C stands for encoding (not to be confused with bit depth), e.g. an HDR image may have PB_C_HDR = 4000 nits (which also defines all relative brightness below, since L_norm = L/PB_C, where L_norm (normalized brightness) lies between 0.0 and 1.0

PB_D : The maximum displayable brightness of any display (also called, peak brightness), e.g., current HDR displays typically have a PB_D of 1000 nits (but values down to 600 nits or up to 2000 nits and even 4000 nits are currently commercially available, and higher PB_Ds are possible in the future).

IDR (Intermediate Dynamic Range): A mechanism by which an image originally defined with PB_C1 (e.g., 10,000 nits) (i.e., primary image) is actually represented as a secondary HDR image with PB_C2 < PB_C1 (e.g., typically a factor of 2 or less, and PB_C2 is typically ＞= 500 nits).

MDR (Medium Dynamic Range; must not be confused with IDR): images with PB_C_MDR are generally between the PB_C (PB_C_H) of the received HDR image and PB_C_SDR=100 nits (by convention in the video field), setting this PB_C_MDR value equal to the PB_D of any display (in this way, incoming HDR images with incorrect dynamic range, and thus more importantly with incorrect relative statistical distribution of normalized luminance with respect to each other, can be optimally regraded for a particular available display of lower dynamic range capability, i.e. PB_D < PB_C_H)

Para : A specific highly programmatically useful function to map brightness defined over a first normalized brightness range corresponding to PB_C1 to normalized by PB_C2, and which is defined above by Equations 4 and 5 and the segment of the parabola therebetween, or formally by ETSI TS 103 433-1 V1.2.1 (2017-08) [ETSI1 in Brief] p. 70 Equation C-20.

Abcara : For any para function (i.e., one with parameters that uniquely define its shape), the inverse shape can also be found intuitively (but sometimes needs to be calculated mathematically) by swapping the axes.

WLO (white level offset): The normalized luminance in the normalized luminance range of the first image (im1) mapped to 1.0 on the second normalized luminance range, such that PB_C_im1>PB_C_im2. In this application, there are several different WLOs defined along the encoding process for various images with different PB_Cs, which are given suffixes such as WLO_gr so they can be easily distinguished.

BLO (black level offset): the normalized luminance in the normalized luminance range of the first image mapped to 0.0 on the second normalized luminance range, such that PB_C_im1>PB_C_im2. In this application, there are several different BLOs defined for various images with different PB_Cs along the encoding process, which are given suffixes such as BLO_IDR so they can be easily distinguished.

P_LUT : The total mapping (including partial rescaling in our codec approach as explained in Figure 4) needed to transform any possible normalized luminance of the first image into the corresponding normalized luminance of the second image, whereby PB_C_im1 != PB_C_im2 (typically differing by at least a multiplication factor of 1.02). Because the P_LUT[L] (which is typically image content dependent and optimized e.g. automatically by intelligent image analysis or by humans) changes the relative distribution of normalized luminances, i.e. the histogram, which is a key aspect of dynamic range conversion, e.g. involved in the definition of IDR images which is key in the present novel HDR codec principles.

C_LUT : Pixel brightness dependent mapping of the chromaticity of the pixel color, together with P_LUT to complete the color transformation (YCbCr_out=T[Y_cbCr_in])

201: Camera 202: OOTF Mapping 203: Classification Unit 204: Image Communication Connection 205: Display 206: User Interface Control Unit 301: Image Source 302: Color Converter 303: Video Compressor 304: Formatter 305: Communication Media 306: Unformatter 307: Video Decompressor 308: Color Converter 309: Display Adapter 310: Display 320: Video Receiver 321: Broadcast station side encoder 401: EOTF application unit/brightness processor/brightness processing track/brightness processing track 402: unit/brightness mapping 403: black-white level shifter/unit/brightness mapping 404: coarse dynamic range converter/unit 405: curve applicator/unit 406: linearizer/unit 450: chroma processing track 451: saturation processing determiner/unit 452: multiplier 453: matrix applicator Adding unit 454: First multiplier 455: Multiplier 601: Shadow man 602: Robot 900: Encoder 901: HDR function generation unit/unit 902: IDR image calculation unit 903: IDR mapping function generator 920: Image input 921: Second meta-data input 922: First meta-data input 923: Third meta-data input 930: Image output 931: Second meta-data output 9 32: First meta-data output 933: Third meta-data output 1100: High dynamic range video decoder/decoder 1102: Color converter 1104: Brightness function determination unit 1110: Image input 1111: First meta-data input 1112: Second meta-data input 1113: Third meta-data input 1120: Image output 1121: Meta-data output 1122: Image output 1501: Brightness mapper / Luminance Processing Track 1502: Sensor/Decoder/Unit 1503: Fine Grading Unit/Unit 1504: Coarse Brightness Mapping Unit/Unit 1505: Anti-Black and White Shifter/Unit 1506: Linearizer/Unit 1550: Chroma Processor 1551: Chroma Processing Decision Unit/Unit 1601: Linearizer 1602: Sensor 1603: Brightness Mapping Unit 1604: HDR to MDR Brightness Mapping Transmitter 1801: Channel adapter 1802: Inverter 1803: Adapter 1804: Limiter 1805: Second channel adapter 1806: Image pixel brightness mapper 1807: IDR metadata determiner 1808: Customizable curve optimizer 2301: Content creation side 2302: Camera 2303: Grading equipment 2304: Grader 2305: Video encoder 2306: Satellite antenna 23 40: Communication satellite 2351: Local satellite receiving dish 2352: Final processing equipment 2353: HDR display 2370: Video encoder 2381: Decoder 2382: Display optimizer 2501: HDR image signal encoded by SLHDR2PLUS 2502: Pixel color matrix 2510: SLHDR2 decoder 2520: SLHDR2PLUS decoder 2601: Raw function calculator 2 602: Reconstruction function determiner 2603: Display optimization function calculation unit 2650: Perceptual equalizer/unit 2651: Brightness mapper 2652: Display optimizer 2653: Linearizer 2654: Brightness code generator 2701: para 2702: IDR generates multilinear function/multilinear/curve 2703: para 2704: Channel multilinear function 2710: Total para 271 1: Total multilinear function BE: Main HDR image luminance range Be2: IDR luminance range BLO_gr: Black level offset BLO_IDR: Parameter BLO_REC: Offset BN: Normalized luminance Bs: Color component Cb: Chroma Cb*: Output chroma CC_XRM: Midrange curve ColHDR: Brightness ColSDR: Brightness Cr: Chroma Cr*: Output chroma DR_1: LDR luminance =DYNAMIC RANGE/1ST DYNAMIC RANGE DR_2:HDR BRIGHTNESS RANGE/2ND DYNAMIC RANGE DRKSPST:Dark Space Station dX:Horizontal Axis EXP:Exposure F_??:Function F_C:Coarse BRIGHTNESS CRUSHING FUNCTION/Coarse Mapping Function F_ct:Color Conversion Function/Color Mapping Function F_E_I2S:Brightness Mapping Function F_ENCINV_H2I:Decoded Brightness Mapping Function F_H2h:Brightness Mapping Function/Upward Grading Function F_H2hCI: Adapted brightness mapping function/brightness mapping function F_H2hCI_1: Output F_H2hCI_2: Output F_H2hCI_3: Output F_H2I_LIM: Brightness mapping function shape F_H2S: Total brightness mapping function/Starting brightness mapping function F_I2s: Function F_I2S: Mapping function F_I2S_or: Residual re-grading function F_I2sCI: Channel brightness Mapping function/brightness mapping function/luminance mapping function F_IDRt50: function F_L_CU: fine-graded luminance mapping function/function/curve F_L_DA: display adaptation function/luminance mapping function F_L_REC: MasterHDR reconstruction luminance mapping function F_L_RHDR: reconstruction para luminance mapping function shape F_L_subsq: luminance mapping function F_Mt1: function/total luminance remapping function F _Mt1_ca: Intermediate function F50t1: Function shape Fd: Distance FL_50t1: Main brightness mapping function/Total span function/Brightness degradation function FL_ca: Curve FL_gr: Curve FL_IDR: Curve Fx: Function GN: Normalized brightness Gs: Color component HG: Slope HG_abc: High brightness gain HG_ca: High brightness gain HG_gr: Second slope/Original high brightness gain H G_IDR: parameter HG_IDR_LIM: encodable value HG_REC: reconstructed high brightness gain IDR: intermediate dynamic range Im1250: display-adapted image Im5000: output image Im_COD: compressed SDR image Im_LDR: output image/low dynamic range image/standard range image Im_RHDR: reconstructed HDR image/original main image Im_RLDR: SDR image ImDA :PB_C image ImSCN1: Clear outdoor image ImSCN2: Night scene ImSCN3: Scene image K: Full action of the function L_HDR: Luminance L_IDR: Brightness L_in: Input/parameter/Normalized luminance L_in_X: Normalized input luminance L_Mster_HDR: Brightness L_REC_M_HDR: Brightness L_RHDR: Normalized reconstructed luminance L_SD R: Luminance Lh: Output Luminance LIN_HDR: Linear Luminance Image Ln_Mster_HDR: Input Luminance Ln_XDR: Luminance Ls: SDR Luminance/Luminance MAST_HDR: Master HDR Image MB: Minimum Black Level/Lower Endpoint MDR: Medium Dynamic Range MDR_300: Medium Dynamic Range Image mip: Midpoint mso1: Absolute Offset Mster_HDR: Image Mster HDR: Image MsterHDR: Master HDR video/Input high dynamic range image/Master high dynamic range image mx: Horizontal coordinate n: Distance OOTF: Optical-Optical Method P_IoH: Brightness P_SoH: Brightness PB_C: Image coding peak brightness/Maximum encodable brightness PB_C_H: Peak brightness PB_C_H50: First maximum pixel brightness/Second expected peak brightness/Content peak brightness PB_CH: Channel peak brightness/Second maximum Pixel brightness PB_MDR: Maximum pixel brightness PQ: Perceptual quantizer REC_M_HDR: Image/Reconstructed master HDR image RecHDR: HDR image RN: Normalized brightness Rs: Color component RS: SDR brightness subrange sc*: Normalized scale factor SDR: Standard dynamic range SG: Shadow gain SG_abc: Shadow gain SG_ca: Shadow gain SG_gr: First slope SG_IDR: IDR image/parameters SG_REC:Shadow gain SG_RM:Curve/total shadow gain WLO_ca:parameters WLO_gr:White level offset WLO_IDR:parameters WLO_REC:Offset WP_IDR:parameters xo1I:Brightness Y:Brightness Y’CL:Output brightness Y’GL:Graded LDR brightness Y’HC:Original HDR brightness Y’HP:Gray brightness Y’HPR:Reconstructed Master_HDR brightness Y’HPS: scaled HDR brightness/normalized HDR input brightness Y’IP: IDR brightness Y’IPG: regraded IDR brightness Y_IDR: brightness value Yglim: display optimized pixel brightness yio1: brightness mapping curve shape YpIDR: perceptually uniform IDR pixel brightness YpMstr: reconstructed master HDR image pixel brightness YPQ: output brightness

These and other aspects of the method and apparatus according to the present invention will become apparent with reference to the embodiments and examples described below and with reference to the accompanying drawings, which are intended only as non-limiting, specific illustrations of the more general concepts, and in which dashed lines are used to indicate that components are optional, and non-dashed line components are not necessarily required. Dashed lines may also be used to indicate elements that are interpreted as being required, but are hidden in the interior of an object, or used for intangible things, such as, for example, the selection of objects/areas (and how they can be displayed on a display).

In the diagram: [Figure 1] schematically illustrates many of the common color transformations that occur when optimally mapping a high dynamic range image to a corresponding optimally color graded and similar looking (as similar as desired and feasible given the differences in the respective first dynamic range DR_1 and second dynamic range DR_2) lower dynamic range image (e.g., a standard dynamic range image of 100 nits peak brightness), which in the case of reversibility (Mode 2) will also correspond to the color transformations received by the receiver (decoder) [Figure 2] shows what a capture system for HDR images may look like; [Figure 3] illustrates possible ways of conveying HDR images as some images of specific (different) peak brightness and a brightness mapping function generally defined as a brightness mapping function (e.g., defined as a conventionally usable SDR image Im_LDR) commonly conveyed in the metadata, such as according to the applicant's preferred method, with high concept [Figure 4] shows various further details of encoding HDR images according to the applicant's specific method standardized in ETSI2, which are required to understand the various details of the various teachings of the novel SLHDR2PLUS codec method written below; [Figure 5] illustrates how color transformations and in particular brightness transformations in the YCbCr color space are performed by re-grading; [Figure 6] explains some useful applications by explaining useful applications Explaining our concept of customizable curves in more detail; [Figure 7] explains the basic idea in the encoding and communication of the intermediate dynamic range (IDR) of the original master HDR image, and the unmistakable concept of the medium dynamic range image (MDR), which is generally calculated from any received image to optimize it for display on any specific display with available display peak brightness PB_D, such as a specific HDR purchased by any end consumer who intends to view the received HDR video. tv; [FIG. 8] further illustrates how to start dealing with the IDR problem, in particular by solving it in a specific automatically computable way by the decoder, and in particular if at least some of the decoders that may receive the content are ETSI2 compliant decoders that are already on the market and may not be easily upgraded with the new SLHDR2PLUS technology (for example, because the owner of the tv or STB does not upgrade it); [FIG. 9] shows the general structure of the technical components generally required for the novel SLHDR2PLUS encoder of this application; [FIG. 10] illustrates some basic technical aspects involved in the continuous derivation of various corresponding brightness mapping functions by the encoder, in particular using the example of the para brightness mapping function; [FIG. 11] shows a possible typical high-level structure of the novel SLHDR2PLUS decoder, which follows some embodiments of the various possibilities described below Instructions to implement SLHDR2PLUS video communication; [FIG. 12] Further explanation of some aspects of black and white offsets when selected by the content creator from his point of view in his master brightness mapping function that defines his video content re-grading needs; [FIG. 13] Describing the technical principle of the preferred method for deriving a channel-adapted version of para based on the principle of diagonal scaling; [FIG. 14] Illustrate some aspects of the inverse curve of para (the so-called abcara); [FIG. 15] Detailed description of some aspects of the SLHDR2PLUS decoder; [FIG. 16] Illustrate a useful way to implement display adaptation to calculate MDR images for any specific PB_D display integrated with the new technology of the SLHDR2PLUS encoding principle of this application; [FIG. 17] Illustrate black and white offsets (BLO & WLO) to accompany and simplify the following mathematics and give some further aspects of the general physical technical basic principles; [Figure 18] shows another embodiment of SLHDR2PLUS encoding (or actually several teachings of various embodiments combined in one diagram), which is particularly useful because the encoded image can be directly decoded by a standard deployed ETSI2 decoder; [Figure 19] illustrates how to determine the corresponding version of the original master customized curve for various dependent peak brightness image representations (whether as input or output images), such as a common standard IDR image with a customized curve, to fine-tune the coarse mapped IDR brightness to a fine tuned according to the needs of, for example, a film creator's grader =The correct final SDR brightness; [Figure 20] shows the basic technical principle of the method of Figure 18 on the spectrum of the regradable image; [Figure 21] illustrates the extrapolation of the adaptation of the brightness mapping function (e.g., para) beyond the highest value of the starting image (corresponding to the uniform transformation or the diagonal line in the diagram), and also illustrates the relationship between the shape of the para function and the corresponding abcara for a particular choice; [Figure 22] schematically illustrates how a specific embodiment of the limitation of the high brightness gain of para works on the brightness regradation between the input normalized brightness and the output normalized brightness; [Figure 23] schematically shows the total HDR communication and chain to clearly indicate the difference between video encoding and display optimization (the former is about the actual definition of the image itself, while the latter is only about optimizing for any specific display of a specific dynamic range capability, loosely speaking customizable, preferably taking into account the re-grading needs of various HDR scene images that may be quite different and quite challenging); [Figure 24] further illustrates how the normalized function change algorithm will operate, which will take any input function shape generated by, for example, a content creator's grader to specify how the normalized luminance of a first peak luminance dynamic range should be mapped to the normalized luminance of a second peak luminance image representation, and how from this input function can be optimized for any given input The input function specifically calculates the function mapping between different particularly low third peak brightness representations and the second one; [FIG. 25] schematically illustrates how the currently taught SLHDR2PLUS video coding mechanism and signal are well compatible with traditional SLHDR2 decoders, but still provide additional possibilities for deployed SLHDR2PLUS decoders; [FIG. 26] schematically summarizes all the following decoding mathematics in a block to recover all the required brightness mapping functions for all the required decoding brightness processing to obtain the desired image from the received IDR image; and [FIG. 27] schematically illustrates a good practical method to numerically specifically obtain a customized version of the customizable curve (if any).

900: Encoder

901: HDR function generation unit/unit

902: IDR image calculation unit

903:IDR mapping function generator

920: Image input

921: Second metadata input

922: First meta-data input

923: Third meta-data input

930: Image output

931: Second metadata output

932: First metadata output

933: Third metadata output

F_H2hCI: Adaptive brightness mapping function

F_I2sCI: Channel brightness mapping function

FL_50t1: Main brightness mapping function

IDR: Intermediate Dynamic Range

MsterHDR: Master HDR video/input high dynamic range image

PB_C_H50: First maximum pixel brightness

PB_CH: Channel peak brightness/second largest pixel brightness

Claims (10)

A high dynamic range video encoder (900) is configured to receive an input high dynamic range image (MsterHDR) via an image input (920), the input high dynamic range image having a first maximum pixel brightness (PB_C_H50), the encoder having a first meta-data input (922) for the first maximum pixel brightness, and the high dynamic range video encoder is configured to receive a master brightness mapping function (FL_50t1) via a second meta-data input (921), the brightness The encoder further comprises a third meta data input (923) for receiving a second maximum pixel brightness (PB_CH) lower than the first maximum pixel brightness (PB_C_H50), and the encoder further comprises:- an HDR function generation unit (901) configured to apply a normalized algorithm to transform the master brightness mapping function (FL_50t1) into an adapted brightness mapping function (F_H2hCI), the adapted brightness mapping function causing the normalized brightness of the input high dynamic range image to be related to the normalized brightness of an intermediate dynamic range image (IDR), the intermediate dynamic range image being characterized by having a maximum possible brightness equal to the second maximum pixel brightness (PB_CH); - an IDR image calculation unit (902) configured to apply the adapted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR) which is the output of this unit; and - an IDR mapping function generator (903) configured to derive a channel brightness mapping function (F_I2sCI) based on the master brightness mapping function (FL_50t1) and the adapted brightness mapping function (F_H2hCI), the channel brightness mapping function defining as output the respective normalized brightnesses of the low dynamic range image (Im_LDR) when the respective normalized brightnesses of the intermediate dynamic range image (IDR) are given as input, the respective brightnesses in turn corresponding to the respective brightnesses of the input high dynamic range image (MsterHDR); the encoder is further characterized by having: - an image output (930) for outputting the intermediate dynamic range image (IDR); - - a first meta-data output (932) to output the second maximum pixel brightness (PB_CH); - a second meta-data output (931) to output the channel brightness mapping function (F_I2sCI); and - a third meta-data output (933) to output the first maximum pixel brightness (PB_C_H50). A high dynamic range video encoder (900) as claimed in claim 1, wherein the normalized algorithm of the HDR function generation unit (901) applies a compression towards the diagonal of the main brightness mapping function (FL_50t1) to obtain the adapted brightness mapping function (F_H2hCI), the compression involving scaling all output brightness values of the function by a scaling factor, the scaling factor being dependent on the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH). A high dynamic range video encoder (900) as claimed in claim 1 or 2, comprising: a limiter (1804) configured to re-determine a slope of the channel brightness mapping function (F_I2sCI) for a sub-range of the normalized brightnesses including the brightest normalized brightness equal to 1.0. A high dynamic range video decoder (1100) has an image input (1110) for receiving an intermediate dynamic range image (IDR), the intermediate dynamic range image having a second maximum pixel brightness (PB_CH) that is less than a first maximum pixel brightness (PB_C_H50) of a main high dynamic range image (MsterHDR) by a multiplication factor preferably 0.8 or less, the second maximum pixel brightness (PB_CH) being received via a second meta-data input (1112), the The decoder has a first meta data input (1111) for receiving a brightness mapping function (F_I2sCI), the brightness mapping function defining the transformation of all possible normalized brightness of the intermediate dynamic range image (IDR) to the corresponding normalized brightness of an LDR maximum pixel brightness low dynamic range image (Im_LDR), the decoder is characterized in that it has a third meta data input (1113) for receiving the first maximum pixel brightness (PB_C_H50), and the decoder comprises:- a brightness function determination unit (1104) configured to apply a normalized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I), the decoded brightness mapping function specifying as output a corresponding normalized HDR brightness of the master high dynamic range image (MsterHDR) for any possible input normalized brightness of a pixel of the intermediate dynamic range image (IDR), the normalized algorithm using the values of the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH); and - A color converter (1102) configured to continuously apply the decoded brightness mapping function (F_ENCINV_H2I) to the input normalized brightness of the intermediate dynamic range image (IDR) to obtain the normalized reconstructed brightness (L_RHDR) of the pixels of a reconstructed master HDR image (REC_M_HDR); the decoder further has an image output (1120) to output the reconstructed master HDR image (REC_M_HDR). A high dynamic range video decoder (1100) as claimed in claim 4, wherein the standardized algorithm of the brightness function determination unit (1104) calculates a scaling factor that depends on the first maximum pixel brightness (PB_C_H50) and the second maximum pixel brightness (PB_CH). A high dynamic range video decoder (1100) as claimed in claim 4 or 5, wherein the brightness mapping function (F_I2sCI) is defined by a brightness map, the brightness map consisting of a first linear segment having a first slope (SG_gr) for a dark normalized brightness range, a second linear segment having a second slope (HG_gr) for a bright normalized brightness range, and a parabolic segment for brightness between the two ranges. A high dynamic range video decoder (1100) as claimed in claim 4 or 5, wherein the color converter (1102) is configured to calculate the pixel brightness of a medium dynamic range image (MDR_300) having a maximum pixel brightness (PB_MDR), the maximum pixel brightness is not equal to the values of the LDR maximum brightness, the first maximum pixel brightness (PB_C_H50), and the second maximum pixel brightness (PB_CH), and the decoder has an image output (1122) for outputting the medium dynamic range image (MDR_300). A high dynamic range video decoder (1100) as claimed in claim 4 or 5, having a meta data output (1121) for outputting a brightness mapping function (F_L_subsq), which defines, for all normalized brightnesses of the reconstructed main HDR image (REC_M_HDR) or alternatively a medium dynamic range image (MDR_300), a corresponding brightness of an image having another maximum pixel brightness, the other maximum pixel brightness being preferably 100 nits, or a value higher or lower than the maximum brightness value of the respective reconstructed main HDR image (REC_M_HDR) or alternatively the medium dynamic range image (MDR_300). A method for high dynamic range video encoding of a received input high dynamic range image (MsterHDR) having a first maximum pixel brightness (PB_C_H50), comprising receiving a master brightness mapping function (FL_50t1), the brightness mapping function defining a relationship between the normalized brightness of the input high dynamic range image and the normalized brightness of a corresponding low dynamic range image (Im_LDR), the low dynamic range image having an LDR maximum pixel brightness preferably having a value equal to 100 nits, the method being characterized in that the encoding further comprises receiving a second maximum pixel brightness (PB_CH), and the encoding comprises:- applying a normalized algorithm to transform the master brightness mapping function (FL_50t1) into an adapted brightness mapping function (F_H2hCI), the adapted brightness mapping function relating the normalized brightness of the input high dynamic range image to the normalized brightness of an intermediate dynamic range image (IDR), the intermediate dynamic range image being characterized by having a maximum possible brightness equal to the second maximum pixel brightness (PB_CH); - applying the adapted brightness mapping function (F_H2hCI) to the brightness of the pixels of the input high dynamic range image (MsterHDR) to obtain the brightness of the pixels of the intermediate dynamic range image (IDR); - A channel brightness mapping function (F_I2sCI) is derived on the basis of the master brightness mapping function (FL_50t1) and the adapted brightness mapping function (F_H2hCI), which defines as output the respective normalized brightnesses of the low dynamic range image (Im_LDR) when the respective normalized brightnesses of the intermediate dynamic range image (IDR) are given as input, which brightnesses in turn correspond to the respective brightnesses of the input high dynamic range image (MsterHDR); - output the intermediate dynamic range image (IDR); and - output the second maximum pixel brightness (PB_CH), the channel brightness mapping function (F_I2sCI) and the first maximum pixel brightness (PB_C_H50). A method for decoding high dynamic range video upon receiving an intermediate dynamic range image (IDR), the image having a second maximum pixel brightness (PB_CH) lower than a first maximum pixel brightness (PB_C_H50) of a master high dynamic range image (MsterHDR) by a multiplication factor preferably 0.8 or less, the second maximum pixel brightness (PB_CH) being received as metadata of the intermediate dynamic range image, the decoding method Also receiving a brightness mapping function (F_I2sCI) in the metadata, the brightness mapping function defines the transformation of all possible normalized brightnesses of the intermediate dynamic range image (IDR) to the corresponding normalized brightness of an LDR maximum pixel brightness low dynamic range image (Im_LDR), and the decoding method is characterized in that it receives the first maximum pixel brightness (PB_C_H50), and the decoding method is characterized in that it comprises:- applying a normalized algorithm to transform the brightness mapping function (F_I2sCI) into a decoded brightness mapping function (F_ENCINV_H2I), the decoded brightness mapping function specifying as output a corresponding normalized HDR brightness of the master high dynamic range image (MsterHDR) for any possible input normalized brightness of a pixel of the intermediate dynamic range image (IDR), the normalized algorithm using the values of the first maximum pixel luminance (PB_C_H50) and the second maximum pixel luminance (PB_CH); - applying the decoded brightness mapping function (F_ENCINV_H2I) to the normalized brightness of the intermediate dynamic range image (IDR) to obtain a normalized reconstructed brightness (L_RHDR) of a pixel of a reconstructed master HDR image (REC_M_HDR); and - Output the reconstructed master HDR image (REC_M_HDR).