WO2013061599A1

WO2013061599A1 - A method of processing image data for an image display panel

Info

Publication number: WO2013061599A1
Application number: PCT/JP2012/006863
Authority: WO
Inventors: Benjamin John Broughton; Masahiro Esashi; Kenji Maeda; Tatsuo Watanabe
Original assignee: Sharp Corp
Current assignee: Sharp Corp
Priority date: 2011-10-28
Filing date: 2012-10-25
Publication date: 2013-05-02
Anticipated expiration: 2014-04-28
Also published as: GB2496114A; JP5756883B2; TWI478141B; GB201118684D0; CN103765499B; CN103765499A; TW201327541A; JP2014525595A

Abstract

A method of processing image data for an image display panel comprises, in a first mode: determining signal voltages to be applied to pixels of the image display panel from received image data constituting an image for display on the image display panel and from a secondary data value for the pixel thereby to generate luminance variations perceivable at a first viewing position (5) but substantially not perceivable at a second viewing position (3). The signal voltages to be applied to pixels in a group of pixels are determined such that the overall luminance of the pixels in the group is dependent on (for example, proportional to) the overall luminance specified for the pixels in the group by the image data. The number of pixels in a group is locally variable and is selected according to the image data and/or the secondary data values for pixels in a region of the image where the group is to be defined. The invention makes it possible to specify luminance redistribution over an increased pixel group size in local regions in which the main image content is relatively uniform, with no high spatial resolution image features, while specifying luminance redistribution over a reduced pixel group size for main image regions with sharp edges and other high resolution features.

Description

A METHOD OF PROCESSING IMAGE DATA FOR AN IMAGE DISPLAY PANEL

The present invention relates to a display device, such as an active matrix liquid crystal display device, which is switchable between a public and private display mode.

Several types of display which switchable between a public and private display mode, with varying degrees of additional cost over a standard display, ease of use and strength of privacy performance are known.

Devices incorporating such displays include mobile phones, Personal Digital Assistants (PDAs), laptop computers, desktop monitors, Automatic Teller Machines (ATMs) and Electronic Point of Sale (EPOS) equipment. Such devices can also be beneficial in situations where it is distracting and therefore unsafe for certain viewers (for example drivers or those operating heavy machinery) to be able to see certain images at certain times, for example an in car television screen while the car is in motion.

Several methods exist for adding a light controlling apparatus to a naturally wide-viewing range display, such as a microlouvre film (USRE27617 (F.O. Olsen; 3M 1973), US4766023 (S.-L. Lu, 3M 1988), and US4764410 (R.F. Grzywinski; 3M 1988)). However, this and other methods involving detachable optical arrangements are not conveniently switchable, requiring as they do manual placement and removal of the film or other apparatus to change the display from the public to the private mode.

Methods of providing an electronically switchable privacy function are disclosed in GB2413394 (Sharp), WO06132384A1 (Sharp, 2005) and GB2439961 (Sharp). In these inventions, a switchable privacy device is constructed by adding one or more extra liquid crystal layers and polarisers to a display panel. The intrinsic viewing angle dependence of these extra elements can be changed by switching the liquid crystal electrically in the well-known way. Devices utilising this technology include the Sharp Sh851i and Sh902i mobile phones. These methods share the disadvantages that the additional optical components add thickness and cost to the display.

Methods to control the viewing angle properties of an LCD by switching the single liquid crystal layer of the display between two different configurations, both of which are capable of displaying a high quality image to the on-axis viewer are described in US20070040780A1 and GB 0721255.8. These devices provide the switchable privacy function without the need for added display thickness, but require complex pixel electrode designs and other manufacturing modifications to a standard display.

One example of a display device with privacy mode capability with no added display hardware complexity is the Sharp Sh702iS mobile phone. This uses a manipulation of the image data displayed on the phone's LCD, in conjunction with the angular data-luminance properties inherent to the liquid crystal mode used in the display, to produce a private mode in which the displayed information is unintelligible to viewers observing the display from an off-centre position. A key advantage of this type of method is that in the public mode, the display consists of, and operates as, a standard display, with no image quality degradation causes by the private mode capability. However, when in the private mode, the quality of the image displayed to the legitimate, on-axis viewer is severely degraded.

Improved schemes which, when in the private mode manipulate the image data in a manner dependent on a second, masking, image, and therefore cause that masking image to be perceived by the off-axis viewer when the modified image is displayed, are given in GB2428152A1, WO2009110128A1, WO201134209 and WO201134208. These methods provide an electronically switchable public/private display with no additional optical elements required, minimal additional cost, and satisfactory privacy performance. These methods all utilise the limited resolution of the human visual system by redistributing the luminance produced to the on-axis viewer by a group of neighbouring pixels within that group while maintaining the same overall luminance produced by the group as a whole. The principle of operation of WO2009/110128 is illustrated in Figure 5, which is taken from WO2009/110128. The voltage applied to a sub-pixel in the private mode is determined by an LUT which receives as inputs the main image data, the side image data, and a spatial "flag" parameter. The spatially dependent flag parameter may be a value indicating which of two or more groups the pixel is deemed to be in based on its spatial position. For example, pixels in odd numbered columns in the image array may be said to form one group and pixels in even columns another. The groups could also constitute odd and even pixel rows, or perhaps the two parts of a chequerboard arrangement of the pixel array, etc. The mapping between the main image data values, the side image data values, and a spatial "flag" parameter is fixed across the display, so that a particular main image data value, a particular side image data value, and a particular value of the flag parameter will always produce the same sub-pixel voltage, regardless of the position of the pixel in the image.

In both WO201134209 and WO201134208, it is described how increasing the size of the group of pixels within which luminance is redistributed increases the maximum contrast of the masking image seen by the off-axis viewer. It is however also described how such in increased size of pixel groups reduces the effective spatial resolution of the main image observed by the on-axis viewer. Essentially, a trade-off exists between the strength of the privacy effect and resolution loss in the main image. WO201134208 describes how this resolution trade-off may be mitigated by careful selection of the spatial parameter which determines which pixels in a group are made brighter or darker, or by temporal inversion of the pattern of the spatial parameter, to cause the eye to average the luminance produced by individual pixels in the group over a number of frames. However, while the perceived resolution loss may be reduced by these methods, there remains an increase in this perceived loss when the luminance averaging is performed over groups consisting of an increased number of pixels.

It is therefore desirable to provide a high quality LCD display which has public and private mode capability, in which no modification to the LC layer or pixel electrode geometry is required from a standard display, has a substantially unaltered display performance (brightness, contrast resolution etc) in the public mode, and in the private mode has a strong privacy effect with minimal degradation to the on-axis image quality, particularly with regard to resolution loss incurred in the private mode.

USRE27617 US4766023 US4764410 GB2413394 WO06132384A1 GB2439961 US20070040780A1 GB 0721255.8 GB2428152A1 WO2009110128A1 WO201134209 WO201134208

A first aspect of the invention provides a method of processing image data for an image display panel, the method comprising, in a first mode: determining signal voltages to be applied to pixels of the image display panel from received image data constituting an image for display on the image display panel and from a secondary data value for the pixel thereby to generate luminance variations perceivable at a first viewing position but substantially not perceivable at a second viewing position; wherein the signal voltages to be applied to pixels in a group of pixels are determined such that the overall luminance of the pixels in the group is dependent on the overall luminance specified for the pixels in the group by the image data; and wherein the number of pixels in a group is locally variable and is selected according to the image data and/or the secondary data values for pixels in a region of the image where the group is to be defined.

As an example, the overall luminance of the pixels in the group may be proportional to the overall luminance specified for the pixels in the group by the image data, although the invention is not limited to this.

This mode of operation of the display device provides a private (narrow-view) display mode. The luminance variations generated as a result of the secondary data values serve to obscure the image that would be generated if the received image data were the sole input, so that a viewer at the first viewing position (for example position 5 in figure 2) which is outside the intended viewing range in the private mode (the narrow viewing range 6 in figure 2) cannot make out the image, or can only see a degraded version of the image, owing to the superposed luminance variations. A viewer at the second viewing position (for example position 3 in figure 2) which is inside the intended viewing range in the private mode perceives little or no intensity variations, and so sees the original image (that is, the image that would be generated if the received image data were the sole input) with little or no degradation in image quality.

Furthermore, the present invention extends the image processing methods of GB2428152A1, WO2009110128A1, WO201134209 and WO201134208 by providing an additional processing step which adaptively and locally determines the size of the groups of pixels within which luminance is redistributed, according to the input image content and/or the side image content. That is, before signal voltages are determined the invention provides the step of defining groups of pixels in the in the pixels of the image display panel, and the signal voltages are determined to provide a redistribution of luminance between pixels in a group (compared to the pixel luminances that would be generated by signal voltages determined from the received image data only). This additional step therefore makes it possible to, for example specify luminance redistribution over an increased pixel group size in local regions in which the main image content is relatively uniform, with no high spatial resolution image features, or in regions in which the side image content requires the increased contrast achievable through luminance averaging over such an increased group size. Likewise, the additional step may specify luminance redistribution over a reduced pixel group size for main image regions with sharp edges and other high resolution features, or in side image regions where the increased contrast achievable with the use of an increased pixel; group size is not required.

The invention can thus provide improved image quality to a viewer at a viewing position in the intended viewing range 6, such as a viewer at position 3 in figure 2, and/or can provide more effective obscuring of the image from a viewer who is at a position outside the intended viewing range, such as a viewer at position 5 in figure 2.

To the accomplishment of the foregoing and related ends, the invention, then, comprises the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative embodiments of the invention. These embodiments are indicative, however, of but a few of the various ways in which the principles of the invention may be employed. Other objects, advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.

In the annexed drawings, like references indicate like parts or features:
Figure 1: is an example schematic of a standard LCD display panel and associated control electronics. Figure 2: is a schematic of a display with a switchable public/private viewing mode, according to an embodiment of the present invention. Figure 3: is a plot of the available off-axis to on-axis luminance space producible by a display incorporating the present invention, when using two pixel groupings. Figure 4: is a plot of the available off-axis to on-axis luminance space producible by a display incorporating the present invention, when using four pixel groupings. Figure 5: is a schematic illustrating how a portion of the control electronics of the prior art for an RGB type display may be implemented in an electronic circuit. Figure 6: is a schematic illustrating how an embodiment of the present invention may be implemented in a portion of the control electronics in an RGB type display. Figure 7: is an illustration of the processing method of an embodiment of the present invention, and its operation on example input image data. Figure 8: is an illustration of A) the results of a processing method of an embodiment of the present invention, and B) the subsequent results of the privacy processing, applied to example input image data. Figure 9: is an illustration of the processing method of a further embodiment of the present invention, and its operation on example input image data. Figure 10: is an illustration of A) the results of a processing method of a further embodiment of the present invention, and B) the subsequent results of the privacy processing, applied to example input image data. Figure 11: is an plot of the different off-axis to on-axis luminance outputs produced by a display incorporating the present invention with four different side image levels when applying A) two pixel and B) four pixel groupings.

In a first embodiment, the display consists of a standard (single wide-viewing (public) mode only), LCD display, with modified control electronics that allow the display to be operated in either the wide-view (public) mode or in a narrow-view (private) mode. An LCD display generally consists of several component parts including:

1. A backlighting unit to supply even, wide angle illumination to the panel.

2. Control electronics to receive digital image data and output analogue signal voltages for each pixel, as well as timing pulses and a common voltage for the counter electrode of all pixels. A schematic of the standard layout of an LCD control electronics is shown in Figure 1 (Ernst Lueder, Liquid Crystal Displays, Wiley and sons Ltd, 2001).

3. A liquid crystal (LC) panel, for displaying an image by spatial light modulation, consisting of two opposing glass substrates, onto one of which is disposed an array of pixel electrodes and active matrix array to direct the electronic signals, received from the control electronics, to the pixel electrodes. Onto the other substrate is usually disposed a uniform common electrode and colour filter array film. Between the glass substrates is contained a liquid crystal layer of given thickness, usually 2-6 micro meter, which may be aligned by the presence of an alignment layer on the inner surfaces of the glass substrates. The glass substrates will generally be placed between crossed polarising films and other optical compensation films to cause the electrically induced alignment changes within each pixel region of the LC layer to produce the desired optical modulation of light from the backlight unit and ambient surroundings, and thereby generate the image.

An embodiment of the present invention is represented schematically in Figure 2. Generally the LCD Control Electronics (referred to herein also as control electronics), 1, will be configured specifically to the electro-optical characteristics of the LC panel, 2, so as to output signal voltages which are dependent on the input image data in such a way as to optimise the perceived quality of the displayed image, i.e. resolution, contrast, brightness, response time etc, for the principal viewer, 3, observing from a direction normal to the display surface (on-axis). The relationship between the input image data value for a given pixel and the observed luminance resulting from the display (gamma curve) is determined by the combined effect of the data-value to signal voltage mapping of the display driver, and the signal voltage to luminance response of the LC panel.

The LC panel, 2, will generally be configured with multiple LC domains per sub-pixel and/or passive optical compensation films so as to preserve the display gamma curve as closely as possible to the on-axis response for all viewing angles, thereby providing substantially the same high quality image to a wide viewing region, 4. However, it is an inherent property of liquid crystal displays that their electro-optic response is angularly dependent and the off-axis gamma curve will inevitably differ from the on-axis one. As long as this does not result in contrast inversion or large colour-shift or contrast reduction, this does not generally result in an obvious perceived fault in the observed image for the off-axis viewer, 5.

When the device of this embodiment is operating in the public mode, a set of main image data, 6, constituting a single image, is input to the control electronics, 1, in each frame period. The control electronics then outputs a set of signal data voltages to the LC panel, 2. Each of these signal voltages is directed by the active matrix array of the LC panel to the corresponding pixel electrode and the resulting collective electro-optical response of the pixels in the LC layer generates the image.

The control electronics has a single mapping of input pixel data value to output pixel data voltage (for example held in a Look-up table), which it applies to the process for all pixels. In some cases a different look-up table may be used for the red, green and blue sub-pixels of the display, but there is no variation in the mapping of an input data value to an output voltage based on the spatial position of the pixel data within the image, or the pixel electrode within the display. Substantially the same image is then perceived by the on-axis viewer, 3, and off-axis viewers, 5, and the display can be said to be operating in a wide viewing mode.

When the device is operating in the private mode, two image datasets are input to the control electronics, 1, in every frame period: main image data, 7, constituting a main image, and side image data, 8, constituting a side image.

The control electronics then outputs a set of signal data voltages, one data voltage for each pixel in the LC panel as previously. However, the control electronics (display controller) now utilises an expanded look-up table (LUT) and the output signal data voltage for each pixel in the LC panel, constituting a combined image, is dependent on the data values for the corresponding pixel (in terms of spatial position in the image) in both the main, 7, and side, 8, images. The output data voltage for each pixel is also dependent on a third parameter determined by the spatial position of the pixel within the display and the main and/or side image content at that image position. The expanded LUT therefore stores an output data value for every combination of input main image data value, input side image data value and spatial "flag" parameter.

In this way, the standard LCD control electronics are modified to receive, and store in a buffer, two, rather than one, images per frame period, and also to map the data values of two input images to a single output voltage per pixel, also taking into account a third, spatially dependent, parameter into this mapping. In this case the mapping of input image data to output pixel voltage is no longer identical for all pixels, or even all sub-pixels of the same colour component, in the display.

The third, spatially dependent, parameter may be a "flag" value indicating which of two or more pixel type classifications the pixel is deemed to be in based on its spatial position. For example, pixels in odd numbered columns in the image array may be said to form one class and pixels in even columns another. The classes could also constitute odd and even pixel rows, or perhaps the two parts of a chequerboard arrangement of the pixel array, etc. If more than two classifications are used, the spatial flag parameter could now have as many values as there are classifications. The number of output data values or signal voltages for each combination of main and side image data value may also be correspondingly increased. Pixel or sub-pixels in the display of the same colour type may be considered as groups, each group containing one or more pixels or sub-pixels of each classification type. The size of each pixel group, and therefore the number of different values the spatial "flag" parameter may have in any image region may be varied for different image regions and may be determined for each image region according to the main image and/or side image content at each region.

The output voltage from the control electronics, 1, then causes the LC panel, 2, to display a combined image which is the main image when observed by the main viewer, 3, with minimal degradation of the main image quality. However, due to the different gamma curve characteristic of the LC panel for the off-axis viewers, 5, these off-axis observers perceive the side image most prominently, which obscures and/or degrades the main image, securing the main image information to viewers within a restricted cone of angles centred on the display normal, 6.

The modified control electronics achieve this by altering the brightness of the individual sub-pixels within the groups of sub-pixels of the same colour type determined by the spatial flag parameter assignment. The individual brightness's are altered from that specified by the main image data, so that while the group maintains a luminance the same or proportional to the overall or average luminance specified by the main image data as observed by the on-axis viewer, 3, the distribution of luminance within the group is changed to a greater or lesser extent.

For example, two sub-pixels of 50% luminance may have their luminance altered in equal and opposite directions, so they maintain the same average to the on-axis viewer, but their average luminance to the off-axis viewer, 5, changes as the degree of alteration is increased. To illustrate this effect, the average normalised off-axis luminance produced by a pair of pixels of a typical VAN mode LCD, with all possible combinations of two data values is plotted against the average normalised on-axis luminance in Figure 3. This plot shows that, for any given average on-axis luminance (except the maximum and minimum luminance), a range of possible average off-axis luminances exist. Due to the viewing angle characteristics of VAN mode displays, in which mid-level data values generally produce a higher luminance relative to the maximum at off-axis angle than on-axis, pixel pairs in which the two data values applied to the pair are equal form the upper edge of the space of available off-axis to on-axis luminance points, and pixel pairs in which the combined luminance is concentrated as much as possible into one pixel of the pair (i.e. the data values are maximally different, one of the two being at 0 or 255 in an 8 bit display) lie along the lower edge of the available space.

It can be seen from Figure 3 that the envelope of available off-axis luminance values is widest, and therefore the available contrast for a side image is greatest, at the 50% on-axis luminance point. At this point, two pixels each producing 50% luminance on-axis can be replaced with two pixels, one of which is fully on and one of which is fully off, in order to reduce their resulting combined luminance to the off-axis viewer by approximately half, whilst remaining unchanged in overall appearance to the on-axis viewer. This allows a side image with a contrast ratio of approximately 2:1 to be produced for main image regions with 50% on-axis luminance.

The amount of redistribution of luminance which may be imposed on each pixel group, and therefore the degree of modification which may be achieved in the perceived brightness of the group to the off-axis viewer (i.e. side image contrast) is dependent on both the average luminance of the group and the number of pixels in the group. Figure 4 shows the equivalent average off-axis to on-axis luminance plot for pixel groups of four, rather than two pixels. It can be seen that the volume of the available off-axis to on-axis luminance space is expanded, and greater off-axis luminance control, and therefore side image contrast, is achievable, particularly at the 25% and 75% on-axis luminance points.

The present invention therefore proposes that the number of pixels in the group within which luminance is redistributed may be determined separately for each local image region, based on an analysis of the main and/or side image content at that region. The key advantage of such an adaptive pixel group size selection process is the simultaneous preservation of perceived image resolution for the on-axis viewer and maximisation of image contrast to the off-axis viewer. Although a larger group size allows increased side image contrast and therefore improved privacy strength, when the luminance produced by the group to the on-axis viewer is maximally redistributed (i.e. concentrated in as few of the sub-pixels of the group as possible) the main image resolution appears reduced by a larger amount. The pixel group size selection step therefore allows regions of the main image with high resolution features, or regions corresponding to low-contrast requirement areas of the side image, to retain a greater degree of the original resolution by redistributing luminance within a smaller group of neighbouring pixels, at the expense of some side image contrast in these image regions. Likewise, the group size selection step allows main image regions which are relatively uniform, or regions corresponding to areas in the side image where high contrast would be desirable, to have the privacy strength increased by redistributing luminance within a larger group of neighbouring pixels. For the purposes of this disclosure, we may take the term "high spatial resolution" to mean any image feature which occurs on a similar scale to the pixel group sizes which may be selected, e.g. in the embodiments described below, any image feature causing significant change in the image data over the area of the 4x2 block being sampled may be considered "high resolution". Likewise, we may consider "significant change" to mean data values which differ by greater than the thresholds suggested in those embodiments.

The invention thus provides an advance on the methods for producing a switchable privacy mode disclosed in GB2428152A1, WO2009110128A1, WO201134209 and WO201134208, through providing the additional processing step of main and/or side image content dependent adaptive pixel group size selection, and the above noted applications are incorporated herein by reference. The remainder of this disclosure will concentrate on the advantageous possible configurations and implementations of the pixel group size selection step so as to simultaneously optimise the perceived resolution of the main image and the contrast of the side image, and minimise the resource requirement.

In a preferred embodiment, the pixel group size selection step is based on analysis of the main image content only, and consists of a sequential analysis of each 2x2 block of like colour type sub-pixels in the main image, plus the two like colour type sub-pixels immediately adjacent horizontally to the 2x2 block on either side (as indicated by the block shown in broken lines in figure 7).. It should be noted that although in the current description, it is the size of groups of like colour sub-pixels in adjacent composite colour pixels which is being selected, for simplicity these are referred to as "pixel groups". This also reflects the fact in in other embodiments, the image elements within which luminance is redistributed, for which the size of grouping is being determined, may be different colour sub-pixels within the same or neighbouring colour pixels, or adjacent whole colour pixels. Figure 6 shows the modified image data processing method of this embodiment, whereby the spatial flag parameter determining block, which outputs a fixed pattern of pixel classification values, is replaced with the pixel group size selection process which analyses the main image content over a given region (if the main image pixel data values are fed into the LCD control electronics sequentially, this then requires a pixel data buffer) and then outputs a customised spatial flag parameter pattern for that image region.

The pixel group size selection step identifies if any of the 8 pixels in the 2x2 block plus 4 adjacent pixels have main image data values over a given threshold, e.g. 150 of a maximum 255 in an 8-bit per colour display. The pixel group size selection step also determines if the difference in the largest and smallest data values applied to the 8 pixels is greater than a given threshold, e.g. 50. These steps serve as a means of simply determining if there are any bright pixels, or high resolution features in the 8 pixel region. If either of the threshold tests is exceeded, then a two pixel classification pattern of spatial parameters is used for the 2x2 block forming the central 4 pixels of the 8 sampled, i.e. two of the four pixels are given one classification and the remaining two are given a second classification. If neither threshold is exceeded, than a four pixel group pattern is applied to the 2x2 block forming the central 4 pixels of the 8 sampled, i.e. each of the four pixels is given an individual classification value. This pixel group size selection method is illustrated in Figure 7. Having determined a pixel group size for the 2x2 block of pixels, the process then moves to the next 2x2 block of pixels and repeats. This continues until each 2x2 block in the image has had pixel classifications assigned to it, and this process is undertaken for each frame of main image input data.

In the preferred embodiment, if either of the threshold tests are exceed, the 2x2 block of pixels is assigned the pattern of classification values [1, 2, 1, 2] for the [top left, top right, bottom right and bottom left] pixels respectively If the tests are not exceeded, the 2x2 block is assigned the pattern of classifications of [3, 6, 4, 5] or [4, 5, 3, 6], depending on the position of the 2x2 block in the image. The pixel classification value is then used to select which of the six output data values, for each main image and side image input data value combination in the LUTs is selected for output for that pixel. The output data values in the LUTs are therefore calculated so that the output values for

pixel classifications

1 and 2 in combination produce the desired average on-axis and off-axis luminance for each main and side image input data value combination, while the same is also true for the

output values

3, 4, 5 and 6 in combination. If the LUT output values for pixel class 1 are larger than those of pixel class 2, then repeated blocks of the [1,2,1,2] output pattern will form a chequerboard pattern of brighter and darker pixels in the output image. If the output data values for the [3, 4, 5, 6] classification set decrease in order from 3 to 6, then the [3, 6, 4, 5] or [4, 5, 3, 6] patterns will produce a sparse chequer pattern of the same phase as the [1,2,1,2] pattern, i.e. the position of the brightest pixels in each pattern matches. This is advantageous as sharp boundaries between 2 pixel and 4 pixel groupings will not result in the brightest pixels of the output set being positioned adjacent to each other, which may cause visible artefacts in the output image.

Providing two possible output patterns for the four pixel group output, [3, 6, 4, 5] and [4, 5, 3, 6] is also advantageous, as for main image regions of 25% luminance, a single pixel in the output set of four pixels may be much brighter than the other three, in this case the position of that bright pixel in the block is varied which provides improved output image appearance to the on-axis viewer. The two output patterns also share the property that the position of the brightest output pixels (type 3 and 4) remains in phase with the position of the brightest output pixels in the [1,2,1,2] pattern output, thereby again ensuring that transitions between 2 pixel groups and 4 pixel groups appear seamless to the on-axis viewer.

These processing methods and results are illustrated in Figures 7 and 8. Figure 7 outlines the pixel group size selection method of the preferred embodiment and shows the method as applied to a main image consisting of a blurred bright diagonal line on a darker background (main image data values of 0-255 are used to illustrate the effect applied to an 8 bit per colour channel display). Figure 8 A) shows the result of the pixel group size selection step in terms of the pixel type classification assigned to each pixel. It can be seen that each 2x2 block of pixels has been assigned the classification pattern [1,2,1,2] where the block is in the vicinity of the bright line, and has been assigned the classification pattern [3, 6, 4, 5] or [4, 5, 3, 6] elsewhere. Figure 8 B) shows the result of such pixel class assignments in the output privacy processed image, for the same input main image region, and a corresponding side image content which causes the luminance produced by each group to be concentrated into as few of the pixels in the group as possible. It can be seen that the result of this is where 4 pixel groups were assigned, only one pixel in the group (that assigned class 3) is turned on, and therefore provides the luminance of the entire 4 pixel group form the input main image. In the vicinity of the brighter line, as the pixel group size selection process detected a higher resolution feature and specified pixel groups of size 2 to be applied, a 2x2 bright-dark chequer pattern is produced. For pixels assigned class 1, the output data values are higher than the input data values (inputs of 64 and 191 are increased to 120 and 255 respectively), whereas the pixels assigned class 2 have their data value reduced to zero.

It can be seen from this example that, although the privacy processing breaks up the continuity of bright diagonal line on a pixel scale, the appearance of the line is significantly better preserved than if luminance redistribution over groups of four pixels was applied over the whole image region. Likewise, the concentration of luminance into just one out of each four pixels in the darker background region improves the privacy strength of the output image compared to standard method where luminance is redistributed among pairs of pixels only.

In a further embodiment, the pixel buffer memory requirement is reduced by analysing each 4x1 pixel block of input main image data, rather than each 2x2 block. As the main image data is typically read into the display control electronics sequentially, from left to right one row at a time, the requirement to analyse the data of pixels on different rows means the pixel memory buffer has to have capacity to store at least a full line of pixel data. This requirement is reduced to just four pixel by the use of a 4x1 analysis kernel.

In Figure 7, Pixel Group Size Selection Process is below:
(1) For each colour channel in the main image,
For each 2x2 block in the main image,
(2) If there is one pixel in the 4x2 block centred on the 2x2 block with data value > threshold 1.
(3) Or the difference between the highest data value and the lowest data value in the 4x2 block > threshold 2.
(4) Then apply 2 x 2 pixel groups in the 2x2 block.
(5) Otherwise apply 1 x 4 pixel group in the 2x2 block.

As with the previous embodiment, the main image data for a single colour channel of the main image at a time are analysed each 4x1 block at a time. While the previous embodiment analysed the neighbouring pixels in addition to the four pixels in the 2x2 block which were having their classification type determined, in order to decide the classification, in this embodiment the pixels having their classification determined are the only ones to be analysed. Also as with the previous embodiment, the four pixels in the current block are analysed to see if any of the pixels have a data value exceeding a first threshold, and also to see if the difference between the largest and smallest data value in the block exceeds a second threshold. If either of these threshold conditions is exceeded, the 4x1 block of pixels is assigned the pattern of classification values [1, 2, 1, 2] or [2, 1, 2, 1] for the [leftmost, centre left, centre right and rightmost] pixels respectively, depending on the position of the 4x1 block. If the tests are not exceeded, the 4x1 block is assigned the pattern of classifications of [3, 6, 4, 5], [5, 4, 6, 3], [4, 5, 3, 6] or [6, 3, 5, 4], depending on the position of the 4x1 block in the image. This ensures that regions in which the threshold tests are not exceeded result in the same pattern of pixel classifications as the previous preferred embodiment produced, after four or more rows have been processed. Figures 9 and 10 show the equivalent process outline, pixel classification result and resultant privacy processing result of Figures 7 and 8, for the process of this embodiment. It can be seen that the method of this embodiment produces a very similar result for bright diagonal line features as the preferred embodiment. The method of this embodiment will also produce similar output images to the preferred embodiment for bright and dark vertical lines and other fine features, but fine horizontal line type features will be indistinguishable from plain background regions due to the single line kernel, the process of this embodiment may produce significantly differing outputs to the preferred embodiment in these cases.

It may be noted that in both of the previous examples, the threshold tests have been arranged so that 2x2 or 4x2 blocks of pixels analysed in each step have not specified output groups of size two in the instances where the edge of the bright feature has fallen within the sampling kernel, but the central, brightest pixels have not. This has produced the result that no more than two 2x2 block horizontally adjacent, or a single 4x1 block have been specified to have pixels groups of size two instead of four, minimising the area required to preserve the appearance of the fine feature at the expense of privacy strength. It may be advantageous for the threshold test to be arranged so this is not the case however, and a broader region of reduced pixel group size is produced around fine features. This is simply a case of tuning the process parameters to produce the preferred output image appearance for the typical main image content of the display.

In Figure 9, Pixel Group Size Selection Process is below:
(1) For each colour channel in the main image,
For each 4x1 block in the main image,
(2) If there is one pixel in the 4x1block with data value > threshold 1.
(3) Or the difference between the highest data value and the lowest data value in the 4x1 block > threshold 2.
(4) Then apply 2 x 2 pixel groups in the 4x2 block.
(5) Otherwise apply 1 x 4 pixel group in the 4x1 block.

It should also be noted that while the above two embodiments have been described with two threshold type tests used to perform the pixel group size selection process, any image content analysis method may be used which provides the desired discrimination between image regions in which are best suited to a particular pixel group size.

In a still further embodiment, rather than solely the main image content in each region being subject to analysis to inform the pixel group size selection process, both the main and side image content in each image region are sampled. In this way, main image regions which have no fine features, and therefore might be subject to four pixel groupings based on main image analysis only, may have two pixel groupings applied instead if the side image content for that region is such that there is no particular advantage to be gained by using increased pixel size groupings. This may be the case for uniformly bright side image regions. This method may be advantageous for use in displays with a pixel pitch and typical viewing distance that means that the sparse pixel pattern produced by four pixel groups causes a visible "grainy" or coarse texture to be perceived by the on-axis viewer in four pixel grouping regions. This is as it allows the use of such regions to be restricted to those areas in which the increased side image contrast is beneficial.

In a still further embodiment, rather than solely the main image content in each region being subject to analysis to inform the pixel group size selection process, solely the side image content is used. This may be advantageous in that it reduces the computing resource requirement to apply the group size selection process.

In a still further embodiment, a process is applied which may be similar to any of the above embodiments, but rather than selecting simply between pixel group sizes of two or four, allows selection between any number of different group sizes, e.g. group of 2, 3 and 4 may be selected between, or two, 4 and 8. This may be advantageous in the future, when even higher ppi displays become commonplace, allowing luminance redistribution within pixel group sizes of larger than four, without the resulting sparsely patterned bright-dark pixel arrangement being visible to the viewer. The process may also have capability to specify a group size of 1 (i.e. no redistribution of luminance). This may be advantages in low ppi displays in which any luminance redistribution around fine main image features would appear too disruptive to the appearance of those features to the on-axis viewer.

In a still further embodiment, in addition to the pixel group size selection process, a further process is included in the display control electronics which registers instances of the pixel group size selection step changing its output from one neighbouring group of pixels to another. As the size of the pixel group determines the range of off-axis luminances which may be produced by the group for any given on-axis luminance, changes in the pixel group size, within an image region where the side image value is constant, may cause a change in the perceived luminance to the off-axis viewer. This effect is illustrated in figure 11, which shows the off-axis to on-axis luminance values produced by the privacy display, as a function of main image input luminance, for four side image levels. The side image levels have been calculated to cover the full available range of off-axis luminances in equally spaced steps. Figure 11 A) shows the resulting outputs for two pixel groupings, and B) shows the result for 4 pixel groupings. It can be seen that due to the expanded off-axis luminance range available in the four pixel group process, there will be a mismatch in off-axis luminance levels when the pixel group size changes, particularly for darker side image levels. These visible boundaries may detract from the side image appearance, so the process of this embodiment may also include a means of modifying the input side image data, or utilising a further expanded privacy process LUT set, in order to blend or smooth this transition. (Although the primary purpose of the side image is to obscure the main image at viewing positions outside the private viewing range (such as the viewing positions 5 in figure 2), in some embodiments the side image may show a logo, a personalised image etc - and in such embodiments it may be desirable for the side image to be as accurately reproduced as possible.)

Although the invention has been shown and described with respect to a certain embodiment or embodiments, equivalent alterations and modifications may occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described elements (components, assemblies, devices, compositions, etc.), the terms (including a reference to a "means") used to describe such elements are intended to correspond, unless otherwise indicated, to any element which performs the specified function of the described element (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein exemplary embodiment or embodiments of the invention. In addition, while a particular feature of the invention may have been described above with respect to only one or more of several embodiments, such feature may be combined with one or more other features of the other embodiments, as may be desired and advantageous for any given or particular application.

When the number of pixels in a group is selected according to the image data, the method may comprise identifying one or more regions of the image in which there are high spatial resolution features, and selecting a small group size for the identified region(s). Use of a small group size retains a higher degree of the original resolution of the image, and it may therefore be preferable to use a small group size in regions of the image in which there are high spatial resolution features or other image features which may have their appearance disrupted by luminance redistribution among a larger group of pixels.

The method may comprise for each distinct group of N pixels in the image, determining the pixel voltages to obtain either a luminance redistribution within all N pixels of the group or a luminance redistribution within M groups of N/M pixels.

Determining the pixel voltages to obtain a luminance redistribution within all N pixels of a group may comprise determining pixel voltages for pixels within the group such that one pixel of the group has maximal luminance. Concentration of luminance into just one pixel of a group of pixels improves the strength of the obscuring effect caused by the luminance variations, and so provide a better privacy effect.

N may be 4 and M may be 2. It should however be understood that the invention is not limited to a luminance redistribution within M groups that each contain the same number (ie, N/M) of pixels. For example, if N = 8, it is possible that the pixels may be divided into one group of four pixels and two groups of two pixels each, as an alternative to dividing the pixels into two groups of four pixels or four groups of two pixels.

The method may comprise determining pixel voltages such that, in the displayed image, the resultant arrangement of brighter and darker pixels in a first region of the image having a first pixel group size is in phase with the resultant arrangement of brighter and darker pixels in a second region of the image, the second region being adjacent to the first region and having a first pixel group size different from the second pixel group size. This prevents visible artefacts appearing at the boundaries between regions of the image which have different pixel group sizes.

The method may comprise, for a block of pixels, defining one or more groups of pixels in the block on the basis of image data for the pixels in the block.

The method may comprise, for a block of pixels, defining one or more groups of pixels in the block on the basis of image data for the pixels in the block and for at least one pixel outside the block.

The method may comprise, for pixels at a boundary between a first image region having groups of a first size and a second image region having groups of a second size different from the first size, modifying the pixel voltages to match luminance levels perceived at the first viewing position. This prevents a mismatch between the luminance levels (perceived at the first viewing position) for pixels in the first image region and for pixels in the second image region which might otherwise lead to visible boundaries appearing in the image perceived at the first viewing position.

The signal voltages to be applied to pixels in a group of pixels may be determined such that the overall luminance of the pixels in the group is equal to the overall luminance specified for the pixels in the group by the image data.

The second viewing position may be a substantially on-axis viewing position.

The method may comprise, in a second mode, determining signal voltages to be applied to pixels of the image display panel from the received image data thereby to generate an image perceivable at both the first viewing position and at the second viewing position. This mode provides a wide-view (public) display mode. Switching between the first (private) mode and the second (public) mode may be effected by, for example, use of a "Privacy mode On/Off" signal as shown in figure 6.

A second aspect of the present invention provides a control circuit for a display panel, the control circuit being adapted to perform a method of the first aspect.

A third aspect of the present invention provides a display comprising a control circuit of the second aspect and a display panel, the control circuit being adapted to, in use, output the determined signal voltages to the display panel.

The display panel may be a liquid crystal display panel.

A fourth aspect of the present invention provides display panel adapted to perform a method of the first aspect.

A fifth aspect of the present invention provides a computer-readable medium containing instructions which, when executed by a processor, cause the processor to perform a method of the first aspect.

The embodiments of this invention are applicable to many display devices, and a user may benefit from the option of a privacy function on their normally wide-view display for use in certain public situations where privacy is desirable. Examples of such devices include mobile phones, Personal Digital Assistants (PDAs), laptop computers, desktop monitors, Automatic Teller Machines (ATMs) and Electronic Point of Sale (EPOS) equipment. Such devices can also be beneficial in situations where it is distracting and therefore unsafe for certain viewers (for example drivers or those operating heavy machinery) to be able to see certain images at certain times, for example an in car television screen while the car is in motion.

1. LCD Control Electronics
2. Liquid crystal panel
3. Principal viewer
4. Angular viewing range of the main image in the public mode
5. Off-axis viewer
6. Angular viewing range of the main image in the private mode.
7. Input main image data
8. Input side image data

Claims

A method of processing image data for an image display panel, the method comprising, in a first mode:
determining signal voltages to be applied to pixels of the image display panel from received image data constituting an image for display on the image display panel and from a secondary data value for the pixels thereby to generate luminance variations perceivable at a first viewing position but substantially not perceivable at a second viewing position;
wherein the signal voltages to be applied to pixels in a group of pixels are determined such that the overall luminance of the pixels in the group is dependent on the overall luminance specified for the pixels in the group by the image data;
and wherein the number of pixels in a group is locally variable and is selected according to the image data and/or the secondary data values for pixels in a region of the image where the group is to be defined.
A method as claimed in claim 1 wherein the number of pixels in a group is selected according to the image data, the method comprising identifying one or more regions of an image defined by the image data in which there are high spatial resolution features, and selecting a small group size for the identified region(s).
A method as claimed in any preceding claim, and comprising for each distinct group of N pixels in the image, determining the pixel voltages to obtain either a luminance redistribution within all N pixels of the group or a luminance redistribution within M groups of N/M pixels.
A method as claimed in claim 3 wherein determining the pixel voltages to obtain a luminance redistribution within all N pixels of a group comprises determining pixel voltages for pixels within the group such that one pixel of the group has maximal luminance.
A method as claimed in claim 3 or 4 wherein N = 4 and M = 2.
A method as claimed in any preceding claim and comprising determining pixel voltages such that, in the displayed image, the resultant arrangement of brighter and darker pixels in a first region of the image having a first pixel group size is in phase with the resultant arrangement of brighter and darker pixels in a second region of the image, the second region being adjacent to the first region and having a first pixel group size different from the second pixel group size. to prevent visible artefacts at the boundaries of such regions.
A method as claimed in any preceding claim and comprising, for a block of pixels, defining one or more groups of pixels in the block on the basis of image data for the pixels in the block.
A method as claimed in any one of claim 1 to 6 and comprising, for a block of pixels, defining one or more groups of pixels in the block on the basis of image data for the pixels in the block and for at least one pixel outside the block.
A method as claimed in any preceding claim and comprising, for pixels at a boundary between a first image region having groups of a first size and a second image region having groups of a second size different from the first size, modifying the pixel voltages to match luminance levels perceived at the first viewing position.
A method as claimed in any preceding claim wherein the signal voltages to be applied to pixels in a group of pixels are determined such that the overall luminance of the pixels in the group is equal to the overall luminance specified for the pixels in the group by the image data.
A method as claimed in any preceding claim wherein the second viewing position is a substantially on-axis viewing position.
A method as claimed in any preceding claim and comprising, in a second mode, determining signal voltages to be applied to pixels of the image display panel from the received image data thereby to generate an image perceivable at both the first viewing position and at the second viewing position.
A control circuit for a display panel, the control circuit being adapted to perform a method as defined in any one of claims 1 to 12.
A display comprising a control circuit as defined in claim 13 and a display panel, the control circuit being adapted to, in use, output the determined signal voltages to the display panel.
A display as claimed in claim 14 wherein the display panel is a liquid crystal display panel.
A display panel adapted to perform a method as defined in any one of claims 1 to 12.
A computer-readable medium containing instructions which, when executed by a processor, cause the processor to perform a method as defined in any one of claims 1 to 12.