JP2005269442A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of appropriately correcting only a natural image without increasing a memory size and a calculation amount in image data including a frame image.
An image processing apparatus includes an image forming unit, an image processing unit, and an image display unit. The image forming unit forms a frame image and a natural image. The image forming unit transmits the frame image to the image processing unit in advance, and the natural image is transmitted to the image processing unit after performing mask processing on a portion (mask portion) where the frame on the image is arranged. The image processing unit detects the mask portion, performs image correction processing only on the non-mask portion, and displays the mask portion without correction using a frame image. As a result, the color of the displayed frame image does not change from the original one, and conversely, the color of the natural image does not change due to the influence of the frame image. Further, since it is not necessary to transfer the frame image and the natural image after each frame is combined, the processing time is shortened and the data transfer amount is also reduced.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus that performs image processing on image data including a frame image.

  Image processing apparatuses that display image data in which graphic data images (hereinafter simply referred to as “frames” or “frame images”) such as frames, characters, and backgrounds are superimposed on natural images such as image data are known. It has been. When image data including such a frame image is subjected to image correction (for example, contrast correction or saturation correction), if the image correction is performed including not only the natural image data but also the frame image data as it is, the color of the frame is changed. As a result, the natural image may not be corrected correctly, or the frame itself may change for each pixel.

  In addition, when a frame image is attached to a moving image instead of a still image, if the correction including the frame image including the frame image is performed so that the correction amount changes temporally depending on the feature amount of each frame of the moving image, the above-described problem occurs. In addition, the color of the frame image that should not change with time changes with time as the moving image changes. For the reasons described above, it is preferable that correction processing is performed only on the natural image using only the natural image information excluding the frame image, and the color of the frame image is not changed.

  As an image processing method for image data including a frame image, there is a method of performing correction processing on natural image data of each frame on the image forming side, and then transmitting and displaying the correction data superimposed on the frame image. In this case, since processing is performed on the image forming side, considerable time is required for correction processing and overlay processing. On the other hand, there is an image processing method in which not only a natural image but also a frame image is transferred to the image processing side together in each frame, and after correction processing is performed on the image processing side, the frame image is displayed superimposed on the frame. In this case, since the frame data that does not change is also sent every frame, the transfer amount increases and the power consumption also increases.

  Therefore, in the image processing apparatuses described in Patent Documents 1 and 2, a frame image is determined on the image processing side based on a luminance distribution, a saturation distribution, or the like on the assumption that the frame image data has a specific color (number of colors limited). However, a method of correcting other than the frame is used. In Patent Document 3, a natural image and a frame image are independently sent to the image processing side, the image processing side performs image processing on the natural image and saves it in a memory, and the frame image is saved in another memory. In addition, an image processing method is described in which both memories are read and a natural image and a frame image are superimposed and displayed.

  However, the techniques described in Patent Documents 1 and 2 have a problem that frame pixels cannot be accurately determined when the frame image itself is an image having an abundant number of colors and distribution. On the other hand, the technique described in Patent Document 3 has a problem in that the amount of memory used increases because it is necessary to save all data of a natural image and a frame image.

JP-A-10-208034 JP 2000-13623 A JP 2003-233366 A

  The present invention has been made in view of the above points, and in image data including a frame image, an image that can be appropriately corrected for only a natural image without increasing the memory size and the calculation amount. It is an object to provide a processing apparatus.

  In one aspect of the present invention, an image processing apparatus includes an image forming unit and an image processing unit, and the image forming unit forms first image data and second image data. And a mask processing means for performing a mask process on the first image data so that a portion where the first image data and the second image data overlap can be identified. Transfer means for separately transferring the first image data and the second image data to the image processing section, wherein the image processing section overlaps the transferred first image data in the overlapping manner. A mask detecting means for detecting a portion, an image correcting means for correcting the first image data or the first image data other than the overlapping portion, and the image corrected first based on the mask detection result. 1 It comprises a means for superimposing the image data and the second image data.

  From the same viewpoint, an image processing method executed in an image processing apparatus including an image forming unit and an image processing unit includes: an image forming processing step for forming first image data and second image data; A mask processing step for performing a mask process on the first image data so that a portion where one image data and the second image data overlap can be identified; and the first image subjected to the mask process And a transfer step of separately transferring the data and the second image data from the image forming unit to the image processing unit, and detecting the overlapping portion in the transferred first image data An image correction step of correcting the first image data or the first image data other than the overlapping portion, and the image corrected first image based on the mask detection result Includes a step of superimposing said the over data a second image data.

  The image processing apparatus includes an image forming unit and an image processing unit. The image forming unit is a host computer system including, for example, a CPU, RAM, ROM, control software, and the like. The image processing unit includes hardware such as an LCD (Liquid Crystal Display) driver and an LCD controller, and control formware thereof. The image forming unit of the image forming unit forms first image data such as a natural image (image data), for example, and forms second image data such as a frame image (graphic data). The mask processing means performs a mask process on the first image data so that a portion where the first image data and the second image data overlap (hereinafter also referred to as “mask portion”) can be identified. In this way, the image forming unit separately transfers the first image data and the second image data after the mask processing to the image processing unit.

  The mask detection means of the image processing unit detects the mask portion in the first image data masked so that the mask portion can be identified. Then, the image correction means performs image correction on the first image data or first image data other than the mask portion (hereinafter also referred to as “non-mask portion”). The superimposing means superimposes the image-corrected first image data and the non-image-corrected second image data based on the mask detection result. As a result, first image data that has undergone image correction (only the image of the non-mask portion) and second image data that has not undergone image correction are created. Therefore, in the above-described image processing apparatus and image processing method, when the created image data is displayed, the color of the displayed second image data changes from the original one, and conversely, the second image data The color of the first image data displayed does not change due to the influence of. In addition, since the image forming unit does not need to transfer the second image data and the first image data after each frame is combined, the processing time is shortened and the data transfer amount is also reduced.

  In one aspect of the present invention, in the image processing apparatus, the mask processing unit adds data for identifying the overlapping portion to the specific bit of the first image data, and the mask detection unit includes the specific bit. The overlapping portion is detected based on the value of. In this aspect, different data is added to the specific bits of the first image data between the image data of the mask portion and the non-mask portion.

  In another aspect of the present invention, in the image processing apparatus, the mask processing unit sets the overlapping portion of the image data to a unique value, and the mask detection unit determines that the transferred first image data is a unique value. The image data is determined to be the overlapping portion. In this aspect, the image data of the mask portion is made data (eigenvalue) that cannot be possessed by the image data of the non-mask portion. Thereby, the image data of the non-mask portion and the mask portion can be identified.

  According to another aspect of the present invention, in the image processing apparatus, the mask processing unit converts a bit number and a color format of the image data of the overlapping portion, and the mask detection unit is configured to transfer the transferred first data. The overlapping portion is detected based on predetermined bit data of the image data. In this mode, by converting the number of bits and the color format of the image data of the mask part, the data of the predetermined part of the image data of the mask part becomes a value that the image data of the non-mask part cannot have, and both can be identified become.

  In another aspect of the present invention, in an image processing apparatus including an image forming unit and an image processing unit, the image forming unit includes image forming processing means for forming first image data and second image data; Converting the color format of the first image data or the second image data, and combining the first image data and the second image data; and combining the combined image data to the image processing unit. A transfer means for transferring, wherein the image processing unit separates the transferred image data into the first image data and the second image data, and the separated first Image correction means for correcting image data, and means for superimposing the image-corrected first image data and the separated second image data.

  From the same viewpoint, an image processing method executed in an image processing apparatus including an image forming unit and an image processing unit includes: an image forming processing step for forming first image data and second image data; The image forming unit converts the color format of the first image data or the second image data, combines the first image data and the second image data, and combines the combined image data with the image forming unit. A transfer step of transferring the transferred image data to the first image data and the second image data, and a transfer step of transferring the transferred image data to the image processing unit. An image correction step of correcting the image data; and a step of superimposing the image-corrected first image data and the separated second image data.

  Also by the image processing apparatus and the image processing method described above, the first image that has been subjected to image correction and the second image data that has not been subjected to image correction can be appropriately displayed. Therefore, the color of the displayed second image data does not change from the original one, and conversely, the color of the first image data displayed does not change due to the influence of the second image data. Further, in the above image processing apparatus and image processing method, the first image data data and the second image data data are simultaneously transferred from the image forming unit to the image processing unit. That is, one image data obtained by combining the first image data data and the second image data data is transferred. As a result, the transfer amount does not increase even when the first image data and the second image data are moving images. Further, even when the image processing unit does not have a frame memory or the like and the second image data and the first image data cannot be transferred separately and superimposed, the image processing can be appropriately performed.

  Preferably, the synthesizing unit converts the number of bits and the color format of the first image data or the second image data, and the separation unit is based on predetermined bit data of the transferred image data. To separate. In this case, by converting the number of bits and the color format of the second image data, the second image data and the image data of the first predetermined bit become different and can be separated.

  Further preferably, the image forming processing unit forms second image data having a smaller number of bits than the first image data, and the synthesizing unit sets the number of bits that the second image data is insufficient. A predetermined value is added to the second image data, and the second image data is rearranged, and the separation unit separates the second image data based on predetermined bit data of the transferred image data. In this case, the first image data and the second image data are separated by looking at the predetermined value added to the bit area where the second image data is insufficient. The number of gradations of the second image data may not necessarily be large, and the above image processing apparatus performs image processing effectively by utilizing the small number of gradations of the second image data. Can do.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
(Configuration of image display device)
First, an image display apparatus including an image processing unit according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 shows a schematic configuration of an image display apparatus 101 according to the first embodiment.

  The image display apparatus 101 mainly includes an image forming unit 1, an image processing unit 2, and an image display unit 3. In the image display device 101, the image processing unit 2 performs image processing on the image formed by the image forming unit 1, and the processed image is displayed on the image display unit 3. The image forming unit 1 and the image processing unit 2 function as the above-described image processing apparatus.

  The image forming unit 1 includes image forming processing units 5 and 7, a composite information forming unit 6, a mask processing unit 8, and image memories 9 and 10. The image forming unit 1 can be a host computer system including a CPU, RAM, ROM, communication device, drawing device, disk drive device, control software, and the like.

  The image forming processor 5 forms a natural image (image data) such as a moving image or a still image. This natural image is, for example, image data of “8” bits for each of RGB, for a total of “24” bits. Further, the image formation processing unit 5 may convert the natural image into a color format indicating luminance / color difference (YUV) in accordance with human visual characteristics. The image processing forming unit 5 supplies a signal S 1 corresponding to the formed natural image to the mask processing unit 8. The image processing forming unit 5 forms the first image data described above.

  The image formation processing unit 7 forms image data such as a frame, a character, and a figure to be superimposed on the natural image as described above (that is, “frame image”). The frame image is graphic data, for example, image data in RGB format. The image processing forming unit 7 writes a signal S4 corresponding to the formed frame image in the image memory 10. If the frame image is a still image, the frame image does not change from frame to frame, so there is no need to send it every frame. The image processing forming unit 7 forms the second image data described above. As described above, the image forming processing units 5 and 7 function as image forming processing means in the image forming unit 1.

  The composite information forming unit 6 forms conditions for superimposing the natural image and the frame image (hereinafter referred to as “composite information”). This combination information indicates information such as which one of the natural image and the frame image is preferentially displayed in each pixel, for example. The combined information forming unit 6 supplies a signal S2 corresponding to the formed combined information to the mask processing unit 8. If the frame image is a still image, the frame image does not change for each frame, so that it is not necessary to send composite information for each frame.

  The mask processing unit 8 performs mask processing on the acquired natural image data S1 based on the synthesis information S2. In the mask process, the natural image data S1 is processed so that a portion where the natural image data and the frame image data overlap can be identified. Hereinafter, a portion where the natural image and the frame image overlap is referred to as a “mask portion”, and a portion which does not overlap the natural image and the frame image is referred to as a “non-mask portion”. The mask processing unit 8 functions as the above-described mask processing unit in the image forming unit 1. The mask processing unit 8 writes a signal S3 corresponding to the masked image data in the image memory 9. The specific mask processing will be described later in detail.

  The image memory 9 is a memory in which the masked natural image data S3 is written and stored. The image memory 10 is a memory for storing frame image data S4 written by the image forming processing unit 7.

  The natural image data S5 is transferred from the image memory 5 to the image processing unit 2. Further, the frame image data S 6 is transferred from the image memory 10 to the image processing unit 2. Here, the transfer of the natural image data S5 and the transfer of the frame image data S6 are not performed simultaneously, but are performed separately in time via the same signal line.

  The image processing unit 2 includes a mask detection unit 11, an image correction unit 12, a memory write control unit 13, and a frame memory 14. The image processing unit 2 includes hardware such as an LCD (Liquid Crystal Display) driver and an LC controller, and control firmware thereof.

  The mask detection unit 11 reads the natural image data S5 from the image memory 9 and performs mask detection. The mask detection unit 11 detects the image data of the mask portion in the acquired natural image data S5. Then, the mask detection unit 11 supplies the image correction unit 12 with the image data S7 of the non-mask portion of the natural image data. Further, the mask detection unit 11 supplies information (hereinafter referred to as “mask instruction information”) S <b> 8 regarding the image data of the mask portion to the memory write control unit 13. The mask instruction information is created based on the synthesis information acquired by the mask processing unit 8. In this case, the mask detection unit 11 delays the mask instruction information S8 by the same number as the required number of clocks in the processing in the image correction unit 12. As described above, the mask detection unit 11 functions as the mask detection unit described above in the image processing unit 2.

  The image correction unit 12 performs image correction processing on the image data S7 of the non-mask portion supplied from the mask detection unit 11. As the image correction, known methods such as brightness adjustment, contrast correction, and saturation correction are performed. Then, the image correction unit 12 supplies the image data S9 after the image correction to the memory write control unit 13. As described above, the image correction unit 12 functions as an image correction unit in the image processing unit 2.

  The memory writing control unit 13 performs control for writing the acquired image corrected image data S9 of the non-masked portion and the signal S6 corresponding to the frame image to the frame memory 14. In this case, the memory write control unit 13 performs write control based on the mask instruction signal S8 acquired from the mask detection unit 11. That is, the memory write control unit 13 writes only the data of the non-masked portion of the natural image data S9 and the data corresponding to the masked portion of the frame image data S6 into the frame memory 14. As a result, the frame memory 14 includes image-corrected natural image data (images excluding a portion overlapping the frame image) and image-corrected frame images (hereinafter collectively referred to as “composite images”). Remembered. As described above, the memory write control unit 13 functions as the superimposing means described above.

  The frame memory 14 stores the composite image data based on the write signal S10 from the memory write control unit 13. The composite image data stored in the frame memory 14 is read out to the image display unit 3.

  The image display unit 3 reads the composite image data S11 stored in the frame memory 14 and displays it. The image display unit 3 includes a display device such as an LCD. The image display unit 3 displays RGB format image data, and can display a total of “24” bits (“8” bits for each RGB) or “18” bits (“6” bits for each RGB). .

  As described above, in the image display apparatus 101 according to the present embodiment, the natural image that has been image-corrected (only the image of the non-mask portion) and the frame image that has not been image-corrected are appropriately displayed on the image display unit 3. . That is, since the frame image is not image-corrected, the color of the displayed frame image does not change from the original one, and conversely, the color of the natural image displayed by the influence of the frame image does not change. Further, in the image display apparatus 101 according to the present embodiment, even if the natural image is a moving image, the image forming unit 1 does not need to transfer the frame image and the natural image for each frame, so that the processing time And the amount of data transferred is also reduced. Further, the image forming unit 1 transfers the frame image and the natural image separately to the image processing unit 2, but the frame memory is only required for one frame, so that the memory cost does not increase.

(Image processing procedure)
Hereinafter, the image processing procedure according to the first embodiment will be specifically described with reference to FIG.

  First, a processing procedure in the image forming unit 1 will be described. The image formation processing unit 5 forms a natural image A1, and the image formation processing unit 7 forms a frame image B1. The mask processing unit 8 performs mask processing on the natural image A1. In the first embodiment, the mask processing unit 8 adds a predetermined value to a specific bit of the natural image data so that the mask portion and the non-mask portion can be identified. Specifically, the mask processing unit 8 sets the specific bit value to “0” for the mask portion A2a, and sets the specific bit value to “1” for the non-mask portion A2b. The natural image A2 masked as described above is stored in the image memory 9 and read by the mask detection unit 11 of the image processing unit 2.

  Next, a processing procedure in the image processing unit 2 will be described. The mask detection unit 11 performs mask detection on the read natural image data S5. Specifically, the mask detection unit 11 detects a mask portion based on a specific bit value of natural image data. That is, the mask detection unit 11 determines that the specific bit value is “0” and is a non-mask portion, and determines that the specific bit value is “1” and is the mask portion. The natural image A <b> 3 (image data of the non-mask part) detected in this way is held and supplied to the image correction unit 12. On the other hand, the result of mask detection is supplied to the memory write control unit 13 after being delayed by the same number of clocks required for image correction, for example, as 1-bit mask instruction information S8 (that is, specific bit information).

  The natural image data A3 having the image held is subjected to image correction by the image correction unit 12. The natural image data S9 subjected to image correction and the frame image data S6 not subjected to image correction are controlled to be written to the frame memory 14 by the memory write control unit 13. First, the memory writing control unit 13 writes the frame image data S6 into the frame memory 14. Next, the memory write control unit 13 writes only the data of the non-masked portion of the natural image data S9 into the frame memory 14 based on the mask instruction information S8 from the mask detection unit 11. As described above, the composite image data is stored in the frame memory 14. Then, the signal S11 corresponding to the composite image data stored in the frame memory 14 is read out to the image display unit 3. The image display unit 3 displays the composite image C1 stored in the frame memory 14.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.

  Since the configuration of the image display apparatus according to the second embodiment is the same as that of the image display apparatus 101 shown in the first embodiment, the description thereof is omitted. Here, an image processing procedure according to the second embodiment will be described.

  FIG. 3 is a diagram illustrating an image processing procedure in the second embodiment. The image processing in the second embodiment is different from that shown in the first embodiment with respect to the mask processing method and the mask detection method. Hereinafter, only the mask processing method and the mask detection method according to the second embodiment will be described in detail.

  The mask processing unit 8 of the image forming unit 1 converts the RGB format natural image data formed by the image forming processing unit 5 into YUV format data according to the following equations (1) to (3) (hereinafter, “ Called YUV conversion). The gradation numbers of RGB and YUV are “0 <= RGB <= 255” and “0 <= YUV <= 255”.

Y = + 0.299 R + 0.587 G + 0.114 B Formula (1)
U =-0.168 R-0.332 G + 0.500 B + 128 Formula (2)
V = + 0.500 R-0.419 G-0.081 B + 128 Formula (3)
At this time, the mask processing unit 8 converts the data of the mask portion of the natural image into eigenvalues. For example, the mask processing unit 8 converts the data of the mask portion of the natural image into Y = U = V = 0. Y = U = V = 0 is a value that cannot exist when natural image data is subjected to YUV conversion, as is apparent from the above equation, so that it is possible to identify the data of the mask portion and the non-mask portion. .

  Thus, the natural image data in the YUV format is supplied to the mask detection unit 11. The mask detection unit 11 identifies a mask portion if the acquired image data is a unique value (for example, Y = U = V = 0), and identifies a non-mask portion if it is not a unique value. Since the processing after mask detection is the same as that described above, description thereof is omitted.

  As described above, even in the image processing method according to the second embodiment, a natural image that has been subjected to image correction (only image data of a non-masked portion) and a frame image that has not been subjected to image correction are appropriately displayed on the image display unit 3. . In the second embodiment, unlike the first embodiment, no dedicated bit is provided for identifying the natural image and the frame image. Therefore, it is not necessary to reduce the gradation value of the image data (since the mask processing in the first embodiment may reduce the number of bits indicating the gradation value in order to secure a specific bit), it is displayed. The image quality is not degraded.

  In the above description, the mask processing in which the data of the non-masked portion is Y = U = V = 0 has been described, but the present invention is not limited to this. For example, the natural image data is not subjected to YUV conversion (as it is in RGB format data), and the mask portion data is converted to R = G = B = 0. In this case, if the data of the non-mask part is originally R = G = B = 0, the data is shifted to R = G = B = 1 so as not to overlap with the image data of the mask part. Strictly speaking, this changes the gradation value of the image data. However, since only the black level is changed by one gradation, there is almost no problem visually.

  In addition, there are possible values such as YUV format (16 <= Y <= 235, 16 <= UV <= 240) such as ITU-R BT.601, etc., where the natural image is in YUV format and not a full range. If a limited format is employed, the mask portion can be identified only by setting the data of the mask portion to Y = 0.

[Third Embodiment]
Hereinafter, a third embodiment of the present invention will be described.

(Configuration of image display device)
FIG. 4 shows a schematic configuration of the image display apparatus 102 according to the third embodiment. The image display device 102 also includes an image forming unit 1, an image processing unit 2, and an image display unit 3.

  The configuration of the image forming unit 1 is the same as that shown in the first embodiment and the second embodiment. In the image forming unit 1 according to the third embodiment, the processing in the mask processing unit 8 is different from the above. First, the mask processing unit 8 performs color conversion on the image data of the non-mask portion or the mask portion so that the acquired image data of the non-mask portion and the image data of the mask portion have different color formats. For example, when the color format of the original natural image data is the RGB format, the mask processing unit 8 converts the data of the mask portion into YUV format data while keeping the data of the non-mask portion in the RGB format. Further, the mask processing unit 8 converts the image data of the mask portion into a bit number different from that of the image data of the non-mask portion. In this case, the mask processing unit 8 can simultaneously convert the color format and the number of bits when converting the color format. As described above, the data of the mask portion and the non-mask portion can be identified.

  Next, processing of the image processing unit 2 will be described. The masked natural image data is stored in the image memory 9 and read by the mask separation unit 15. The mask separation unit 15 performs mask detection on the acquired natural image data S5 (mask detection performed by the mask separation unit 15 is hereinafter referred to as “mask separation”), and further performs color conversion. First, the mask separation unit 15 separates a mask portion and a non-mask portion by looking at predetermined bit data of natural image data. Then, the mask separation unit 15 performs color conversion on the image data of the non-mask portion or the mask portion so that the color formats of the image data of the non-mask portion and the mask portion are the same. The mask separation unit 15 supplies natural image data S12 (including image data of a mask portion and a non-mask portion) S12 having the same color format to the image correction unit 12, and also sends mask instruction information S14 to the memory write control unit 13. To supply. The mask instruction information S14 is delayed by the same number of clocks required for image correction and color conversion described below.

  The image correction unit 12 performs image correction as described above on the acquired natural image data S12. The image-corrected natural image data S13 is supplied to the color conversion unit 16. The color conversion unit 16 converts the color format of the acquired natural image data S13. This is to convert the data into color format data that can be handled by the image display unit 3 that displays the image. Then, the color conversion unit 16 supplies the natural color image data S15 in the first color format to the writing control unit 13.

  The memory write control unit 13 performs control for writing the acquired natural image data S15 and frame image data S6 into the frame memory 14. In this case, the memory write control unit 13 performs write control based on the mask instruction signal S14 acquired from the mask separation unit 15. Thus, the above-described composite image data is stored in the frame memory 14 based on the write signal S10 from the memory write control unit 13. The composite image data S11 stored in the frame memory 14 is read out and displayed on the image display unit 3.

  As described above, in the image display device 102 according to the third embodiment of the present invention, natural image data including the image data of the mask portion is included without increasing the transfer amount while including the image data of the mask portion. Perform image correction. Furthermore, correction equivalent to the correction result of the original image without the image data of the mask portion can be performed on the natural image data including the image data of the mask portion. Therefore, a natural image that has been image-corrected (only image data of a non-mask portion) and a frame image that has not been image-corrected are appropriately displayed on the image display unit 3. That is, the color of the displayed frame image does not change from the original one, and conversely, the color of the natural image displayed by the influence of the frame image does not change.

(Image processing procedure)
Hereinafter, a specific image processing procedure in the third embodiment will be described with reference to FIG.

  First, a processing procedure in the image forming unit 1 will be described. The image formation processing unit 5 forms natural image data D11 in the RGB format (this color format is referred to as “first color format”). FIG. 5 shows that the natural image D11 has “8” bits for each of RGB and a total of “24” bits. The mask processing unit 8 performs mask processing and color conversion on the natural image data D11. In the third embodiment, the mask processing unit 8 color-converts the mask portion of the natural image D11 into the YUV format (this color format is referred to as “second color format”) in the YUV 4: 2: 2 format. Specifically, in the image data, two pixels in the horizontal direction are considered as a reference, Y is sampled by “8” bits for each pixel, and UV is sampled by “8” bits on average every two pixels. Since the image data of the mask portion after conversion has a data amount of “16” bits per pixel, the remaining “8” bits (for example, upper “8” bits) are set to “0”. As a result, as shown in FIG. 5, the image data of the mask portion is converted into that indicated by reference numeral D21.

  On the other hand, the image data of the non-masked portion of the natural image D11 is still RGB format data, but is converted to 255 gradations from 1 to 255 instead of 256 gradations from 0 to 255. In this case, for example, linear conversion or simply “0” pixels are converted to “1”. This is because if the data in the non-mask portion has “0”, it cannot be identified from the data in the mask portion. By such processing, the image data of the non-masked part is converted into the data indicated by the symbol D31. The natural image D12 masked as described above is stored in the image memory 9, and is read by the mask separation unit 15 of the image processing unit 2.

  Next, a processing procedure in the image processing unit 2 will be described. The mask separation unit 15 performs mask separation and color conversion on the read natural image data D12. First, the mask separation unit 15 looks at the value of “8” bits (for example, the upper “8” bits) that are not used in the YUV format data, and if the value is “0”, it is a mask portion. If it is not “0”, it is determined that it is a non-mask portion. Then, the mask separation unit 15 performs YUV conversion on the image data of the separated non-mask portion. The mask separation unit 15 supplies the image correction unit 12 with natural image data D13 including the non-mask portion data D32 and the mask portion data D22 which have been YUV converted.

  Further, the mask separation unit 15 delays the mask instruction information S14 related to the mask portion by the same number as the required number of clocks for image correction and the required number of clocks for color conversion, and supplies them to the memory writing control unit 13.

  The image correction unit 12 corrects the acquired natural image data S12. Then, the color conversion unit 16 acquires the natural image data S13 whose image has been corrected, and performs color conversion. The color converter 16 converts natural image data in YUV format (including image data of a non-mask portion and a mask portion) into RGB format data (hereinafter referred to as “RGB conversion”). As a result, natural image data D14 in the RGB format including the image data D33 of the non-mask portion and the image data D23 of the mask portion is generated. The color conversion unit 16 supplies the RGB natural image data D14 to the memory read control unit 13.

  First, the memory writing control unit 13 writes the frame image data S6 into the frame memory 14. Next, the memory writing control unit 13 writes only the data of the non-masked portion of the natural image data S15 into the frame memory 14 based on the mask instruction information S14 from the mask separating unit 15. As described above, the composite image data is stored in the frame memory 14. Thus, the composite image data is stored in the frame memory 14. The composite image data S11 stored in the frame memory 14 is read out and displayed on the image display unit 3.

  In the above description, the first color format is the RGB format and the second color format is the YUV format. However, the present invention is not limited to this. For example, the first color format may be the YUV format and the second color format may be the RGB format. That is, the color format of the image data in each processing unit is reversed between RGB and YUV. In addition, the first color format may be YUV 4: 2: 2 (total “16” bits), and the second color format may be YUV 4: 4: 4 (total “24” bits).

(Modification)
Next, a modification of the third embodiment will be described. This modification differs from the above in the color format of the image data of the non-mask portion and the image data of the mask portion, color conversion in each processing unit, and the like.

  FIG. 6 is a diagram illustrating an image processing procedure according to the modification.

  First, a processing procedure in the image forming unit 1 will be described. The image formation processing unit 5 forms natural image data E11 in YUV format. In FIG. 6, the natural image E <b> 11 has YUV “8” bits and a total of “24” bits. The mask processing unit 8 performs mask processing and color conversion on the natural image data E11. In the modification, the mask processing unit 8 converts the image data of the mask portion in the YUV format into image feature amount data in the YS format (S saturation) (hereinafter referred to as “YS conversion”). Specifically, YS is set to “16” bits, and the remaining “8” bits (for example, lower “8” bits) are set to “0”. As a result, as shown in FIG. 6, the image data of the mask portion is converted into that indicated by reference numeral E21. On the other hand, the image data of the non-masked portion of the natural image E11 remains YUV format data, but is converted to 255 gradations from 1 to 255 instead of 256 gradations from 0 to 255. As a result, the image data of the non-masked portion is converted into the data indicated by the symbol E31. The natural image E12 masked as described above is stored in the image memory 9 and read out to the mask separation unit 15 of the image processing unit 2.

  Next, a processing procedure in the image processing unit 2 will be described. The mask separation unit 15 performs mask separation and color conversion on the read natural image data E12. First, the mask separation unit 15 looks at the value of “8” bits (for example, the lower “8” bits) that are not used in the YS format data, and if the value is “0”, it is a mask portion. If it is not “0”, it is determined that it is a non-mask portion. Then, the mask separation unit 15 similarly performs YS conversion on the image data of the separated non-mask portion. In this way, image feature amount data E13 including the YS converted non-mask portion data E32 and the mask portion data E22 is generated. The mask separation unit 15 supplies the image correction unit 12 with the signal S12 corresponding to the image feature data E13 and the image data S21 of the original non-mask portion in the YUV format.

  Further, the mask separation unit 15 delays the mask instruction information S14 related to the mask portion of the natural image data by the same number as the required number of clocks for image correction and the required number of clocks for RGB conversion, and supplies them to the memory write control unit 13. .

  The image correction unit 12 analyzes the natural image data S21 based on the image feature amount data S12 in the YS format, and performs image correction. In this image analysis, for example, if the saturation S is small as the feature amount, the vividness is insufficient and the color difference UV is enhanced. Such processing is performed by the image correction unit 12 (that is, between S12 and S13) in the image processing procedure shown in FIG. As described above, the image correction unit 12 supplies the image data S22 in the YUV format subjected to the image correction to the color conversion unit 16.

  The color conversion unit 16 performs RGB conversion on natural image data in YUV format (only the non-mask portion). In this case, since the mask portion disappears as image data at the time of the first mask processing (YS conversion), there is no problem because the mask portion is not displayed and only the image feature amount can be extracted. As a result, the color conversion unit 16 generates image data E33 of a non-mask portion. The color conversion unit 16 supplies the RGB image data E33 to the memory read control unit 13.

  First, the memory writing control unit 13 writes the frame image data S6 into the frame memory 14. Next, the memory writing control unit 13 writes only the data of the non-masked portion of the natural image data S15 into the frame memory 14 based on the mask instruction information S14 from the mask separating unit 15. As described above, the composite image data is stored in the frame memory 14. The composite image data S11 stored in the frame memory 14 is read out and displayed on the image display unit 3.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described.

(Configuration of image display device)
FIG. 7 shows a schematic configuration of the image display apparatus 103 according to the fourth embodiment. The image display device 103 also includes an image forming unit 1, an image processing unit 2, and an image display unit 3.

  The image forming unit 1 includes image forming processing units 5 and 7, a composite information forming unit 6, a composite processing unit 8, and an image memory 21. The image formation processing unit 5 forms natural image data, and supplies a signal S31 corresponding to the natural image data to the synthesis processing unit 8. The image formation processing unit 7 forms frame image data, and supplies a signal S32 corresponding to the frame image data to the synthesis processing unit 8. The composite information forming unit 6 forms composite information similar to that described above, and supplies data S33 corresponding to the composite information to the composite processing unit 20.

  First, the composition processing unit 20 performs color conversion on the natural image or the frame image so that the acquired natural image and the frame image have different color formats. For example, when the color format of the original image data is the RGB format, the composition processing unit 20 converts the natural image data to the RGB format and converts the frame image data to the YUV format data. Further, the composition processing unit 20 converts the frame image data into a bit number different from that of the natural image data. In this case, the composition processing unit 8 can simultaneously convert the color format and the number of bits when converting the color format. Thereby, natural image data and frame image data can be identified. Furthermore, the composition processing unit 20 synthesizes the natural image data and the frame image data. The composition processing unit 20 stores the synthesized image data S35 in the image memory 21. As described above, the composition processing unit 20 functions as the composition unit described above in the image forming unit 1.

  The fourth embodiment is greatly different from that shown in the above-described embodiment in that the image data formed by the image forming unit 1 is transferred to the image processing unit 2 at a time. The image data S36 formed by the image forming unit 1 is supplied to the image separation unit 22 of the image processing unit 2. The image separation unit 22 separates the natural image and the frame image based on the acquired natural image data S36. The image separation unit 22 separates the natural image and the frame image based on the predetermined number of bits of the acquired image data. Then, the image separation unit 22 performs color conversion on the natural image or the frame image so that the color formats of the natural image and the frame image are the same.

  Further, the image separation unit 22 supplies the natural image data S37 to the image correction unit 12, and also supplies the frame image data S39 to the composition processing unit 23. Furthermore, the image separation unit 22 also supplies information (separated information described above, hereinafter referred to as “separation information”) S41 relating to the separation of the natural image and the frame image to the synthesis processing unit 23. Note that the frame image data S39 and the separation information S41 are delayed by the same number of clocks as required for image correction and color conversion performed on natural image data. As described above, the image separation unit 22 functions as the above-described separation unit in the image processing unit 2.

  The image correction unit 12 performs image correction on the acquired natural image data S37. Then, the natural image data S38 after the image correction is supplied to the synthesis processing unit 23. First, the composition processing unit 23 performs color conversion on the acquired natural image data S38. This is because the data is converted into data in a color format handled by the image display unit 3 that displays an image. Then, the composition processing unit 23 synthesizes the natural image data and the frame image data after color conversion based on the separation information S41. That is, the composition processing unit 23 creates composite image data. The generated composite image data S40 is supplied to the image display unit 3. Then, the image display unit 3 reads out and displays the composite image data S40.

  As described above, also in the image display device 103 according to the fourth embodiment, the natural image subjected to the image correction and the frame image not subjected to the image correction are appropriately displayed on the image display unit 3. Therefore, the color of the displayed frame image does not change from the original one, and conversely, the color of the natural image displayed by the influence of the frame image does not change. Here, in the image processing according to the fourth embodiment, unlike the first to third embodiments, natural image data and frame image data are simultaneously transferred from the image forming unit 1 to the image processing unit 2. doing. That is, one image data obtained by combining natural image data and frame image data is transferred. Thereby, even when the natural image and the frame image are moving images, the transfer amount does not increase. Furthermore, the image processing method according to the fourth embodiment can appropriately cope with the case where the image processing unit 2 does not have a frame memory or the like and the frame image and the natural image cannot be separately transferred and superimposed.

(Image processing method)
Hereinafter, a specific image processing procedure in the fourth embodiment will be described with reference to FIG.

  First, a processing procedure in the image forming unit 1 will be described. The image formation processing unit 5 forms natural image data F11 in the RGB format (this color format is referred to as “first color format”). The image forming processor 7 forms frame image data F21 in RGB format. FIG. 8 shows that the natural image F11 and the frame image F21 are “8” bits for each of RGB, and the total is “24” bits. For the natural image data F11, the composition processing unit 20 is still in RGB format data, but for the same reason as described above, the composition processing unit 20 converts the natural image data F11 to 255 gradations of 1 to 255 instead of 256 gradations of 0 to 255. In addition, the composition processing unit 20 performs color conversion on the frame image F21 in the YUV 4: 2: 2 format to the YUV format (this color format is referred to as “second color format”). Since the converted frame image data has a data amount of “16” bits per pixel, the composition processing unit 20 sets the remaining “8” bits (for example, upper “8” bits) to “0”. The natural image data F13 and the frame image data F23 that have been converted as described above are combined into image data indicated by reference numeral F31. The image data F31 synthesized as described above is stored in the image memory 21 and read out to the image separation unit 22 of the image processing unit 2.

  Next, a processing procedure in the image processing unit 2 will be described. The image separation unit 22 performs image separation and color conversion on the read image data F31. First, the image separation unit 22 looks at the value of “8” bits that are not used in the YUV format data. If the value is “0”, the image separation unit 22 determines that the frame image is not “0”. Judged as a natural image. Then, the image separation unit 22 performs YUV conversion on the separated natural image data. The image separation unit 22 supplies the natural image data F14 subjected to YUV conversion to the image correction unit 12. Further, the image separation unit 22 supplies the frame image data S39 and the separation information S41 to the synthesis processing unit 23. The frame image data S39 and the separation information S41 are delayed by the same number of clocks as required for image correction and color conversion performed on natural image data.

  The image correction unit 12 corrects the acquired natural image data S37. Then, the image correction unit 12 supplies the natural image data S38 after the image correction to the synthesis processing unit 23. The composition processing unit 23 first performs color conversion on the acquired natural image data S38. That is, the composition processing unit 23 converts the natural image data S38 in the YUV format into RGB format data. Then, the composition processing unit 23 synthesizes the frame image data S39 and the natural image data after color conversion based on the acquired separation information S41. As a result, composite image data F32 including the natural image data F15 and the frame image data F25 is created. The image display unit 3 reads and displays the signal S40 corresponding to the composite image data F32.

  In the above description, the first color format is the RGB format and the second color format is the YUV format. However, the present invention is not limited to this. For example, the first color format may be the YUV format and the second color format may be the RGB format. That is, the color format of the image data in each processing unit is reversed between RGB and YUV. In addition, the first color format may be YUV 4: 2: 2 (total “16” bits), and the second color format may be YUV 4: 4: 4 (total “24” bits).

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described.

  The configuration of the image display device according to the fifth embodiment is the same as that of the image display device 103 shown in the fourth embodiment. That is, natural image data and frame image data are simultaneously transferred from the image forming unit 1 to the image processing unit 2. Hereinafter, an image processing procedure according to the fifth embodiment will be described.

  FIG. 9 shows a specific image processing procedure in the fifth embodiment.

  First, a processing procedure in the image forming unit 1 will be described. The image forming processor 5 forms natural image data G11 in RGB format. The natural image data G11 is data of “24” bits in total of “8” bits for each of RGB. In the fifth embodiment, the image formation processing unit 7 forms a total of k-bit frame image data G21 in RGB format (16 <= k <24). FIG. 9 shows an example in which k = “18” bits. The image processing according to the fifth embodiment differs from that shown in the fourth embodiment in that the number of bits of the natural image and the frame image are different.

First, the composition processing unit 20 performs YUV conversion on the natural image data G21. At this time, the composition processing unit 20 converts the conversion to U not with 256 gradations of 0 to 255 (in the case of “8” bits), but with (256-m) gradations of gradation values m to 255 gradation values 255. (Y and V remain normal). Here, m = 2 ^(k−16) , and m = “4” in the example shown in FIG. As described above, the natural image data becomes data indicated by the symbol G12. For the YUV conversion performed on the natural image data G21, for example, the following equations (4) to (6) can be used.

Y = +0.299 R + 0.587 G + 0.114 B Formula (4)
U = (255−m) / 255 × (−0.168 R−0.332 G + 0.500 B + 128) + m Formula (5)
V = +0.500 R-0.419 G-0.081 B + 128 Formula (6)
On the other hand, the composition processing unit 20 uses a total of “16” bits of “Y” and “V” of YUV and a lower (k−16) bit range of “U” for the frame image data G21. It is converted into an RGB format arrangement with a total of “k” bits. The upper (24-k) bits of “U” in the frame image data G21 are set to “0”. As a result, the frame image data becomes data indicated by the symbol G22. The natural image data G13 and the frame image data G23 that have been converted as described above are combined into image data indicated by reference numeral G31. The image data G31 synthesized in this way is stored in the image memory 21 and read out to the image separation unit 22 of the image processing unit 2.

  Next, a processing procedure in the image processing unit 2 will be described. The image separation unit 22 performs image separation on the read image data F31. The image separation unit 22 determines that the frame is a frame image when the upper (24-k) bits of “U” are “0”, and determines that the image is a natural image when it is not “0”. Then, the image separation unit 22 supplies the natural image data S37 subjected to the image separation to the image correction unit 12. Further, the image separation unit 22 supplies the frame image data S39 and the separation information S41 obtained by the image separation to the synthesis processing unit 23. At this time, the frame image data S39 and the separation information S41 are delayed by the same number of clocks required for image correction and color conversion performed on natural image data.

  The image correction unit 12 corrects the acquired natural image data S37. Then, the image correction unit 12 supplies the natural image data S38 after the image correction to the synthesis processing unit 23. The composition processing unit 23 first performs color conversion on the acquired natural image data S38. The composition processing unit 23 converts the natural image data S38 in the YUV format into RGB format data. At this time, the composition processing unit 23 performs RGB conversion that expands to 256 gradations (0 to 255) similar to the original natural image data G11. For example, the composition processing unit 23 performs RGB conversion according to the following formulas (7) to (10).

U ′ = 255 / (255−m) × (U−m) Equation (7)
R = Y + 1.402 (V-128) Formula (8)
G = Y-0.344 (U'-128)-0.714 (V-128) Equation (9)
B = Y + 1.772 (U'-128)-0.001 (V-128) Formula (10)
The synthesis processing unit 23 synthesizes the frame image data S39 and the natural image data after color conversion based on the acquired separation information S41. As a result, composite image data G32 composed of the natural image data G14 and the frame image data G24 is created. Then, the image display unit 3 reads and displays the signal S40 corresponding to the composite image data G32.

  As described above, the image processing method according to the fifth embodiment appropriately displays an image-corrected natural image and a non-image-corrected frame image even when the frame image has a small number of gradations. Part 3 can be displayed. Since the number of gradations of the frame image does not necessarily need to be large, the image processing method according to the fifth embodiment effectively performs image processing using the fact that the number of gradations of the frame image is small.

  Note that if there is no “m” gradation from U = 0 to U = (m−1) from the beginning, there is no need for gradation compression, so YUV conversion performed on the image data (formulas (4) to (4)). (6)) and RGB conversion (formulas (7) to (10)) may be performed using ordinary conversion formulas. For example, in the case of a YUV format (16 <= Y <= 235, 16 <= UV <= 240) that is not the full range of "8" bits, such as ITU-R BT.601, it is not necessary to perform gradation compression on natural image data. Therefore, a section outside this range can be used for the frame image. For example, if it is known that U = 0 to 15 does not exist (that is, the upper 4 bits of U are always “0001” to “1111”), the upper 4 bits of U = “0000”. Certain data can be used as a frame image (ie, corresponding to the above-mentioned k = 20).

Further, when the gradation value of the natural image data is not 0 side or 255 side but has a halftone, when the number of bits of the frame image = “k” bits (16 <= k <24), the natural image data If there is an empty gradation area of 2 ^(k-24) gradations, it can also be used as a frame image area. For example, when k = 18, 2 bits are allocated to “U” (which may be “Y” or “V”), but in this case, 2 ² = 4 gradations are required. Therefore, in the case of natural image data as described below, when the upper “6” bit = “101101”, it can be a frame image, and other than that can be a natural image. Thereby, a frame image can be arranged without gradation-compressing a natural image. Therefore, no natural image restoration (gradation expansion / contraction) is performed, and no error occurs.

“1011 0011”; a pixel with this gradation value exists “1011 0100”; since there is no pixel with this gradation value, the lower 2 bits are used for the frame “1011 0101”; there is no pixel with this gradation value Therefore, the lower 2 bits are used for the frame “1011 0110”; since there is no pixel of this gradation value, the lower 2 bits are used for the frame “1011 0111”; since the pixel of this gradation value does not exist, the lower 2 bits are framed “1011 1000” used for the pixel having the gradation value exists. In addition, the same thing as described above can be performed by decoding the higher-order pixels other than the continuous four gradations. For example, a frame image can be arranged as follows. Specifically, if the upper 7 bits are all “0”, “0” and the least significant bit are combined and “00” or “01” is used for the frame image. If all the upper 7 bits are “1”, “10” or “11” is used for the frame image by combining “1” and the least significant bit.
“0000 0000”; upper 7 bits are “0”, 2 bits for frame image = “00”
“0000 0001”; upper 7 bits are “0”, 2 bits for frame image = “01”
“0000 0010”; natural image
“1111 1101”; natural image “1111 1110”; upper 7 bits are “1”, 2 bits for frame image = “10”
“1111 1111”; upper 7 bits are “1”, 2 bits for frame image = “11”

1 is a diagram illustrating a schematic configuration of an image display device according to a first embodiment of the present invention. It is a figure which shows the image processing procedure which concerns on 1st Embodiment of this invention. It is a figure which shows the image processing procedure which concerns on 2nd Embodiment of this invention. It is a figure which shows schematic structure of the image display apparatus which concerns on 3rd Embodiment of this invention. It is a figure which shows the image processing procedure which concerns on 3rd Embodiment of this invention. It is a figure which shows the image processing procedure which concerns on the modification of 3rd Embodiment. It is a figure which shows schematic structure of the image display apparatus which concerns on 4th Embodiment of this invention. It is a figure which shows the image processing procedure which concerns on 4th Embodiment of this invention. It is a figure which shows the image processing procedure which concerns on 5th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image formation part, 2 Image processing part, 3 Image display part, 5, 7 Image formation process part, 8 Mask processing part, 11 Mask detection part, 12 Image correction part, 13 Memory write control part, 14 Frame memory, 101, 102, 103 Image display device

Claims (9)

An image processing apparatus including an image forming unit and an image processing unit,
The image forming unit includes:
Image forming processing means for forming first image data and second image data;
Mask processing means for performing mask processing on the first image data so that a portion where the first image data and the second image data overlap can be identified;
Transfer means for separately transferring the masked first image data and the second image data to the image processing unit,
The image processing unit
Mask detection means for detecting the overlapping portion in the transferred first image data;
Image correcting means for correcting the first image data or the first image data other than the overlapping portion;
An image processing apparatus comprising: means for superimposing the image-corrected first image data and the second image data based on the mask detection result. The mask processing means adds data for identifying the overlapping portion to a specific bit of the first image data,
The image processing apparatus according to claim 1, wherein the mask detection unit detects the overlapping portion based on a value of the specific bit. The mask processing means makes the image data of the overlapping portion a unique value,
2. The image processing apparatus according to claim 1, wherein if the transferred first image data is an eigenvalue, the mask detection unit determines that the image data is the overlapping portion. The mask processing means converts the number of bits and color format of the image data of the overlapping portion,
The image processing apparatus according to claim 1, wherein the mask detection unit detects the overlapping portion based on predetermined bit data of the transferred first image data. An image processing apparatus including an image forming unit and an image processing unit,
The image forming unit includes:
Image forming processing means for forming first image data and second image data;
A combining unit that converts a color format of the first image data or the second image data and combines the first image data and the second image data;
Transfer means for transferring the combined image data to the image processing unit,
The image processing unit
Separating means for separating the transferred image data into the first image data and the second image data;
Image correction means for correcting the image of the separated first image data;
An image processing apparatus comprising: means for superimposing the image-corrected first image data and the separated second image data. The synthesizing unit converts the number of bits and the color format of the first image data or the second image data,
6. The image processing apparatus according to claim 5, wherein the separation unit separates the transferred image data based on predetermined bit data. The image forming processing unit forms second image data having a smaller number of bits than the first image data;
The synthesizing unit adds a predetermined value to the insufficient number of bits of the second image data, and rearranges the second image data;
6. The image processing apparatus according to claim 5, wherein the separation unit separates the transferred image data based on predetermined bit data. An image processing method executed in an image processing apparatus including an image forming unit and an image processing unit,
An image forming process for forming the first image data and the second image data;
A mask processing step for performing mask processing on the first image data so that a portion where the first image data and the second image data overlap can be identified;
A transfer step of separately transferring the masked first image data and the second image data from the image forming unit to the image processing unit,
A mask detection step for detecting the overlapping portion in the transferred first image data;
An image correction step of performing image correction on the first image data or the first image data other than the overlapping portion;
An image processing method comprising: superimposing the image-corrected first image data and the second image data on the basis of the mask detection result. An image processing method executed in an image processing apparatus including an image forming unit and an image processing unit,
An image forming process for forming the first image data and the second image data;
Converting a color format of the first image data or the second image data, and combining the first image data and the second image data;
A transfer step of transferring the combined image data from the image forming unit to the image processing unit,
A separation step of separating the transferred image data into the first image data and the second image data;
An image correction step of performing image correction on the separated first image data;
An image processing method comprising: superimposing the image-corrected first image data and the separated second image data.

Images