JP2010041571A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for improving the following performance of a dynamic correction even if a scene change occurs in an input video image. <P>SOLUTION: The image processing apparatus is configured so that a frame rate conversion section obtains a motion vector indicating a change in video contents for each of frames forming the input video image, a scene change detecting section detects a scene change indicating an abrupt change in the input video image based on the obtained motion vector, an LUT-DB modifying section modifies contents of a correction database when the scene change is detected, and a correcting section corrects colors of the input video image based on the correction database whose details are changed. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for correcting an input video based on a correction database obtained by accumulating correction data generated from an input video, and in particular, even when a scene change occurs in the input video. The present invention relates to an image processing apparatus that can improve the follow-up performance of the optical correction.

  In recent years, in-vehicle DTV (digital television) receivers have become widespread. The DTV broadcast wave conforms to the OFDM (Orthogonal Frequency Division Multiplexing) system, and so-called OFDM demodulation is performed when information is demodulated from the DTV broadcast wave.

  In the case of the above-described in-vehicle DTV receiver, since the reception environment constantly changes according to the speed and direction of the vehicle and the environment around the vehicle, the image of the DTV is likely to deteriorate. For this reason, as a technique for improving the image quality, a technique has been proposed in which correction data obtained by analyzing an input video is accumulated in a database and the input video is dynamically corrected while referring to the database. .

  Specifically, correction data called LUT (Look Up Table) is generated by analyzing each frame image constituting the input video, and the generated LUT is stored in the LUT database (LUT-DB). Then, the input video is dynamically corrected by applying the time series LUT stored in the LUT-DB to the input video.

  In Patent Document 1, as a technique related to image compression, a scene change is detected while using a predicted image predicted from a past image at a normal time when there is no scene change (indicating that the video content changes suddenly). In this case, a technique is disclosed in which the latest image, not the predicted image, is used to achieve both improvement in compression efficiency and prevention of image quality deterioration due to an increase in prediction error at the time of a scene change.

JP 9-238353 A

  However, when a method of dynamically correcting an input video using LUT-DB is used, there is a problem that image quality deterioration occurs due to deterioration in tracking ability of dynamic correction when a scene change occurs. This is because the LUT-DB stores correction data corresponding to the video before the scene change, and even if the correction data corresponding to the video before the scene change is applied to the video after the scene change. This is because a predetermined time is required until an effect appears.

  Since the technique of Patent Document 1 is a technique in the field of image compression, even if the technique of Patent Document 1 is used to cope with a scene change, it is difficult to improve the follow-up performance of dynamic correction. .

  For these reasons, it is a major issue how to realize an image processing apparatus capable of improving the follow-up performance of dynamic correction even when a scene change occurs in the input video.

  The present invention has been made to solve the above-described problems caused by the prior art, and can improve the follow-up performance of dynamic correction even when a scene change occurs in an input video. The purpose is to provide.

  In order to solve the above-described problems and achieve the object, the present invention is an image processing apparatus that corrects the input video based on a correction database obtained by accumulating correction data generated from the input video. A motion vector acquisition unit that acquires a motion vector indicating a motion of video content for each frame constituting the input video, and a scene that shows a sudden change in the input video based on the motion vector acquired by the motion vector acquisition unit A scene change detection means for detecting a change, a change means for changing the contents of the correction database when a scene change is detected by the scene change detection means, and a correction database whose contents have been changed by the change means. And correction means for correcting the input video based on the input image.

  According to the present invention, a motion vector indicating a motion of video content is acquired for each frame constituting an input video, a scene change indicating a sudden change in the input video is detected based on the acquired motion vector, and a scene change is detected. Is detected, the content of the correction database is changed, and the input video is corrected based on the correction database whose content has been changed, so even if a scene change occurs in the input video, There is an effect that it is possible to improve the follow-up performance of the automatic correction.

  Exemplary embodiments of an image processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, an outline of an image processing method according to the present invention will be described with reference to FIG. 1, and then an embodiment of an image processing apparatus to which the image processing method according to the present invention is applied will be described.

  First, the outline of the image processing method according to the present invention will be described with reference to FIG. FIG. 1 is a diagram showing an outline of an image processing method according to the present invention. The image processing method according to the present invention detects a scene change based on a motion vector obtained by image analysis of an input video, and uses an LUT-DB (Look-DB) according to an average value of the number and size of the motion vectors. By changing the contents of (Up Table Data Base), the follow-up performance of dynamic correction at the time of a scene change is improved. FIG. 1 shows a case where the contents of the LUT-DB are changed based on the number of motion vectors.

  Conventionally, when dynamic correction using LUT-DB is performed, even when a scene change occurs, dynamic correction is performed using correction data (LUT) before scene change included in the LUT-DB. Was done. For this reason, appropriate correction is not performed by dynamic correction for a while after the scene change, resulting in image quality degradation.

  Therefore, in the image processing method according to the present invention, motion vectors are used for determination of scene change detection, and the contents of the LUT-DB are changed based on the average number and size of motion vectors.

  Here, the motion vector indicates the direction and size of movement of the partial image included in the predetermined image. The motion vector is calculated for each rectangular block obtained by dividing a predetermined image into a grid. That is, the number of motion vectors is the number of rectangular blocks in which motion vectors are detected, and the average value of motion vector sizes is the average value of motion vector sizes calculated for each rectangular block. Point to.

  Specifically, the image processing method according to the present invention compares the number of detected motion vectors with a predetermined threshold A and threshold B. Note that the threshold A is a threshold larger than the threshold B. As shown in the figure, when the number of motion vectors is larger than a predetermined threshold, normal LUT correction processing is performed (see (1) in the figure). Here, the normal LUT correction process refers to a correction process applied to an input video using a normal weighting value for the LUT accumulated in the LUT-DB.

  If the number of motion vectors is equal to or less than the threshold A and greater than the threshold B, the LUT weighting value is changed (see (2) in the figure). Here, the weighting value refers to a weighting factor to be multiplied to each time-series LUT, and takes a value of 0 to 1.0, for example. Usually, the weighting value corresponding to the immediately preceding image is the largest, and adjustment is performed so that the weighting value becomes smaller as the past image.

  If the number of motion vectors is equal to or less than the threshold value B, the contents of the LUT-DB are deleted (see (3) in the figure). When the contents of the LUT-DB are deleted, all accumulated LUTs are deleted. After the deletion, the LUT is accumulated in the LUT-DB as time elapses.

  As described above, in the image processing method according to the present invention, on the premise that it is difficult to accurately detect a scene change, a motion vector is used as an index representing the likelihood of the scene change. Then, the change processing for the LUT-DB is changed in multiple stages by comparing the average value of the number or size of motion vectors with a predetermined threshold value.

  That is, when a scene change is strongly predicted, the contents of the LUT-DB are deleted to improve the followability of the dynamic correction after the scene change. When the occurrence of a scene change is uncertain, the LUT -By changing the weight value of each LUT included in the DB, it was decided to make a gentle correction change. In this way, it is possible to improve the followability of dynamic correction even when a scene change occurs in the input video and to prevent frequent deletion of the contents of the LUT-DB. Can do.

  Below, the Example about the image processing apparatus to which the image processing method which concerns on this invention is applied is described. In the following embodiments, an image processing apparatus provided in a moving body such as an automobile will be described. However, the present invention is applied to a receiving apparatus installed indoors or outdoors, and a mobile terminal such as a mobile phone. It is good.

  FIG. 2 is a block diagram illustrating the configuration of the image processing apparatus 10 according to the present embodiment. As shown in FIG. 1, the image processing apparatus 10 is connected to an antenna and a display apparatus 100 and includes a demodulation unit 11, a decoding unit 12, and a color correction processing unit 13.

  The color correction processing unit 13 includes a frame rate conversion unit 13a, a thumbnail generation unit 13b, an LUT (Look Up Table) storage unit 13c, an LUT-DB (Look Up Table Data Base) 13d, and a scene change detection unit. 13e, a LUT-DB changing unit 13f, and a correcting unit 13g.

  Here, the antenna is installed, for example, in a roof portion or a window portion of the vehicle, and receives a digital television (DTV) broadcast wave. Then, the received signal is output to the image processing apparatus 10. The display device 100 is a display device such as a liquid crystal display, and displays an output video from the image processing device 10.

  The demodulator 11 performs a process of demodulating the signal received from the antenna and passing the demodulated data to the decoder 12. In addition, the decoding unit 12 decodes the demodulated data received from the demodulation unit 11 to reproduce the input video and pass the reproduced input video to the frame rate conversion unit 13 a of the color correction processing unit 13. It is a processing part to perform.

  The frame rate conversion unit 13a includes a frame rate conversion circuit and the like, and performs a process of converting the frame rate of the input video received from the decoding unit 12. The frame rate conversion unit 13a also performs a process of generating a motion vector by comparing time-series frames. Here, the motion vector generated by the frame rate conversion unit 13a will be described with reference to FIG.

  FIG. 3 is a diagram showing motion vectors. In (1) of the figure, the motion vector is detected in most blocks in the input video, and in (2) in the same figure, the motion vector is not detected in most blocks in the input video. Each case is shown.

  In the case shown in (1) of FIG. 3, a rightward motion vector is detected in most blocks. It is assumed that the magnitude of the motion vector 31a in the block 31 is L1, and the magnitude of the motion vector 32a in the block 32 is L2.

  It is assumed that the motion vector itself is not detected in the block 33. FIG. 3 (1) shows 15 blocks. Since 14 of these blocks detect motion vectors, the number of motion vectors is 14.

  In addition, the average value of the motion vectors in the figure is the sum of the motion vectors (for example, 31a and 32a) (for example, L1 and L2) and divided by the total number of blocks (for example, 15). It is obtained by doing.

  On the other hand, in the case shown in (2) of FIG. 3, the motion vector 34a is detected in the block 34, but no motion vector is detected in the blocks other than the block 34, so (2) of FIG. The number of motion vectors in is 1.

  Here, when the average value of the motion vectors exceeds a predetermined threshold and when the number of motion vectors falls below the predetermined threshold, there has been a sudden video change, that is, a scene change has occurred. It is expected that. The scene change detection unit 13e described later receives the average value of the number of motion vectors and / or the magnitude of the motion vector from the frame rate conversion unit 13a, and detects the scene change by comparing the received value with a predetermined threshold value.

  Returning to the description of FIG. 2, the thumbnail generation unit 13 b will be described. The thumbnail generation unit 13b is a processing unit that receives the input video after the frame rate conversion from the frame rate conversion unit 13a and performs a process of generating a thumbnail image smaller than the frame image constituting the input video. Then, the thumbnail generation unit 13b outputs the generated thumbnail images to the LUT storage unit 13c.

  The LUT accumulation unit 13c performs image analysis on each thumbnail image received from the thumbnail generation unit 13b, thereby generating correction data (LUT) for performing dynamic color correction, and the generated LUT is stored in the LUT-DB 13d. It is a processing part which performs processing to accumulate.

  The LUT-DB 13d is a database that accumulates LUTs received from the LUT accumulation unit 13c in time series. The LUT-DB 13d performs a predetermined weighting on each LUT and provides it to the correction unit 13g. The LUT-DB 13d also performs processing for deleting the accumulated LUT and changing the weighting value for each LUT in accordance with an instruction from the LUT-DB changing unit 13f.

  Here, the weight values in the LUT-DB 13d will be described with reference to FIG. FIG. 4 is a diagram illustrating weight values in the LUT-DB 13d. Note that (1) in the figure shows a graph of weight values in normal times (see 41 in the figure), and (2) and (3) in the figure show graphs of weight values when the weight is lowered. (See 42 and 43 in the figure), respectively. Further, in (1) to (3) in the figure, the vertical axis represents the weight value for each frame, and the horizontal axis represents the time-series frame (see the vertical broken line in the figure).

  As shown in (1) of the figure, in normal times, the weighting value corresponding to the current frame (see 0 on the horizontal axis) is set to 0, and then the previous frame (see -1 on the horizontal axis) is handled. Each weighting value is adjusted so that the weighting value to be given is the largest and the weighting value becomes smaller in the past frames. In the figure, the case where the weighting value is 0 in the 10th previous frame (see −10 on the horizontal axis) is illustrated.

  Here, the LUT-DB 13d changes the graph 41 to the graph 42 or the graph 43 when receiving an instruction to change the weighting value from the LUT-DB changing unit 13d. For example, as shown in (2) of the figure, the weighting value corresponding to the immediately preceding frame (refer to -1 on the horizontal axis) is smaller than that of the graph 41, and the 10th previous frame (horizontal axis as in the graph 41). The graph 42 is changed to a graph 42 in which the weighting value is 0.

  As shown in (3) of the figure, the weighting value corresponding to the immediately preceding frame (refer to -1 on the horizontal axis) is made smaller than that of the graph 41, for example, the 5th previous frame (− on the horizontal axis). 5)) may be changed to the graph 43 in which the weighting value becomes zero. Moreover, although FIG. 4 shows a case where the weighting decreases linearly, it may be decreased along a predetermined curve.

  Returning to the description of FIG. 2, the scene change detection unit 13 e will be described. The scene change detection unit 13e is a processing unit that performs a process of detecting a scene change of the input video based on the information regarding the motion vector received from the frame rate conversion unit 13a.

  Specifically, the scene change detection unit 13e compares the number of motion vectors or the average value of the motion vectors with a predetermined threshold value to determine whether to instruct the LUT-DB changing unit 13f. In the case of giving an instruction, it is determined whether to give an instruction to delete the contents of the LUT-DB 13d or to give an instruction to change the weight value in the LUT-DB changing unit 13f.

  Here, the contents of the instruction given by the scene change detection unit 13e to the LUT-DB changing unit 13f will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of the operation of the image processing apparatus 10 according to the number of motion vectors.

  In the figure, the vertical axis represents the number of detected motion vectors, and the horizontal axis represents the passage of time, and shows a curve 51 when the number of motion vectors decreases with the passage of time. Note that the threshold value A shown in the figure is larger than the threshold value B. In the following, description will be given along the passage of time (from the left side to the right side in the figure).

  As shown in the figure, when the number of motion vectors exceeds the threshold A, the scene change detection unit 13e does not give an instruction to the LUT-DB change unit 13f. Therefore, in this case, normal correction using the LUT stored in the LUT storage unit 13c and the normal weighting value is performed by the correction unit 13g described later.

  If the number of motion vectors is equal to or less than the threshold A (see 51a in the figure), the scene change detection unit 13e instructs the LUT-DB changing unit 13f to change the weighting value. In this case, as the weighting value, a weighting value having a smaller value than the normal weighting value is used, such as 42 and 43 in FIG.

  Note that a section in which the number of motion vectors is equal to or less than the threshold A and greater than the threshold B is shown as the LUT weighting change section. In this section, the LUT-DB changing unit 13f is only used once in 51a in FIG. It is good also as giving the change instruction of a weighting value, and it is good also as giving a change instruction in multiple times according to the change of the number of motion vectors.

  For example, when a change instruction is made a plurality of times, the LUT-DB changing unit 13f can be instructed to further reduce the weighting value as the number of motion vectors decreases.

  Subsequently, if the number of motion vectors is equal to or less than the threshold value B (see 51b in the figure), the scene change detection unit 13e instructs the LUT-DB changing unit 13f to delete the contents of the LUT-DB 13d. Therefore, in this case, all the LUTs accumulated in the LUT-DB 13d are deleted.

  Returning to the description of FIG. 2, the LUT-DB changing unit 13f is a processing unit that performs a process of changing the LUT-DB 13d in accordance with an instruction from the scene change detecting unit 13e. Specifically, the LUT-DB changing unit 13f changes the weighting value corresponding to each LUT stored in the LUT-DB 13d when receiving a weighting value changing instruction (for example, 42 or FIG. 4). 43). When receiving an instruction to delete the contents of the LUT-DB 13d, all the LUTs stored in the LUT-DB 13d are deleted.

  The correction unit 13g is a processing unit that performs correction using the LUT-DB 13d on the input video after frame rate conversion received from the frame rate conversion unit 13a. When the LUT-DB 13d is changed by the scene change detection unit 13e and the LUT-DB changing unit 13f, the correcting unit 13g performs correction processing using the changed LUT-DB 13d. .

  Next, a processing procedure when scene change detection is performed using the number of motion vectors will be described with reference to FIG. FIG. 6 is a flowchart illustrating a processing procedure executed by the image processing apparatus. Note that the threshold value A shown in the figure is larger than the threshold value B as in FIG.

  As shown in the figure, when an image is input to the color correction processing unit 13 (step S101), the frame rate conversion unit 13a determines whether reception for one frame is completed (step S102). When reception for one frame is completed (Yes in step S102), a motion vector is generated (step S103). If the determination condition in step S102 is not satisfied (No in step S102), the process in step S102 is repeated.

  Subsequently, the scene change detection unit 13e determines whether or not the number of motion vectors is smaller than the threshold value B (step S104). If the number of motion vectors is smaller than the threshold value B (step S104, Yes), the LUT The DB changing unit 13f deletes the contents of the LUT-DB 13d (Step S105), and repeats the processes after Step S101.

  On the other hand, when the determination condition of step S104 is not satisfied (step S104, No), it is determined whether the number of motion vectors is smaller than the threshold A (step S106), and the number of motion vectors is smaller than the threshold A. In this case (step S106, Yes), the LUT-DB changing unit 13f changes the weight of the LUT in the LUT-DB 13d (step S107), and repeats the processing after step S101.

  If the determination condition of step S106 is not satisfied (step S106, No), the LUT-DB changing unit 13f does not change the LUT-DB 13d (step S108), and repeats the processes after step S101.

  By the way, in FIG. 5 and FIG. 6, although the case where the scene change detection unit 13e gives an instruction to the LUT-DB changing unit 13f based on the number of motion vectors has been described, it is also possible to use an average value of the magnitudes of motion vectors. Good. Therefore, hereinafter, a case where the scene change detection unit 13e instructs the LUT-DB change unit 13f based on the average value of the motion vectors will be described with reference to FIGS. 7 and 8. FIG.

  FIG. 7 is a diagram illustrating an example of the operation of the image processing apparatus according to the average value of the magnitudes of motion vectors. The vertical axis in the figure represents the average value of detected motion vectors (hereinafter referred to as “motion vector average value”), and the horizontal axis represents the passage of time. A curve 71 when the number of motion vectors increases is shown. In addition, the threshold value α shown in the figure is assumed to be larger than the threshold value β. In the following, description will be given along the passage of time (from the left side to the right side in the figure).

  As shown in the figure, when the number of motion vectors is equal to or less than the threshold value β, the scene change detection unit 13e does not give an instruction to the LUT-DB change unit 13f. Therefore, in this case, normal correction using the LUT accumulated by the LUT accumulation unit 13c and the normal weighting value is performed by the correction unit 13g.

  If the number of motion vectors exceeds the threshold β (see 71a in the figure), the scene change detection unit 13e instructs the LUT-DB changing unit 13f to change the weighting value. In this case, as the weighting value, a weighting value having a smaller value than the normal weighting value is used, such as 42 and 43 in FIG.

  Note that the section in which the number of motion vectors is greater than the threshold β and less than or equal to the threshold α is shown as the LUT weighting change section. A value change instruction may be given, or a change instruction may be given multiple times in accordance with a change in the motion vector average value.

  For example, when a change instruction is made a plurality of times, the LUT-DB changing unit 13f can be instructed to further reduce the weighting value as the motion vector average value increases.

  Subsequently, if the number of motion vectors exceeds the threshold value α (see 71b in the figure), the scene change detection unit 13e instructs the LUT-DB changing unit 13f to delete the contents of the LUT-DB 13d. Therefore, in this case, all the LUTs accumulated in the LUT-DB 13d are deleted.

  FIG. 8 is a flowchart illustrating a modification of the processing procedure executed by the image processing apparatus. It is assumed that the threshold value α shown in the figure is larger than the threshold value β as in FIG. As shown in the figure, when an image is input to the color correction processing unit 13 (step S201), the frame rate conversion unit 13a determines whether reception for one frame is completed (step S202).

  When reception for one frame is completed (step S202, Yes), a motion vector is generated (step S203). If the determination condition in step S202 is not satisfied (No in step S202), the process in step S202 is repeated.

  Subsequently, the scene change detection unit 13e determines whether or not the motion vector average value is larger than the threshold value α (step S204). If the motion vector average value is larger than the threshold value α (step S204, Yes). The LUT-DB changing unit 13f deletes the contents of the LUT-DB 13d (Step S205), and repeats the processes after Step S201.

  On the other hand, when the determination condition of step S204 is not satisfied (step S204, No), it is determined whether the motion vector average value is larger than the threshold value β (step S206), and the motion vector average value is higher than the threshold value β. Is larger (step S206, Yes), the LUT-DB changing unit 13f changes the weight of the LUT in the LUT-DB 13d (step S207), and repeats the processing after step S201.

  If the determination condition in step S206 is not satisfied (No in step S206), the LUT-DB changing unit 13f does not change the LUT-DB 13d (step S208) and repeats the processes in and after step S201.

  As described above, in the present embodiment, the frame rate conversion unit acquires a motion vector indicating the motion of the video content for each frame constituting the input video, and the scene change detection unit is based on the acquired motion vector. When a scene change indicating a sudden change in the input video is detected, the LUT-DB changing unit changes the content of the correction database when the scene change is detected, and the correction unit changes the content of the correction database. The image processing apparatus is configured to perform color correction of the input video based on the above. Therefore, the followability of dynamic correction can be improved even when a scene change occurs in the input video.

  As described above, the image processing apparatus according to the present invention is useful when it is desired to improve the follow-up performance of dynamic correction even when a scene change occurs in an input video, and in particular, a moving body such as an automobile. It is suitable for the environment where the reception situation is easy to change.

It is a figure which shows the outline | summary of the image processing method based on this invention. It is a block diagram which shows the structure of the image processing apparatus which concerns on a present Example. It is a figure which shows a motion vector. It is a figure which shows the weighting value in LUT-DB. It is a figure which shows an example of operation | movement of the image processing apparatus according to the number of motion vectors. It is a flowchart which shows the process sequence which an image processing apparatus performs. It is a figure which shows an example of operation | movement of the image processing apparatus according to the average value of the magnitude | size of a motion vector. It is a flowchart which shows the modification of the process sequence which an image processing apparatus performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Demodulation part 12 Decoding part 13 Color correction processing part 13a Frame rate conversion part 13b Thumbnail generation part 13c LUT accumulation | storage part 13d LUT-DB
13e Scene change detection unit 13f LUT-DB change unit 13g Correction unit 100 Display device

Claims (4)

An image processing apparatus for correcting the input video based on a correction database obtained by accumulating correction data generated from the input video,
Motion vector acquisition means for acquiring a motion vector indicating the motion of video content for each frame constituting the input video;
Scene change detection means for detecting a scene change indicating a sudden change in the input video based on the motion vector acquired by the motion vector acquisition means;
Changing means for changing the contents of the correction database when a scene change is detected by the scene change detecting means;
An image processing apparatus comprising: correction means for correcting the input video based on the correction database whose contents have been changed by the changing means. The changing means is
The image processing apparatus according to claim 1, wherein when the scene change is detected by the scene change detection unit, all the correction data stored in the correction database are deleted. The changing means is
The image processing apparatus according to claim 1, wherein when a scene change is detected by the scene change detection unit, a weighting value corresponding to each of the correction data stored in the correction database is changed. The changing means is
4. The image processing apparatus according to claim 3, wherein when a scene change is detected by the scene change detection unit, the weighting value is changed based on the motion vector at the time of the scene change detection.

Images