JP2001024988A - Image signal motion compensation frame number conversion method and apparatus - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a motion compensation frame number conversion device with high image quality and low cost. When a prediction error component of a representative vector is equal to or larger than a threshold value, a plurality of search modes are set according to the direction and magnitude of a motion vector that has been detected. In the search area set in the search mode, a motion vector is detected by re-searching, and in the singular vector correction and smoothing, signal processing specified in a plurality of types of search modes is performed according to the direction and magnitude of the motion vector detected. , The magnitude of the motion vector and the reliability of the motion compensation process,
The signal of the interpolation frame is adaptively selected according to the distance from the interpolation frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for converting the number of frames of an image signal, and more particularly, to realize a frame number conversion by generating an interpolated frame signal by motion compensation signal processing with high image quality and low cost. The present invention relates to a method and an apparatus for converting the number of motion-compensated frames of an image signal, which are suitable for performing the method.

[0002]

2. Description of the Related Art In recent years, with the development of multimedia,
Information home appliances, television receivers, and the like need to flexibly cope with various input sources having different image formats, such as various types of television signals and personal computer image signals.

In the display system, flat displays such as plasma displays, liquid crystal displays, and micromirror devices are expected to be widely used in addition to conventional CRTs.

Therefore, in order to display various input sources on these display systems, a function of converting the number of frames for converting an input image signal into a signal sequence having a frame frequency of the display system is required.

Heretofore, regarding the conversion of the number of frames of an image signal, a device using motion compensation processing has been put to practical use in the broadcasting field. However, the apparatus is large in scale and extremely expensive, and cannot be used for an information home appliance terminal, a television receiver, or the like.

On the other hand, signal processing such as frame repetition is known as a method for realizing the function of converting the number of frames at low cost. However, motion judder disturbance that impairs the smoothness of motion in moving images occurs, and image quality is reduced. Greatly deteriorates.

In the frame number conversion by the motion compensation processing, a motion vector of an image is detected, and the position of the image of the previous and next frames is moved by the motion vector to generate a signal of an interpolation frame. Regarding this, for example, JP-A-7-170496, JP-A-7-336
Many devices have been devised, such as Japanese Patent Publication No. 650. However, the former has a problem in the detection accuracy of the motion vector, and the latter has a problem in the complexity of the signal processing. There is a problem in realizing high image quality and low cost applicable to information home appliances and television receivers.

[0008]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object of the present invention is to provide a frame number conversion method and apparatus for motion compensation processing with high image quality and low cost.

[0009]

In the frame number conversion by the motion compensation processing, when a false detection occurs in a motion vector,
An isolated point degradation occurs in which a part of the image is replaced with an inappropriate image. Therefore, in order to realize frame number conversion by motion compensation processing with high image quality and low cost, it is necessary to detect a motion vector with high accuracy and with a small amount of calculation. Therefore, the present invention employs the following technical means of motion vector detection.

The detection of a motion vector is performed by a block search and a mini-block search for gradually reducing the block size. Then, in the block search, a representative vector is set using the detected motion vector around the block of interest as a reference vector, and a motion vector with a low correlation is corrected to a motion vector with a high correlation. Further, in the mini-block search in which the block size is reduced by subdividing the blocks in the horizontal and vertical directions, a mini-block division search in which the motion vector with the smallest prediction error is assigned to the target mini-block, Is smoothed.

Further, in the representative vector setting, a plurality of types of search modes are set according to the direction and the magnitude of the motion vector that has been detected, and when the prediction error component of the representative vector is equal to or larger than a threshold, a tree search defined by the magnitude is performed. In the block matching search, a search area set in the search mode is searched again to detect a motion vector. In the singular vector correction and smoothing, a plurality of types of search modes are set according to the direction and magnitude of a motion vector that has been detected, and signal processing defined in the above search mode is performed. This technical means makes it possible to detect a motion vector with high accuracy and with a small amount of calculation.

Next, in generating an interpolated frame signal for frame number conversion, a first signal sequence generated by motion compensation processing on a signal of a current frame of an image signal sequence and a signal of a previous frame are generated. A second signal sequence generated by the motion compensation process, a third signal sequence generated by the linear interpolation process of the signals of the current frame and the previous frame, and a fourth signal sequence generated by the signal of the current frame. An interpolated frame signal is generated by adaptively selecting and using the size of the motion vector, the reliability of the motion compensation processing, and the distance from the interpolated frame. With this technical means, it is possible to effectively suppress the occurrence of an isolated point deterioration or the like peculiar to the motion correction processing, and to achieve high image quality.

Further, in a scene change area of an image, the operation of the motion compensation processing (detection of a motion vector and generation of an interpolation frame signal by the motion compensation processing) is stopped, and an enormous amount of calculation required for detecting the motion vector in this area is performed. Avoid the occurrence of volume.

By the technical means described above, a high-quality, low-cost motion compensation frame number conversion method and apparatus can be achieved.

[0015]

(Embodiment 1) A first embodiment of the present invention will be described with reference to FIGS. In this embodiment, the display system is assumed to be in the form of sequential scanning at a frame frequency of 60 Hz.

FIG. 1 is a block diagram of an apparatus for converting the number of motion-compensated frames of an image signal according to a first embodiment of the present invention.
IP conversion unit 1, frame delay unit 2, motion vector detection unit 3, MC vector generation unit 4, MC signal generation units 5, 6, linear interpolation unit 7, reliability verification unit 8, setting unit 9, selection unit 10,
It comprises a time series converter 11 and a controller 12.

The input image signal S 1 (luminance signal component and color difference signal component) is input to the IP conversion unit 1. When the input image signal S1 is a signal of a general image in the form of interlaced scanning, for example, the signal processing according to the related art such as motion adaptive scanning conversion, vertical time interpolation filter, and motion compensation scanning conversion is performed to perform interlaced scanning to sequential scanning. To perform double-speed scan conversion to generate a sequentially scanned image signal sequence S2 (a luminance signal component and a color difference signal component). Also, if the input image signal S1 is 2−
In the case of a telecine image that has been subjected to a three-pull-down process, the signal is converted into a signal for sequentially scanning a film image of 24 frames per second by a scanning line rearrangement process. In the case where the input image signal S1 is a signal in the form of progressive scanning, the signal is output as it is as a signal S2 without performing double-speed scan conversion processing.

The progressive scanning image signal S2 (hereinafter abbreviated as the current frame signal) generates an image signal S3 (hereinafter abbreviated as the previous frame signal) delayed by one frame period of the progressive scanning in the frame delay unit 2.

The motion vector detecting section 3 detects a motion vector MV in one frame period by using, for example, a luminance signal component of the current frame signal S2 and the previous frame signal S3.

The MC vector generation unit 4 uses the motion vector MV to calculate a motion compensation vector Vct, which moves the current frame signal to an interpolated frame for frame number conversion.
A motion compensation vector Vpr for moving the position of the previous frame signal onto an interpolation frame for frame number conversion is generated.

The MC signal generator 5 performs a motion compensation process for moving the position of the current frame signal using the motion compensation vector Vct, and generates a first signal sequence SMc (a luminance signal component and a color difference signal component). Further, the MC signal generation unit 6
A motion compensation process for moving the position of the previous frame signal is performed using the motion compensation vector Vpr, and the second signal sequence SMp
(A luminance signal component and a color difference signal component).

On the other hand, the linear interpolation section 7 performs a linear interpolation process between the current frame signal and the previous frame signal to generate a third signal sequence ST.

The reliability verifying unit 8 verifies the reliability of the motion compensation process using the difference signal component between the first and second signal sequences SMc and SMp generated by the motion compensation process and the motion vector MV. The signal ER is output.

The setting unit 9 determines which of the first, second, and third signal sequences and the fourth signal sequence of the current frame signal is to be selected based on the reliability signal ER and the motion vector MV. A control signal SLT to be specified is generated.

The selecting section 10 selects a signal specified by the control signal SLT and generates an interpolated frame signal S4 (a luminance signal component and a color difference signal component) required for the frame number conversion process.

The time series converter 11 performs time axis compression processing and time series rearrangement processing of the current frame signal and the interpolated frame signal S4, and sequentially scans the image signal S5 ( Luminance signal component and color difference signal component)
Is output.

Although not explicitly shown in the drawings, the control unit 12 controls the signal processing operation in each unit and generates signals necessary for the operation.

In the following, the configuration and operation of each unit in the apparatus according to the present embodiment will be further described. First, the motion vector detection unit will be described with reference to FIGS.

FIG. 2 shows an example of the configuration of the motion vector detecting section.
Representative vector setting unit 13, singular vector correction unit 14, memory unit 15, mini-block division search unit 16, smoothing unit 17
And the motion vector MV in one frame period is detected from, for example, a luminance signal component of the current frame signal S2 and the previous frame signal S3.

FIG. 3 shows a signal processing flowchart for detecting a motion vector in the motion vector detecting section. The signal processing of the first, second, and third steps is performed by the representative vector setting unit 13 in FIG.

First, a difference signal component between the signals S2 and S3 is calculated in units of a block having a block size of, for example, horizontal 8 pixels × vertical 8 pixels. Are determined to be moving image blocks.

The stationary block is a representative vector BT.
Set to zero. On the other hand, for a moving image block, a motion vector that has been detected at the periphery of the block of interest is set as a reference vector RV, and the one with the smallest prediction error ME {RV} is set as a representative vector BT. Here, the prediction error ME {RV} is
It is calculated by the following equation (1).

[0033]

ME {RV} = {| S2 (x, y) -S3 (x + RVx, y + RVy) | (1) where S2 (x, y) is a pixel (x, y) in the block
Of the current frame signal, S3 (x + RVx, Y + RV
y) represents the pixel (x, y) as the x component RV of the reference vector RV.
The value of the previous frame signal of the point moved to the position of (x + RVx, y + RVy) in the x and y components RVy, || is its absolute value, and Σ is the sum of the pixels in the block.

FIG. 4 shows an example of a reference vector used for setting the representative vector. The motion vectors that have been detected in the eight blocks around the target block are used as reference vectors. For example, the motion vectors detected in the current frame are set as reference vectors RV1 to RV4 in the blocks in the shaded area, and the reference vectors RV5 to RV8 in the other blocks are those detected in the previous frame.

Next, the prediction error ME ｛B of the representative vector
When T｝ is equal to or larger than the set value, the motion vector is re-searched according to the search mode determined by the direction and the size of the motion vector, and the representative vector is set.

FIG. 5 shows an example of this search mode setting.
In the figure, the average value a of the horizontal component of the detected motion vector is shown.
The following three search modes are set for ve | Vx | and the average value ave | Vy | in the vertical direction.

V pan mode: ave | Vy |> 8 ave | Vx |
(Equivalent to almost vertical movement) ・ Normal mode: ave | Vy | ≦ 8 ・ ave | Vx
| And ave | Vx | ≦ 8 · ave | Vy | H pan mode: ave | Vx |> 8 · ave | Vy |
(Equivalent to the movement in the horizontal direction) Note that the above ave | Vx | and ave | Vy | can be calculated for each block included in the entire screen or for each block included in the N-divided screen. .

FIG. 6 shows an example of the re-search processing. FIG. 7A shows the prediction error ME {BT} of the representative vector and the form of the re-search processing. When the prediction error is less than the set value TH, it is determined that the accuracy of the representative vector is high, and the re-search processing is not performed. If the prediction error is equal to or more than TH and less than 2TH, it is determined that the accuracy is slightly low, and a re-search process by a tree search as shown in FIG. That is, from the representative vector BT as a starting point, a region indicated by a solid line in the V pan mode, an alternate long and short dash line in the normal mode, and a region indicated by a dotted line in the H pan mode are searched again by block matching processing,
Among them, the one with the smallest prediction error is used as the representative vector.

On the other hand, when the prediction error is 2TH or more, it is determined that the accuracy is low, and the re-search processing by the block matching search as shown in FIG. That is, in the V pan mode, the vertical region, in the normal mode, the square region, H
In the pan mode, re-search is performed by block matching processing in which a horizontally long area is set as a search area. At this time, the search area is first searched for in the area 1 and then in the area 2,..., And finally in the area sequentially enlarged with the area N. When a motion vector with a prediction error smaller than the set value TH is obtained, Is set as the representative vector.
Then, the re-search processing in the subsequent area is stopped.

Returning to FIG. 2, the singular vector correction unit 14
The signal processing shown in the fourth step of FIG. 3 is performed, and the correction vector BT1 is output. This operation is schematically shown in FIG.

FIG. 7A shows an example of a block used for detecting a singular vector. The motion vectors BTu, BTd, and BT of four blocks (upper, lower, left, and right) adjacent to the target block are shown.
1, BTr is used.

FIG. 4B shows an example of the detection of a singular vector. In V pan mode, motion vector B of the block of interest
The difference value between T and the motion vector of the vertically adjacent block is not less than the set value TH1, the difference value between the motion vector of the upper, lower, left and right blocks is 4TH1 or more in the normal mode, and the difference value between the motion vector of the left and right blocks in the H pan mode. Those having a difference value of TH1 or more are detected as singular vectors.

FIG. 9C shows an example of the singular vector correction. The block having the singular vector is calculated from the motion vector of the top and bottom blocks in the V pan mode, the top and bottom left and right blocks in the normal mode, and the left and right blocks in the H pan mode with the least prediction error. Correction processing for a motion vector is performed.

Returning to FIG. 2, the memory unit 15 stores the modified vector BT1 and outputs a reference vector RV necessary for setting a representative vector.

Further, the mini-block division search section 16 applies a mini-block (for example, a size of two horizontal pixels × two vertical pixels) obtained by subdividing the block in the horizontal and vertical directions.
The signal processing of the fifth step in FIG. 3 is performed to generate a motion vector MV1 for each mini-block. This operation is schematically shown in FIG.
Shown in

For each mini-block in the block of interest, the prediction error ME ｛B due to the motion vector BT of the block of interest
Calculate T｝. Then, for a mini-block whose prediction error is smaller than the set value TH, a motion vector MV1 of the mini-block is generated by the motion vector BT. On the other hand, the mini-blocks with TH or more are the motion vectors with the smallest prediction error among the motion vectors BT1, BT2,.
1 is generated. In this signal processing, even when different motions are mixed in a block, each mini-block can generate a motion vector corresponding to each motion.

Finally, the smoothing unit 17 performs the signal processing of the sixth step in FIG. 3 and outputs a motion vector MV. This operation is schematically shown in FIG.

FIG. 7A shows a form of a mini-block for performing a smoothing process. In the V pan mode, upper and lower mini blocks a and b adjacent to the target mini block are used. In the normal mode, peripheral mini blocks a, b,..., G, h are used. In the H pan mode, left and right mini blocks c and d are used. .

FIG. 9B shows an example of the smoothing process. V
In the pan mode and the H-pan mode, a motion vector of the mini-block having a small error with respect to the motion vector of the vertical block or the horizontal mini-block is set as the motion vector of the target mini-block. On the other hand, in the normal mode, the weighted average of the motion vectors of the target mini-block and the peripheral mini-block (ΣWi · MVi weight coefficient Wi
Is, for example, Wi = 1/10 for a peripheral mini-block and Wi = 2/10 for a mini-block of interest) as the motion vector of the mini-block of interest.

The configuration of the motion vector detecting section described above,
By the operation, the amount of calculation is small and the motion vector can be detected with high accuracy.

Next, in order to facilitate understanding of the subsequent operation, an outline of the conversion of the number of motion compensation frames in this embodiment will be described with reference to FIG. The figure (a) is 50Hz-60Hz
Conversion of the number of frames, that is, when the frame frequency is 50H
A case where the input signal sequence of the progressive scanning of z is converted into the output signal sequence of the progressive scanning at a frame frequency of 60 Hz is shown. Signals in the frame order 1, 2, 3, 4, 5, 6 of the output signal sequence are generated from the signals in the frame order 1, 2, 3, 4, 5 of the input signal sequence. Among them, the signal of the frame order 1 uses the signal of the frame order 1 of the input signal sequence, and the remaining motion-compensated interpolation frames of the frame orders 2 to 6 are generated by the motion compensation processing.

At this time, the signal of the frame sequence of the input signal sequence close to the position of the interpolation frame is subjected to motion compensation processing to generate the signal of the interpolation frame. For example, the signals of the interpolation frames 2 and 3 are generated by moving the signals of the input signal sequence in the frame order 2 and 3 by the motion compensation vector Vct.
In addition, the signal of the interpolation frame 4 is generated by a signal obtained by moving the position of the signal of the frame order 3 of the input signal sequence by the motion compensation vector Vct or the signal of the frame order 4 by the motion compensation vector Vct. On the other hand, the signals of the interpolated frames 5 and 6 are generated by moving the signals of the input signal sequence in the frame order 4 and 5 by the motion compensation vector Vpr. The motion compensation vectors Vct and Vpr are generated by the motion vector MV in one frame period of the input signal sequence of the progressive scanning, as described later.

FIG. 9B shows a case where the number of frames is converted from 24 Hz to 60 Hz, that is, the input signal sequence of a film image having a frame frequency of 24 Hz is converted into an output signal sequence of progressive scanning at a frame frequency of 60 Hz. The signals in the frame order 1, 2, 3, 4, 5 of the output signal sequence are generated from the signals in the frame order 1 and 2 of the input signal sequence. Among them, the signal of frame order 1 is the frame order 1 of the input signal sequence.
, And the remaining motion-compensated interpolation frames in order of 2 to 5 are generated by the motion compensation processing.

At this time, the signal of the frame sequence of the input signal sequence close to the position of the interpolation frame is subjected to motion compensation processing to generate the signal of the interpolation frame. For example, the signal of the interpolation frame 2 is generated by a signal obtained by shifting the position of the signal of the frame order 1 of the input signal sequence by the motion compensation vector Vpr. The signals of interpolation frames 3 and 4 are obtained by converting the signal of frame order 2 of the input signal sequence into motion compensation vectors Vct and Vpr.
Generated by the signal whose position has been moved by. On the other hand, the signal of the interpolation frame 5 is generated by a signal in which the position of the signal of the frame order 1 of the input signal sequence is shifted by the motion compensation vector Vct.

FIG. 9C is a schematic diagram of the generation of an interpolated frame signal by the motion compensation processing. The signal S (x, y) at the position of the point (x, y) of the interpolation frame is obtained by converting the signal S2 of the n frame corresponding to the current frame signal by the motion compensation vector Vct (x component Vctx, y component Vcty). The signal S2 of the moved point (x-Vctx, y-Vcty)
(Xct, yct) or the signal S3 of the (n-1) frame corresponding to the previous frame signal is
A signal S3 of a point (x + Vprx, y + Vpry) whose position has been shifted by pr (x component Vprx, y component Vpry)
(Xpr, ypr).

Hereinafter, description will be returned to each part. FIG.
It is an operation | movement schematic diagram of a vector generation part. The motion compensation vectors Vct and Vpr are generated by the conversion process of the motion vector MV for one frame period so that the motion compensation interpolation frames are arranged at equal intervals in the time direction.

FIG. 9A shows the case of 50 Hz-60 Hz conversion. In frame order 2 of the motion compensation interpolation frame, V
ct is 1/6 times MV, Vpr is 5/6 times, in frame order 3, Vct is 2/6 times MV, Vpr is 4/6 times, and in frame order 4, Vct is 3/6 times MV, Vpr. Is 3 /
6 times, in frame order 5, Vct is 4/6 times MV, Vp
r is 2/6 times, and in frame order 6, Vct is 5/6 of MV.
Double and Vpr are generated with a motion vector multiplied by 1/6.

FIG. 7B shows a case of 24 Hz-60 Hz conversion. In frame order 2 of the motion compensation interpolation frame, V
ct is 3/5 times MV, Vpr is 2/5 times, in frame order 3, Vct is 1/5 times MV, Vpr is 4/5 times, and in frame order 4, Vct is 4/5 times MV, Vpr. Is 1 /
5 times, in frame order 5, Vct is 2/5 times MV, Vp
r is generated by a motion vector multiplied by 3/5.

Next, an example of the configuration of the reliability verification unit 8 is shown in FIG.
Shown in The subtraction unit 18 includes a first signal sequence SMc generated by performing motion compensation processing on the current frame signal using the motion compensation vector Vct, and a second signal sequence generated by performing motion compensation processing on the previous frame signal using the motion compensation vector Vpr. A subtraction operation with SMp is performed, and the result is output as signal S10. When the accuracy of the motion vector is high, the first signal sequence SMc
And the signal values of the second signal series SMp substantially match. Therefore, the signal S10 has a value close to zero. On the other hand, when the accuracy of detecting the motion vector is low, the two have different signal values, and thus a significant difference occurs in the signal S10.

The quantization section 19 performs a binary quantization process on the signal S10. At this time, the threshold is adaptively changed according to the magnitude of the motion vector MV. If the motion vector MV is small, the detection accuracy tends to be high, and the detection accuracy tends to decrease as the motion vector MV increases. Therefore, the threshold value is set to be high when the magnitude of the motion vector MV is small, and to be low when the magnitude is large. Then, a binary signal S11 of 0 is obtained when the motion vector is accurate and the reliability of the motion compensation process is high, and 1 when the motion vector is incorrect and the reliability of the motion compensation process is low. The spatial smoothing unit 20 performs the area connection processing of the signal S11 in the horizontal and vertical spatial areas, and generates a reliability signal ER in which the missing part is supplemented.

Next, an example of the characteristics of the setting section 9 will be described with reference to FIG. FIG. 3A shows an example of 50 Hz-60 Hz conversion. When the motion vector MV is equal to the first set value V1 (for example, 1
動 き 2 pixels / frame), the control signal SL is selected so as to select the third signal sequence ST generated by the linear interpolation between the current frame signal and the previous frame signal.
Set T.

When the motion vector MV is equal to or larger than the first set value V1 and smaller than the second set value V2 (for example, V2 is a movement that allows the line of sight to follow), when the reliability signal ER is 0, the graph shown in FIG. 10, the first signal sequence SMc in the interpolation frames 2 and 3, the first signal sequence SMc or the second signal sequence SMp in the interpolation frame 4, and the second signal sequence SMp in the interpolation frames 5 and 6. Control signal SLT is set to select signal sequence SMp. On the other hand, when the reliability signal ER is 1, the control signal SLT is set so as to select the third signal sequence in any interpolation frame.

On the other hand, when the motion vector MV is the second set value V
In the case of two or more (movements in which the line of sight cannot follow), the control signal SLT is set so that the fourth signal sequence S2 is selected in any of the interpolation frames.

FIG. 9B shows an example of 24 Hz-60 Hz conversion. If the motion vector MV is smaller than the first set value V11 (for example, a motion of about 1 to 2 pixels / frame), a third signal sequence generated by linear interpolation between the current frame signal and the previous frame signal The control signal SLT is set so as to select ST.

When the motion vector MV is equal to or larger than the first set value V11 and smaller than the second set value V21 (for example, V21 is a movement that allows the line of sight to follow), the reliability signal ER is set to 0.
In the case of, as shown in FIG.
Then, the control signal SLT is set so as to select the second signal sequence SMp and the first signal sequence SMc in the interpolation frames 3 and 5. On the other hand, when the reliability signal ER is 1, the control signal SLT is set so as to select the third signal sequence in any interpolation frame.

On the other hand, when the motion vector MV is the second set value V
In the case of 21 or more (movement in which the line of sight cannot follow), the control signal SLT is set so as to select the fourth signal sequence S2 in any of the interpolation frames.

Finally, regarding the time series conversion unit 11, FIG.
This will be described in Section 4. FIG. 1A shows an example of the configuration, which is composed of a buffer memory unit 21 and an RW control unit 22. The outline of this operation is as follows.
The case of z will be described as an example with reference to FIG. The image signal S2 of the progressive scanning has the frame order shown in FIG.
The signals of frame order 1 out of 5 are recorded in the buffer memory unit 21 by the WT operation. On the other hand, the frame order of the interpolated frame signal S4 generated by the motion compensation processing is 2, 3, 4, 5, 6
(Signals indicated by dots) are sequentially recorded in the buffer memory unit 21 by the WT operation.

On the other hand, the RD operation from the buffer memory unit 21 is performed every frame cycle of the sequential scanning at a frame frequency of 60 Hz, and the image signal S2
After reading out the signal of the frame order 1 of the above, the operation of reading out the frame orders 2, 3, 4, 5, and 6 of the interpolated frame signal S4 is performed. Then, an image signal S5 of the progressive scanning obtained by converting the frame frequency to 60 Hz is obtained as an output.

The RW control unit 22 generates a control signal WCT necessary for the WT operation and a control signal RCT required for the RD operation, and supplies them to the buffer memory unit 21.

As described above, according to the first embodiment of the present invention, a motion vector required for a motion compensation process can be detected with high accuracy and a very small amount of calculation. Further, the image quality deterioration peculiar to the motion compensation processing can be largely suppressed by the function of simple signal processing reliability verification. For this reason, a high-quality, low-cost motion-compensation frame number conversion device can be easily realized, and a remarkable effect can be obtained in improving the image quality of a multimedia-compatible television receiver or various information home appliances.

(Embodiment 2) Next, a second embodiment of the present invention will be described with reference to FIGS. In this embodiment, the display system is assumed to be in the form of sequential scanning at a frame frequency of 60 Hz.

FIG. 15 is a block diagram of the apparatus for converting the number of motion-compensated frames of an image signal according to the present embodiment, which is constructed by adding a scene change detecting section 23 to the first embodiment. Then, the IP conversion unit 1, the frame delay unit 2, the motion vector detection unit 3, the MC vector generation unit 4, the MC signal generation units 5 and 6, the linear interpolation unit 7, the reliability verification unit 8, the setting unit 9, and the selection unit 10. The configuration and operation of the time series converter 11 are the same as those of the first embodiment.

The scene change detecting section 23 outputs the sequentially scanned image signal S2 and the signal S3 delayed by one frame period.
Is used to detect an area where a scene change has occurred in the image, and is 1 in a scene change area and 0 in other areas.
Is output.

The control section 12 performs the same operation as in the first embodiment in the region where the signal SCF is 0. On the other hand, the signal SCF
In the region where is 1, the operation of stopping the motion compensation processing is performed for each unit. By this operation, the motion vector detection unit 3
Stops the motion vector MV detection process. The setting unit 9 sets the control signal SLT so as to select the fourth signal sequence S2.

FIG. 16 shows an example of the configuration of the scene change detecting section. The subtraction unit 24 performs a subtraction process on the signals S2 and S3 to extract a frame difference signal component S20.
The quantization unit 25 performs binary quantization with a predetermined threshold, and outputs a binary signal S21 (0 is smaller than the threshold, 1 is equal to or larger than the threshold). In the region where the scene change has occurred, the signal level of the frame difference signal component is considerably large. It is desirable that the threshold value be set higher than in normal motion detection.

The integrating section 26 integrates the number of pixels which become 1 in the signal S21 over one frame period, and outputs the accumulated value to the signal S22. Then, when the cumulative value is, for example, the number of pixels of half or more of one screen, the determining unit 27 determines that the area is the scene change area and outputs 1 to the signal SCF. On the other hand, if the accumulated value is less than half of one screen, the signal SC
0 is output to F.

As described above, according to the second embodiment of the present invention, a motion vector required for a motion compensation process can be detected with high accuracy and a very small amount of calculation. Further, it is possible to suppress the generation of an enormous amount of calculation required for detecting a motion vector in a region where a scene change has occurred. In addition, with the function of simply verifying the reliability of the signal processing, it is possible to significantly suppress the deterioration of the image quality peculiar to the motion compensation processing. For this reason, a high-quality, low-cost motion-compensation frame number conversion device can be easily realized, and a remarkable effect can be obtained in improving the image quality of a multimedia-compatible television receiver or various information home appliances.

(Embodiment 3) Next, a third embodiment of the present invention will be described with reference to FIGS. In this embodiment, the display system is assumed to be in the form of sequential scanning at a frame frequency of 60 Hz.

FIG. 17 is a block diagram of the apparatus for converting the number of motion-compensated frames of an image signal according to the present embodiment. this is,
The time series conversion unit of the first embodiment is omitted, and a frame number up unit 28 is added at the subsequent stage of the IP conversion unit 1.

The signal sequence S2 of the progressive scanning obtained from the IP conversion unit 1 is input to the frame number increasing unit 28, and is converted into a signal sequence S31 of the progressive scanning of 60 Hz by performing frame processing signal processing. FIG. 18 shows a configuration example and an operation outline of the frame number increasing unit 28.

FIG. 9A shows an example of the configuration, which is composed of a buffer memory unit 30 and a memory control unit 31. FIG.
Is a signal sequence S of progressive scanning with a frame frequency of 50 Hz.
2 is converted to a signal sequence having a frame frequency of 60 Hz. The signals in the frame order 1, 2, 3, 4, 5 of the signal sequence S2 of the progressive scanning with the frame frequency of 50 Hz are W
The data is stored in the buffer memory unit 30 by the writing process by the T operation. On the other hand, in the read processing by the RD operation, for example, signal processing of frame repetition for reading a signal in frame order 1 twice consecutively is performed, and the frame order is 1, 1, 2, 3,
A signal sequence S31 of progressive scanning with a frame frequency of a sequence of 4 and 5 of 60 Hz is generated.

FIG. 11C shows that the frame frequency is 2
The signal sequence S2 of the progressive scanning of 4 Hz has a frame frequency of 6
The case of converting to a 0 Hz signal sequence is shown. The signals in the frame order 1 and 2 of the signal sequence S2 of the progressive scanning with the frame frequency of 24 Hz are stored in the buffer memory unit 30 by the writing process by the WT operation. On the other hand, in the read processing by the RD operation, for example, signal processing of frame repetition for successively reading out a signal in frame order 1 twice and a signal in frame order 2 three times is performed.
A signal sequence S31 of progressive scanning having a frame frequency of 60 Hz in the sequence of 2, 2 is generated.

Note that the memory control unit 31
Control signals WT and RD necessary for the operation and the RD operation are generated.

Referring back to FIG. 17, the frame frequency 60H
Based on the signal sequence S31 converted into z and the signal sequence S32 delayed by one frame period by the frame delay unit 2, the subsequent blocks generate an interpolated frame signal by motion compensation processing, and A signal sequence S5 of the progressive scanning obtained by converting the number of frames is obtained.

FIG. 19 shows an outline of these operations. FIG. 3A shows that the signal sequence S31 is in frame order 1, 1, 2, 2, 3, 4, 5, 6, 7, 8, and 9.
3, 4, and 5. For this signal sequence, the motion vector MV in one frame period using the luminance signal components of frame order 1, 2, 2, 3, 3, 4, 4, 5, 5, 1
Is detected. When the frame order is 1, 1, the motion vector detecting operation is stopped. The motion-compensated interpolation frames 2 and 3 are subjected to motion compensation processing for moving the position of the current frame signal by the motion compensation vector Vct generated from the motion vector MV, and the motion-compensated interpolation frame 4 is generated from the motion vector MV. Motion compensation vector Vct (V
pr), the motion compensation processing for moving the position of the current (previous) frame signal, and the motion compensation interpolation frames 5 and 6 perform the motion compensation for moving the position of the previous frame signal with the motion compensation vector Vpr generated from the motion vector MV. Generated by processing.

FIG. 13B shows a case where the signal sequence S31 is in the frame order of 1, 1, 2, 2, and 2. With respect to this signal sequence, a motion vector MV in one frame period is detected using the luminance signal components in the frame order 1, 2, 2, and 1. When the frame order is 1, 1, 2, or 2, the motion vector detection operation is stopped. And motion compensation interpolation frame 2
And 4 are motion compensation processing for moving the position of the previous frame signal with the motion compensation vector Vpr generated from the motion vector MV. Motion compensation interpolation frames 3 and 5 are motion compensation vectors Vct generated from the motion vector MV. It is generated by motion compensation processing for moving the position of the current frame signal.

Returning to FIG. 17, the motion vector detecting section 3 is configured in the same manner as in the first embodiment, and stops the detecting operation in the frame repetition area to detect the motion vector MV as described above. Perform the operation. The configuration and operation of the MC vector generation unit 4, the MC signal generation units 5 and 6, the linear interpolation unit 7, the reliability verification unit 8, the setting unit 9, and the selection unit 10 are the first.
This is the same as the embodiment of the present invention.

Although not explicitly shown in the drawing, the control unit 29 controls the signal processing operation in each block and generates signals necessary for the operation.

As described above, according to the third embodiment of the present invention, as in the first embodiment, it is possible to easily realize a high-quality, low-cost motion-compensation frame number conversion apparatus. In addition, a remarkable effect can be obtained for improving the image quality of multimedia-compatible television receivers and various types of information home appliances.

(Embodiment 4) Next, FIG. 20 shows a block configuration example of a fourth embodiment of the present invention. In the present embodiment, a scene change detection unit 23 is newly added to the third embodiment, and a signal SCF of 1 is generated in a region where a scene change has occurred, and 0 is generated in other regions. Then, the control unit 29
In the region where the signal SCF is 0, as in the third embodiment, the SCF
In the region where is 1, the control of each block is performed so that the operation of detecting the motion vector and generating the interpolation frame signal by the motion compensation processing is stopped. The configuration and operation of each unit can be easily understood in the embodiments described above, and thus the description is omitted.

As described above, according to the fourth embodiment of the present invention, the generation of an enormous amount of calculation required for detecting a motion vector in an area where a scene change has occurred can be suppressed, and high image quality and low cost can be achieved. A simple apparatus for converting the number of motion compensation frames can be easily realized. In addition, a remarkable effect can be obtained in improving the image quality of a multimedia-compatible television receiver and various types of information home appliances.

(Embodiment 5) Finally, an embodiment in which the present invention is applied to a television receiver will be described with reference to the drawing of FIG. The high-efficiency coded signal BCD transmitted by digital broadcasting or the like is input to the digital demodulation unit 32, performs a predetermined decoding process, and decodes the image signal S40 (luminance signal and color difference signal). A signal BCA transmitted by a conventional analog broadcast or the like is input to an analog demodulation unit 33, where a predetermined demodulation process is performed, and an image signal S41 (luminance signal and color difference signal) is obtained.
Is demodulated.

The selector 34 outputs one of the signals to the image signal S1 (luminance signal and color difference signal) under the control of the control unit 39 (not explicitly shown in the drawing).

The MCFRC unit 35 performs signal processing for frame number conversion for motion compensation according to the present invention, and outputs a progressively scanned image signal S5 that matches the frame frequency of the display system.

The scaling processing unit 36 converts the number of pixels in the horizontal and vertical directions to match the number of pixels in the display system, and generates an image signal S42.

The image quality adjustment unit 37 performs signal processing such as image quality adjustment and color space conversion, and outputs a sequential scanning signal S43 of three primary colors. Then, the progressive scan display section 38 displays an image in a progressive scan mode.

The control section 39 generates various operation control signals necessary for displaying an image according to a channel or a display form selected by a viewer using a remote control terminal or the like, and supplies the generated signals to the respective sections.

As described above, according to the present embodiment, a multimedia-compatible television receiver that can flexibly handle various input sources can be realized with high image quality and low cost.

It is clear that high-quality images can be realized for information home appliances such as PCs and DVDs by incorporating the function of converting the number of frames in the motion compensation processing according to the present invention.

[0100]

According to the present invention, it is possible to easily realize a high-quality, low-cost motion-compensation frame number conversion apparatus, which is remarkable for improving the image quality of multimedia-capable television receivers and various information home appliances. The effect is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram of a motion compensation frame number conversion device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a motion vector detection unit according to the first embodiment.

FIG. 3 is a signal processing flowchart of a motion vector detection unit.

FIG. 4 is an explanatory diagram of an example of a reference vector used for setting a representative vector.

FIG. 5 is an explanatory diagram showing an example of a search mode setting of a motion vector detection unit.

FIG. 6 is an explanatory diagram of an example of a re-search process of a motion vector detection unit and a search mode.

FIG. 7 is an explanatory diagram of an operation outline of a singular vector correction unit of the motion vector detection unit.

FIG. 8 is an explanatory diagram of an operation outline of a mini-block division search unit of the motion vector detection unit.

FIG. 9 is an explanatory diagram of an outline of operation of a smoothing unit of a motion vector detection unit.

FIG. 10 is a diagram illustrating an outline of an operation of converting the number of motion compensation frames in the first embodiment.

FIG. 11 is an explanatory diagram illustrating an outline of the operation of an MC vector generation unit according to the first embodiment.

FIG. 12 is a block diagram illustrating a configuration example of a reliability verification unit according to the first embodiment;

FIG. 13 is an explanatory diagram illustrating a characteristic example of a setting unit according to the first embodiment.

FIG. 14 is a block diagram and an operation explanatory diagram illustrating a configuration example of a time-series conversion unit according to the first embodiment.

FIG. 15 is a block diagram of a motion compensation frame number conversion device according to a second embodiment of the present invention.

FIG. 16 is a block diagram illustrating a configuration example of a scene change detection unit according to the second embodiment.

FIG. 17 is a block diagram of a motion compensation frame number conversion device according to a third embodiment of the present invention;

FIG. 18 is a block diagram and an operation explanatory diagram of a configuration example of a frame number increasing unit according to the third embodiment;

FIG. 19 is an explanatory diagram of an outline of the operation of converting the number of motion compensation frames in the third embodiment.

FIG. 20 is a block diagram of a motion compensation frame number conversion device according to a fourth embodiment of the present invention.

FIG. 21 is a block diagram showing a configuration of a television receiver according to a fifth embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... IP conversion part, 2 ... frame delay part, 3 ... motion vector detection part, 4 ... MC vector generation part, 5, 6 ... MC signal generation part, 7 ... linear interpolation part, 8 ... reliability verification part, 9 ... setting part, 10 ... selection part, 11 ... time series conversion part, 12,29,
39 control unit, 13 representative vector setting unit, 14 singular vector correction unit, 15 memory unit, 16 miniblock division search unit, 17 smoothing unit, 18, 24 subtraction unit, 1
9, 25 ... quantization unit, 20 ... spatial smoothing unit, 21, 30, ...
Buffer memory unit, 22 RW control unit, 23 scene change detection unit, 26 integration unit, 27 determination unit, 28 frame number up unit, 31 memory control unit, 32 digital demodulation unit, 33 analog demodulation Part, 34 ... selector, 3
5 ... MCFRC part, 36 ... Scaling processing part, 37 ...
Image quality adjusting unit, 38: progressive scanning display unit.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Haruki Takada 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture Inside Digital Media System Division, Hitachi, Ltd. (72) Inventor Takashi Hasegawa Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa 292 Hitachi Media, Ltd. Digital Media System Division (72) Inventor Kazuo Ishikura 5-2-1, Josuihoncho, Kodaira-shi, Tokyo In the System LSI Development Center of Hitachi, Ltd. (72) Inventor Sugiyama Masato 292, Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture Inside Digital Media Development Division, Hitachi, Ltd. (72) Inventor Mitsuo Nakajima 292, Yoshida-cho, Totsuka-ku, Yokohama City, Kanagawa Prefecture Digital Media Development Division, Hitachi, Ltd. (72 Inventor Yasutaka Tsuru Yokohama City, Kanagawa Prefecture 292 Yoshidacho Hitachi Media Digital Media Development Division F-term (reference) 5C063 BA03 BA08 BA12 BA20 CA03 CA05 CA07 CA12 CA16 CA38 CA40

Claims (16)

[Claims] In a method of converting the number of frames for motion compensation of an image signal, means for double-speed scan conversion for converting an image signal sequence of interlaced scanning into an image signal sequence of progressive scanning, and a block for the image signal sequence of progressive scanning. Means for detecting a motion vector by means of block search and mini-block search for reducing the size stepwise, and converting the number of frames by the motion compensation processing using the motion vector for the signal of the current frame of the progressively scanned image signal sequence Means for generating a first signal sequence of an interpolated frame necessary for the interpolation, and an interpolated frame required for frame number conversion by a motion compensation process using the motion vector with respect to a signal of a previous frame of the progressively scanned image signal sequence. Means for generating a second signal sequence, and a linear interpolation process of signals of a current frame and a previous frame of the progressively scanned image signal sequence. Means for generating a third signal sequence of an interpolated frame required for conversion, and generating a fourth signal sequence of an interpolated frame required for frame number conversion from a signal of a current frame of the progressively scanned image signal sequence Means for verifying the reliability of the motion compensation process based on the difference component between the first and second signal sequences of the interpolated frame and the magnitude of the motion vector. In the generation of a signal sequence of a frame, an area where the magnitude of the motion vector is smaller than the first set value is the third signal sequence, and the magnitude of the motion vector is equal to or more than the first set value and the second set value. , The region where the reliability of the motion compensation process is high is a signal sequence whose distance to the interpolation frame is short among the first and second signal sequences, the region where the reliability is low is the third signal sequence,
In a region where the magnitude of the motion vector is equal to or greater than the second set value, an interpolated frame signal is generated by the fourth signal sequence, and a sequentially-scanned image signal sequence obtained by converting the number of motion-compensated frames is output. A method for converting the number of motion-compensated frames of an image signal. And means for converting the progressively scanned image signal sequence converted by the double speed scan conversion means into an image signal sequence having a desired frame frequency by frame repetition processing. Item 1. A method of converting the number of motion-compensated frames of an image signal, wherein the interpolation frame is generated by the motion compensation process described in Item 1. 3. A system for detecting a scene change area of an image from an input image signal series, wherein in the detected scene change area, an interpolated frame signal required for frame number conversion is generated as a fourth signal series. The method according to claim 1 or 2, wherein the number of frames for motion compensation of an image signal is converted. 4. In detecting a motion vector, in a block search, means for setting a representative vector using a detected motion vector around a target block as a reference vector, and correcting a motion vector having a low correlation to a motion vector having a high correlation. In the mini-block search in which the block size is reduced by subdividing the blocks in the horizontal and vertical directions, a mini-block division search for allocating a motion vector with the smallest prediction error to the target mini-block is provided. 4. A method for converting the number of motion-compensated frames of an image signal according to claim 1, further comprising means for smoothing a motion vector of a peripheral edge of the target mini-block. 5. The means for setting a representative vector includes:
A plurality of types of search modes are set according to the direction and magnitude of the motion vector that has been detected. When the prediction error component of the representative vector is equal to or larger than a threshold, the tree search and block matching search defined by the magnitude are used in the above search mode. 5. The method according to claim 4, wherein the motion vector is set in the re-search processing of the search area to be set. 6. A singular vector correcting means for setting a plurality of types of search modes according to the direction and magnitude of a motion vector already detected, and detecting and correcting a singular vector defined by the search mode. 5. The method according to claim 4, wherein the number of frames for motion compensation of the image signal is converted. 7. The means for smoothing a motion vector includes:
5. The motion compensation frame number conversion of an image signal according to claim 4, wherein a plurality of types of search modes are set in accordance with the direction and the magnitude of the already detected motion vector, and a smoothing process defined in the search mode is performed. method. 8. An apparatus for converting the number of frames for motion compensation of an image signal, comprising: a double-speed scanning converter for converting an interlaced scanning image signal sequence into an image signal sequence of progressive scanning; A motion vector detecting unit that detects a motion vector by a block search and a mini-block search method that gradually reduces the size of a frame, and a motion compensation process using the motion vector for the current frame signal of the progressively scanned image signal sequence. An MC signal generation unit that generates a first signal sequence of an interpolation frame necessary for the number conversion,
M for generating a second signal sequence of an interpolated frame required for frame number conversion by the motion compensation processing using the motion vector with respect to the signal of the previous frame of the progressive scanning image signal sequence
A C signal generation unit; and a linear interpolation unit that generates a third signal sequence of an interpolation frame required for frame number conversion by performing a linear interpolation process on signals of the current frame and the previous frame of the image signal sequence of the progressive scanning. A generation unit that generates a fourth signal sequence of an interpolation frame necessary for frame number conversion using a signal of a current frame of the image signal sequence of the progressive scanning, and a first signal sequence between the first and second signal sequences of the interpolation frame. A reliability verification unit that verifies the reliability of the motion compensation process with the difference component and the magnitude of the motion vector, and in generating a signal sequence of an interpolation frame required for frame number conversion, the magnitude of the motion vector Is smaller than the first set value, and the region where the magnitude of the motion vector is equal to or larger than the first set value and smaller than the second set value and the reliability of the motion compensation process is high is set as Of the first and second signal sequences The distance between the interpolation frame by Chi is close signal sequence, region reliability is low the third signal sequence,
In a region where the magnitude of the motion vector is equal to or larger than the second set value, an interpolated frame signal is generated using the fourth signal sequence, and a sequentially-scanned image signal sequence converted into the number of motion-compensated frames is output. Device for converting the number of motion-compensated frames of an image signal to be converted. 9. A frame number increasing section for converting a sequentially-scanned image signal sequence converted by double-speed scanning conversion means into an image signal sequence having a desired frame frequency by frame repetition processing, An apparatus for converting the number of motion-compensated frames of an image signal, comprising generating an interpolated frame by the motion compensation processing according to claim 8. 10. A scene change detecting section for detecting a scene change area of an image from an input image signal sequence, wherein in the detected scene change area, a signal of an interpolation frame required for frame number conversion is converted into a fourth signal. The image signal motion compensation frame number conversion apparatus according to claim 8, wherein the apparatus is generated as a sequence. 11. In detecting a motion vector, in a block search, a representative vector setting unit that sets a representative vector using a detected motion vector around a target block as a reference vector, and a motion vector with a low correlation to a motion vector with a high correlation. In a miniblock search in which a singular vector correction unit that corrects a vector is provided and the block size is reduced by subdividing the blocks in the horizontal and vertical directions, a miniblock division search unit that allocates a motion vector with a minimum prediction error to the target miniblock The apparatus according to any one of claims 8 to 10, further comprising: a smoothing unit for smoothing a motion vector of a peripheral edge of the target mini-block. 12. A representative vector setting unit for setting a plurality of types of search modes according to the direction and magnitude of a motion vector that has been detected, and when the prediction error component of the representative vector is greater than or equal to a threshold, a tree defined by the magnitude thereof. 12. The apparatus according to claim 11, wherein a motion vector is set by a re-search process of the search area set in the search mode in the search and the block matching search. 13. The singular vector correction unit sets a plurality of types of search modes according to the direction and magnitude of a motion vector that has been detected, and detects and corrects a singular vector defined by the search mode. The apparatus for converting the number of motion-compensated frames of an image signal according to claim 11, wherein 14. A motion vector smoothing unit, wherein a plurality of types of search modes are set according to the direction and magnitude of a motion vector that has been detected, and a smoothing process defined by the search mode is performed. Item 12. An apparatus for converting the number of motion compensation frames of an image signal according to Item 11. 15. A motion compensation frame number conversion system according to any one of claims 1 to 7 or a motion compensation frame number conversion device according to any one of claims 8 to 14, comprising an image display section of a progressive scanning mode. A television receiver for converting the number of frames to a frame frequency of an image display unit. 16. A method of converting the number of frames to a frame frequency of an image display system, comprising the steps of: An information home appliance terminal device that performs signal processing by any of the above.