JPH05236335A - Motion vector detector - Google Patents

Abstract

(57) [Abstract] [Purpose] To reduce false detection and detection error of motion vector. [Structure] Motion vector detection systems 12a, 12b, 12
The c, 12d, and 12e detect the motion vector from the image signal from the input terminal 10 by using different detection algorithms and / or different detection characteristics. The preprocessing filter circuits 18a, 18b, 18c extract specific spatial frequency components suitable for the two-dimensional phase difference detection circuits 20a, 20b, 20c from the input image signal. The circuits 20a and 20c have the same detection algorithm but different template sizes. The circuit 20b differs from the circuit 20a in the residual calculation and evaluation method. The comparison circuit 14 compares the outputs of the respective systems 12a to 12e, selects them in a predetermined priority order, and outputs them to the output terminal 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting device for detecting a motion of a subject image from an image signal, and more specifically, a TV camera, a video camera, an electronic still camera.
The present invention relates to a motion vector detection device in an image pickup device such as a camera.

[0002]

2. Description of the Related Art As a motion vector detecting method in image processing, a cross-correlation method using a cross-correlation coefficient, or a template method using a residual in a predetermined block or template
There is a matching method (or block matching method).
The cross-correlation coefficient h (ξ, η) in the cross-correlation method is given by the following equation. That is, h (ξ, η) = ∫∫g0 (x−ξ, y−η) · g1 (x, y) dxdy where g0 and g1 are two images that are temporally adjacent to each other, x,
y is a position coordinate in the screen, and ξ and η are displacement amounts or deviations in the x and y directions. Further, the residual e (ξ, η) in the template matching method (or the block matching method) (hereinafter, referred to as residual) is given by the following equation. That is, e (ε, η) = Σb | g0 (x−ε, y−η) −g1 (x, y) | Σb indicates the summation operation in the attention area B. The square of the difference may be used instead of the absolute value of the difference.

Both methods are substantially equivalent in principle. The cross correlation method finds the deviation (ξ, η) that maximizes h (ξ, η), and the block matching method uses e.
Find the deviation (ε, η) that minimizes (ε, η). The deviation (ε, η) thus obtained is used as the motion vector.

[0004]

In the method of detecting a motion vector by image processing, the detection accuracy of the motion vector and its detectable range (hereinafter referred to as the detection range) are strong in the input image, the detection algorithm and its detection characteristic. If the detection algorithm and its detection characteristics are not appropriate for the input image, the detection accuracy may be very poor.

Further, since it is premised that the objects are not deformed or the objects do not overlap with each other, when these occur, a theoretical error (that is, erroneous detection or detection error) occurs.

An object of the present invention is to provide a motion vector detecting device that solves such a problem.

[0007]

A motion vector detecting apparatus according to the present invention includes a plurality of motion vector detecting systems for detecting a motion vector from an input image, each of which has a different detection algorithm or detection characteristic. It is characterized in that it comprises a comparison means for comparing detection values by the motion vector detection system and outputting an appropriate motion vector.

[0008]

Since each of the motion vector detection systems has a different detection algorithm or detection characteristic from each other, errors and erroneous detection that a particular detection algorithm and detection characteristic may have in principle with respect to an input image are possible. Sex can be offset or alleviated. Therefore, an appropriate motion vector detection value can be selected by the comparison calculation in the comparison means.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a schematic block diagram of an embodiment of the present invention. An image signal (NTSC
System, PAL system or SECAM system luminance signal). The image signal input to the input terminal 10 includes a plurality of motion vector detection systems 12a, 12b, 12c, 12 having different motion vector detection algorithms and / or different detection characteristics.
Input to d, 12e, .... The comparison circuit 14 includes motion vector detection systems 12a, 12b, 12c, 12d, 1
The outputs of 2e, ... Are compared and calculated, and an appropriate motion vector value is output to the output terminal 16 by a predetermined priority order, majority decision, weighted averaging, or the like.

For example, the motion vector detection systems 12a and 12
Reference numerals b and 12c are circuits for performing a two-dimensional phase difference comparison between screens of specific spatial frequency components of an image signal and detecting a motion vector by the above-mentioned residual. The size and shape of the template (or block) are respectively provided. , And the residual calculation / evaluation method and its threshold value are different. The motion vector detection system 12d detects a motion vector by the Fourier transform method, and the motion vector detection system 12e is used in US Pat. No. 3,890,462 and 1985 patent application publication No. 46.
The motion vector is detected by the spatiotemporal gradient method described in No. 878.

The motion vector detection system 12a includes an input terminal 1
A preprocessing filter circuit 18a for extracting a specific spatial frequency component from the image signal from 0, and the preprocessing filter circuit 1
The two-dimensional phase difference detection circuit 20a detects the motion vector from the above-mentioned residual by comparing the output of 8a with the two-dimensional phase difference between adjacent screens. Motion vector detection system 12b, 12
Similarly, c also includes preprocessing filter circuits 18b and 18c and two-dimensional phase difference detection circuits 20b and 20c.

The two-dimensional phase difference detection circuits 20a and 20b use templates of the same size, but the residual calculation and evaluation methods are different. The two-dimensional phase difference detection circuit 20c sets a template having a size corresponding to the subject image whose motion vector is to be detected, and the two-dimensional phase difference detection circuit 20a.
The motion vector is detected by the same residual calculation and evaluation method as. Of course, the template used by the two-dimensional phase difference detection circuit 20c may have a fixed size and shape, which is different from that of the two-dimensional phase difference detection circuit 20a.

Preprocessing filter circuits 18a, 18b, 18
The frequency characteristic of c is determined by the two-dimensional phase difference detection circuit 20 in the subsequent stage.
It is set so as to supply the optimum spatial frequency components to a, 20b, and 20c. For example, it goes without saying that one of the preprocessing filter circuits 18a and 18b can be omitted if the spatial frequency components required by the two-dimensional phase difference detection circuits 20a and 20b are the same.

Preprocessing filter circuits 18a, 18b, 18
FIG. 2 shows a circuit configuration example when c is configured by a digital circuit.
Illustrated in. In FIG. 2, H is a line memory, D is a pixel memory that adds a delay between adjacent pixels, and a1 to a1
n, b1 to bn, c1 ... are filter coefficients a1 to an, b1 to bn, c
1 ... Multiplying circuit, Σ is an adding circuit. In the circuit of FIG. 2, a specific frequency component is extracted by two-dimensional convolution (masking). Filter coefficient a1 ~
The spatial frequency to be extracted can be adjusted by changing an, b1 to bn, c1 ...

The two-dimensional phase difference detection circuit 20a detects a motion vector from the magnitude (minimum value) of the residual, as in the conventional example. In this detection method, as described in the conventional example, there are cases where erroneous detection or a large error is included. Therefore, in this embodiment, the two-dimensional phase difference detection circuit 20 having different detection characteristics is used.
b and 20c are provided, and further, motion vector detection systems 12d and 12e of different detection methods are provided, the motion vector detection values are compared, and an appropriate motion vector detection value is output.

For example, the two-dimensional phase difference detection circuit 20b is
The difference of each position (pixel) between the screens to be compared is compared with an appropriate threshold value, and the similarity is determined by the number of positions (pixels) exceeding the threshold value. That is, the degree of similarity is determined based on the area of the region in which a significant difference occurs, in other words, the similarity of the shape and size of the subject image is examined.

As described above, the two-dimensional phase difference detection circuits 20a and 20b use templates of the same size, but the two-dimensional phase difference detection circuit 20c uses a smaller template and the two-dimensional phase difference The motion vector is detected by the same detection method as that of the detection circuit 20a.

A more detailed description will be given with reference to FIGS. 3 and 4. FIG. 3 shows an image of a template and a detection range for detecting a motion vector by the template. Pasting rate 30
Within the image 32 of the subject whose motion vector is to be detected
However, on the other hand, in the detection range, a large number of subject images 3
There are 6,38,40,42,44,46,48,50. Here, it is assumed that the subject image 32 and the subject image 40 are before and after the movement of the same subject. Then, the subject image 40
Then, a large subject image 38 is approaching. It is assumed that the subject image 48 is slightly smaller than the subject image 32.

The two-dimensional phase difference detection circuit 20a calculates the residual at each position while moving the template 32 within the detection range 34, and detects the motion vector. Objectively, when the template 30 is at the position (deviation (ξ0, η0)) indicated by reference numeral 30a in FIG. 4, the object images 32 and 40 should almost overlap with each other, and the residual should be almost zero.
However, as the subject image 40 approaches, the calculated residual error increases by the shaded portion 52. The greater the brightness of the subject image 40, the greater the influence of the shaded portion 52 on the residual error.

On the other hand, the template 30 is designated by reference numeral 30 in FIG.
At the position (deviation (ξ1, η1)) indicated by b, the object image 32 and the object image 48 are different only in the shaded portion 48, and the residual is smaller than the residual at the position 30a.

Therefore, the two-dimensional phase difference detection circuit 20a is
Instead of the deviation (ξ0, η0), the deviation (ξ1, η1) is output as a motion vector.

On the other hand, the two-dimensional phase difference detection circuit 20
b detects a motion vector as follows. That is, the shaded portions 52, 54 are set by an appropriate threshold value for the pixel difference.
Can detect a significant difference. The shaded portion 52 is the shaded portion 54
Since the area is smaller, the two-dimensional phase difference detection circuit 20b
Outputs the deviation (ξ0, η0) as a motion vector.

The size of the template does not have to be constant. For example, the two-dimensional phase difference detection circuit 14c sets the template to a size just barely including the subject image 32 of the motion vector detection target. FIG. 5 shows the relationship between the template 56 and the detection range in the two-dimensional phase difference detection circuit 14c. The template 56 is smaller than the template 30, and does not include the subject image 38 at the position (deviation (ξ0, η0)) indicated by reference numeral 56a within the detection range, and thus the residual is also minimized. On the other hand, at the position (deviation (ξ1, η1)) indicated by reference numeral 56b, there is a residual error corresponding to the difference in size between the subject image 32 and the subject image 48, and it does not become the minimum within the detection range. Therefore, the two-dimensional phase difference detection circuit 14c outputs the deviation (ξ0, η0) as a motion vector.

In this way, the plurality of motion vector detection systems 12a, 12b, 12c, 12d, 12e, ...
Detects a motion vector by different detection methods and detection characteristics, and supplies it to the comparison circuit 14. Comparison circuit 14
Compares and calculates the respective detection values, and outputs the final detection value to the output terminal 16 based on a predetermined priority order.

For example, if the detected values of the first, second and third priorities are all different, the detected value of the first highest is output, and the detected value of the first highest priority is the detected value of the second highest and third priority. And the detection values of the second and third priorities match, the detected values of the second and third priorities that match are output, and the detection values of the first and third priorities match. If so, this detection value is output.

Of course, a simple majority vote or weighted averaging may be used. If there is enough time, each motion vector detection system 12a, 12b, 12c, 12d, 12
A plurality of detection values (candidate values) are output from e, and the comparison circuit 1
At 4, the candidate values may be compared and calculated with appropriate weights and priorities.

In both the cross-correlation method and the template matching method, the product and difference of each pixel are evaluated with the same weight in the area (block) for calculating the image similarity. However, most of the entry of other subjects and the overlap with other subjects occur around the edge part of the subject and the periphery of the template, so even if a motion vector detection system that evaluates the center part more heavily than the periphery is added Good.

In the embodiment shown in FIG. 1, the motion vector detection system 12d by the Fourier transform method and the motion vector detection system 12e by the spatiotemporal gradient method are arranged in parallel with the motion vector detection systems 12a, 12b, 12c by the two-dimensional phase difference detection. However, this is merely an example, and it goes without saying that some of the motion vector detection systems 12a to 12e can obtain a certain degree of detection accuracy and detection accuracy.

[0030]

As can be easily understood from the above description, according to the present invention, the motion vector detecting method and the detection characteristic are less likely to affect the motion vector, so that the motion vector can be accurately detected in any image. Like

[Brief description of drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 Pre-processing filter circuits 18a, 18b, 18c
2 is an example of the circuit configuration of FIG.

FIG. 3 is an explanatory diagram of a template and a detection range.

FIG. 4 is an explanatory diagram of template matching.

5 is an explanatory diagram of template matching using a template smaller than that in FIG. 4. FIG.

[Explanation of symbols]

10: Input terminals 12a, 12b, 12c, 12d, 1
2e: Motion vector detection system 14: Comparison circuit 16: Motion vector output terminals 18a, 18b, 18c: Pre-processing filter circuits 20a, 20b, 20c: Two-dimensional phase difference detection circuit 30, 30a, 30b: Template 3
2: subject image 34: detection range 36, 38, 40,
42, 44, 46, 48, 50: subject image 56, 56
a, 56b: template

Claims (1)

[Claims] 1. A plurality of motion vector detection systems for detecting a motion vector from an input image with different detection algorithms and detection characteristics from each other, and the detected values by the respective motion vector detection systems are compared to determine an appropriate motion. A motion vector detection device comprising: a comparison means for outputting a vector.