JPH07288817A - Motion vector detector - Google Patents

Abstract

(57) [Summary] [Object] To provide a motion vector detection device capable of maintaining sufficient performance and reducing the circuit scale when a plurality of motion vector detection units are arranged to expand a search area. A wide search area 10 is secured by providing a plurality of motion vector detection units for predetermined search areas 1 to 4. In the search region 10, the search region 5 of the further motion vector detection unit is arranged in a region where there is a high possibility that a correct motion vector is present, shifted so that the search points increase. As a result, the detection performance increases in an area in which a correct motion vector is likely to exist.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting method and a motion vector detecting device suitable for use in a predictive coding device for an image.

[0002]

2. Description of the Related Art MPEG (Moving Picture Coding Exper)
A predictive coding method is known in which an image is coded by utilizing the correlation with other frames, such as the ts Group) method.
FIG. 29 is an example of such a predictive coding device.

In FIG. 13, image data is supplied to an input terminal 201. This image data is supplied to the motion vector detection circuit 202 and the subtraction circuit 203. The motion vector detection circuit 202 determines a motion vector between frames. This motion vector is supplied to the motion compensation circuit 204.

On the other hand, the image data of the reference frame is stored in the frame memory 205. Frame memory 205
Is output to the motion compensation circuit 204. The motion compensation circuit 204 performs motion compensation on the image data from the frame memory 205 based on the motion vector obtained by the motion vector detection circuit 202. The motion-compensated image data is supplied to the subtraction circuit 203 and the addition circuit 206.

The subtraction circuit 203 is supplied with the image data of the current frame from the input terminal 201 and the image data of the reference frame which has been motion-compensated by the motion compensation circuit 204. The subtraction circuit 203 subtracts the image data of the current frame from the image data of the motion-compensated reference frame to obtain difference data between consecutive frames. This difference data is supplied to the DCT circuit 207. The DCT circuit 207 performs DCT conversion on the difference data. The output of the DCT circuit 207 is supplied to the quantizer 208. The quantizer 208 quantizes the output of the DCT circuit 207. The output of the quantizer 208 is the output terminal 2
It is output from 09.

The DCT-transformed and quantized difference data is used as the inverse quantizer 210 and the inverse DCT circuit 21.
1 and is returned to the original difference data, and the adder 206
Is supplied to. The adder 206 includes a motion compensation circuit 204.
Supplies the image data of the reference frame. Adder 2
At 06, the difference data between the reference frame and the current frame is added to the image data of this reference frame to obtain the image data of the current frame. The obtained image data of the current frame is stored in the frame memory 205 as the next reference frame.

As described above, in the inter-frame predictive coding process, the difference data between the reference frame motion-compensated based on the motion vector and the current frame is coded. A block matching method is known as a motion vector detection method used in such interframe predictive coding processing.

FIG. 14 is for explaining such a block matching method. In FIG. 14, 22
Reference numeral 1 indicates a reference frame, and 222 indicates a search frame. A reference block 223 is set in the reference frame 221, and a candidate block 224 is set in the search frame 222. Candidate block 224 of search frame 222
Are moved within a predetermined search range. Then, the residual is detected, and the reference block 223 of the reference frame 221
It is determined to what degree the candidate block 224 of the search frame 222 matches. The candidate block 224 that best matches the reference block is set as the matching block. A motion vector is obtained from this matching block.

Conventionally, various methods have been proposed for obtaining the residual difference between the reference block and the search block and for moving the search block.

That is, regarding the method of obtaining the residual difference between the reference block and the search block, all the pixels of the reference block and all the pixels of the search block are compared to obtain the residual difference between the reference block and the search block. Is proposed.
Such a configuration for performing all-pixel comparison has a very large amount of calculation and a large circuit scale, but has high detection performance. Further, the reference block and the search block are further divided into a plurality of small blocks, the feature value is extracted from each of the small blocks of the reference block and the search block, and the feature value extracted from the reference block and the feature value extracted from the search block It has been proposed to compare and to obtain the residual. As described above, in the configuration in which the feature values extracted from the small blocks are compared, the amount of calculation can be reduced, but the detection performance is deteriorated.

As a method of moving the search block, it has been proposed that the search block is moved pixel by pixel over the entire search range to obtain a motion vector with one pixel accuracy. In this way, moving one pixel at a time
It is called a full search. In addition, the search block is moved, for example, by 2 pixels in the search range to obtain the approximate value of the motion vector, and the search block is moved by 1 pixel in the vicinity of the obtained motion vector, so that the accuracy of 1 pixel is finally obtained. A method for obtaining a motion vector has been proposed. This type is called a multi-step search, and the one that obtains a motion vector with final accuracy twice in this way is called a two-step search. In the multi-step search, if the initially calculated approximate motion vector deviates from the correct motion vector, the correct motion vector can no longer be reached, but the circuit scale can be reduced as compared with the full search.

From the viewpoint of motion vector detection performance,
All pixel comparison and full search are the best. However, the all-pixel comparison and full-search types have the largest circuit scale. Also, the fact that the circuit scale is large means that
It also leads to higher costs. From the viewpoints of circuit size reduction and cost reduction, feature value extraction comparison and multi-step search are the best. However, the detection performance of the feature value extraction comparison and the multi-step search is not good. Therefore, it is necessary to select the type of motion vector detection circuit according to the required detection performance and circuit scale.

By the way, the search area of the motion vector detection unit that has been conventionally developed is limited. On the other hand, a wider search range is required for predictive coding of a pre-image like the MPEG method. So M
In a motion vector detection device for performing predictive coding of an image such as the PEG method, a plurality of motion vector detection units are arranged side by side and the search range is widened.

As described above, when a plurality of motion vector detection units are arranged to widen the motion vector search range, conventionally, the same type of motion vector detection units are arranged. For example, in order to improve the performance, a plurality of motion vector detection units having high detection performance, such as an all pixel comparison and full search configuration, are arranged. Further, in an attempt to reduce the circuit scale, a plurality of motion vector detection units having a small circuit scale are arranged, such as a feature value extraction / comparison and a multi-step search configuration.

[0015]

However, even if all the motion vector detection units have high detection performance, the performance of all the motion vector detection units is not always effectively utilized. Further, even with a motion vector detection unit having poor detection performance, it is considered that the detection performance can be improved by devising the arrangement method. Furthermore, by combining the motion vector detection unit with high detection performance and the motion vector detection unit with low detection performance, the performance of the motion vector detection unit is effectively used, and the circuit scale is reduced without degrading the detection performance. It is possible to do it.

Therefore, an object of the present invention is to provide a motion vector detecting device which can maintain a sufficient performance and reduce the circuit scale when a plurality of motion vector detecting units are arranged. .

[0017]

According to the present invention, a plurality of motion vector detection units for a first search area are provided so that a motion vector can be detected in a second search area wider than the first search area. In the detection device,
The motion vector detecting device is configured to improve the detection performance in a region where a correct motion vector is likely to exist in the second search region.

According to the present invention, a plurality of motion vector detection units for multi-step search of the first search area are provided,
In a motion vector detection device capable of detecting a motion vector in a second search area wider than the first search area, a multi-step search is performed in an area in which a correct motion vector is likely to exist in the second search area. Is a motion vector detecting device in which the search areas of the motion vector detecting unit are overlapped.

In the present invention, the search areas are overlapped so that the search points are increased.

The present invention provides a plurality of motion vector detection units for detecting motion vectors in the first search area,
In a motion vector detection device capable of detecting a motion vector in a second search area wider than the first search area, the detection accuracy is set in an area in which a correct motion vector is likely to exist in the second search area. This is a motion vector detecting device in which a high motion vector detecting unit is arranged.

In the present invention, the region in which the correct motion vector is likely to exist is determined by determining the number of motion vectors found in the plurality of regions.

[0022]

The motion vector detection performance is improved by overlapping motion vector detection areas or using a motion vector detection unit with high detection accuracy at a position where a correct motion vector is likely to exist. In the area other than the area where the correct motion vector exists, the detection accuracy does not decrease even if the motion vector detection unit has a simple configuration, and the overall circuit scale can be reduced. In addition, the performance of the motion vector detection unit can be extracted without waste depending on the size of the required search area.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. In this example, (−7) pixels to +8
A motion vector detection device is configured by using five motion vector detection units of a two-step search of a search area up to a pixel (hereinafter referred to as (-7 / + 8)).

1A to 1C, 1 to 5 are (-
7 / + 8) motion vector detection units. These motion vector detection units are configured to obtain a motion vector by a two-step search. As shown in FIG. 1A, the search areas 1 to 4 of the four motion vector detection units are arranged so that the search areas are adjacent to each other. Therefore, in this state, the search area 10 that is approximately four times larger is secured. Furthermore, FIG.
As shown in B, another motion vector detection unit is prepared. As shown in FIG. 1C, the search area 5 of this motion vector detection unit is arranged at a position where there is a high probability that a correct motion vector exists, for example, substantially in the center of a search area 10 composed of four motion vector detection units. It At this time, the search area 5 is in a state in which the search pixel is shifted by one pixel from the search areas 1 to 4 of the four motion vector detection units.

By doing so, the drawbacks peculiar to the multi-step search are remedied. That is, the multi-step search is based on the assumption that the residual increases monotonically as the distance from the correct point increases. Figure 2
The characteristics in such an ideal case are shown. In FIG. 2, the horizontal axis indicates the position of the search block, and the vertical axis indicates the residual difference between the reference block and the standard block. The scale on the horizontal axis shows one pixel. In this example, in order to simplify the description, the moving direction of the image is shown in one dimension, and the number of pixels is also shown smaller than it actually is.

As shown in FIG. 2, ideally, there is one minimum value, and a correct motion vector exists at this minimum value. In the case of such an ideal characteristic, since there is only one local minimum value, there is no problem even in the multi-step search. That is, for example, in the case of a two-step search, the search block positions are first in the order of X1, X3, X5 ... (Circle points).
Is moved in 2 pixel steps. When the search block is moved in steps of two pixels in this manner, X5 is detected as a point near the correct answer. Then, the motion vector is detected in the one pixel step around X5, and the correct point X4 is detected.

However, the change of the residual is not limited to such a monotonic case. There may be a local minimum value other than the minimum value indicating the correct point. Figure 3
One such example. In this case, the correct motion vector cannot be detected by the multi-step search. That is, in the case of the two-step search, the position of the search block is first moved in the order of X1, X3, X5, ... When the search block is moved in steps of two pixels in this way, X7 is detected as a point near the correct answer. However, the point at X7 is a local minimum value, and is not a correct answer point. Next, although a motion vector is detected in a one-pixel step around X7, since the motion vector is detected centering on the local minimum value X7, it is impossible to reach the correct point X4.

On the other hand, in one embodiment of the present invention,
On the search area 10 formed by four motion vector detection units, the search pixel is set to 1 so that the search range is located at a position where there is a high probability that a correct motion vector exists.
The search areas 5 of the motion vector detection unit are overlapped with each other in such a state that the search areas 5 are shifted by the amount of pixels. In this state, 2
The step search is as follows.

First, in the search area 10 composed of four motion vector detection units, the search block can be moved in two pixel steps in the order of X1, X3, X5, ... Is detected as a point near. At the same time, in the search area 5, the search block is moved by 2 pixel steps in the order of X2, X4, and X6 (points indicated by triangles), and X
4 is detected as a correct motion vector. The residual of the point X7 detected in the search area 10 is compared with the residual of the point X4 detected in the search area 5. In this case, point X4
Since the residual at is smaller than the residual at point X7,
Is a point near the correct answer. Then, around point X4, 1
The motion vector is detected in the pixel step, and the correct point X4
Is detected.

It should be noted that the search area 5 is not fixed and is shown in FIG.
As shown in, it may be moved to a position where there is a high probability that a correct motion vector exists.

As a method of superimposing the search area composed of four motion vectors and the search area 5, FIG.
As shown in FIG. 5, the pixels may be arranged on the fifth eye, and the pixels may be aligned in the vertical direction as shown in FIG. 5B, or the pixels may be aligned in the horizontal direction as shown in FIG. 5C.

FIG. 6 shows an embodiment of the present invention. In FIG. 6, 11 to 15 are motion vector detection units. These motion vector detection units 1
1 to 15 are for detecting a motion vector by a two-step search, and the search area is (-7 / + 8)
It is said that. Motion vector detection units 11-15
Are integrated circuits or modules. The motion vector detection units 11 to 15 have an input port P1 for data in the search area, an input port P2 for the search area delayed by 15H, and an input port P3 for the reference block.

Reference numeral 21 is a search frame memory. Image data in the search area is read from the search frame memory 21. Reference numeral 22 is a reference frame memory. The image data of the reference block is read from the reference frame memory 22.

Port P of the motion vector detection unit 11
The output of the search frame memory 21 is supplied to 1, and the output of the search frame memory 21 via the 15H delay circuit 23 is supplied to the port P2. Further, the output of the reference frame memory 22 is supplied to the port P3 of the motion vector detection unit 11.

Port P of motion vector detection unit 12
1 is supplied with the output of the search frame memory 11 via the 225 sample delay circuit 24, and 1 is supplied to the port P2.
The output of the search frame memory 11 via the 5H delay circuit 23 and the 225 sample delay circuit 25 is supplied. In addition, the port P3 of the motion vector detection unit 12 is
The output of the reference frame memory 22 is supplied.

Port P of motion vector detection unit 13
The output of the search frame memory 21 via the 15H delay circuit 23 is supplied to 1, and the output of the search frame memory 11 via the 15H delay circuit 23 and the 15H delay circuit 26 is supplied to port P2. Further, the output of the reference frame memory 22 is supplied to the port P3 of the motion vector detection unit 13.

Port P of motion vector detection unit 14
1 is supplied with the output of the search frame memory 21 via the 15H delay circuit 23 and the 225 sample delay circuit 27, and the port P2 is supplied with the 15H delay circuit 23, the 15H delay circuit 26 and the 225 sample delay circuit 28. The output of the search frame memory 11 is supplied. Further, the output of the reference frame memory 22 is supplied to the port P3 of the motion vector detection unit 14.

Port P of motion vector detection unit 15
1 is supplied with the output of the search frame memory 21 via the variable delay circuit 30, and port P2 is supplied with the output of the search frame memory 21 via the variable delay circuit 30 and the 15H delay circuit 31. The reference frame memory 2 is connected to the port P3 of the motion vector detection unit 15.
2 outputs are provided. The variable delay circuit 30 is, for example,
It is composed of a FIFO. The delay amount of the variable delay circuit 30 is controlled by the control signal from the input terminal 32.

The motion vector detection units 11 to 14 constitute a motion vector detection circuit whose search area is quadrupled. The motion vector unit 15 is a state in which the search pixel is displaced by one pixel at a position where there is a high probability that a correct motion vector exists on the search area of the motion vector detection circuit configured by the motion vector detection units 11-14. Then, the motion vector is detected.

The outputs of the motion vector detection units 11-14 are supplied to the minimum value detection circuit 33. Further, the control signal from the input terminal 32 is supplied to the minimum value detection circuit 33. The minimum value detection circuit 33 detects the motion vector using the outputs of the motion vector detection units 11 to 15. The obtained motion vector is output from the output terminal 34.

In the above description, the search area is expanded by the four motion vector detection units, and the search of one motion vector detection unit is performed on the search area of the motion vector detection circuit constituted by the four motion vector detection units. Although the regions are located, it is not necessary to fix the role of each motion vector detection unit. It is possible to adaptively change the role of each motion vector detection unit according to the size of the required search area and the detection performance.

For example, in the case of performing the predictive coding like the MPEG system, the required search range changes depending on the distance from the reference picture. In such a case, it is effective to adaptively change the role of each motion vector detection unit.

FIG. 7 shows a preferred embodiment used in MPEG or the like. As shown in FIG. 7, ten motion vector detection units are prepared. As shown in FIG. 7A, when the distance from the reference picture is large, the search areas 51 to 59 of the nine motion vector detection units secure 9 times as many search areas as the search areas 51 to 59. The search region 60 is overlapped on the region at a position where there is a high possibility that a correct motion vector exists, with the pixel shifted by one pixel.

As shown in FIG. 7B, when the distance from the reference picture is smaller than that, four search vectors are secured by the four motion vector detection units 51 to 54, and four motion vectors are provided on the search area. The search areas 55 to 58 of the detection unit are overlapped with each other with one pixel shifted.

As shown in FIG. 7C, when the distance from the reference picture is small, the two motion vector detection units 51 and 52 are overlapped with each other with one pixel shifted.

In the above description, two search areas are overlapped, but three search areas may be further overlapped. FIG. 8 shows an example. In FIG.
In the central area 61, three search areas are overlapped, in the central peripheral area 62, two search areas are overlapped, and in the peripheral area 63, the search areas are not overlapped. By doing so, the detection accuracy is improved in the central portion where the correct motion vector is likely to exist.

FIG. 9 shows another embodiment of the present invention. In the above embodiments, the motion vector detection units are all of the same type. On the other hand, in this embodiment, a plurality of types of motion vector detection units with different detection accuracy are used.

In FIG. 9, 71 to 79 are search areas of the motion vector detecting unit. The search areas 71 to 79 are arranged so that the search areas are adjacent to each other. As a result, 9 times as many search areas 80 can be secured.

Among the motion vector detection units forming the search areas 71 to 79, the motion vector detection unit in the central area 75 where the correct motion vector is likely to exist is considered to have high detection performance. . For example, the motion vector detection unit in the central area 75 has a configuration of all pixel comparison and full search. Other area 7
The motion vector detection unit forming 1, 72, 73, 74, 76, 77, 78, 79 may have a small circuit scale. Areas 71, 72, 73, 7
As the motion vector detection unit forming 4, 76, 77, 78, 79, for example, a feature value extraction / comparison unit and a multi-step search unit are used. In this way, the motion vector is reliably detected in a region in which there is a high probability that a correct motion vector exists, so that the drawback peculiar to the multi-step search is improved.

The position at which the motion vector detecting unit with high detection performance is arranged can be changed according to the position where the correct motion vector exists. FIG. 10 shows control for changing the position of the motion vector detection unit with high detection accuracy. In addition, each search area M1
M9 is set as shown in FIG.

First, a highly accurate motion vector detection unit is set in the search area M5 (step 81), and a motion vector is detected (step 82). Each area M1
The number of motion vectors in M9 is calculated (step 8)
3). The number of motion vectors in the region M5 is the threshold value Vth1
It is determined whether it is greater than (step 84). If the number of motion vectors in the area M5 is larger than the threshold value Vth1, a highly accurate motion vector detection unit is placed in the area M5 (step 85). If the number of motion vectors in the area M5 is smaller than the threshold value Vth1, each area M1
A region Mx having the maximum number of motion vectors is detected from among M9 to M9, and the number of motion vectors in this region Mx is the threshold value V.
It is determined whether it is larger than th2 (step 8).
6). The number of motion vectors in the area Mx is the threshold value Vth2
If it is smaller, the procedure goes to step 85, and a highly accurate motion vector detection unit is arranged in the area M5. Area Mx
If the maximum value of the number of motion vectors in is larger than the threshold value Vth2, a highly accurate motion vector detection unit is arranged in the area Mx (step 87).

FIG. 12 shows the configuration of another embodiment of the present invention. In FIG. 12, 101 to 109 are
It is a motion vector detection unit. Of these motion vector detection units 101 to 109, the motion vector detection unit 101 is of a highly accurate type, for example, all-pixel comparison and full search configuration, and the other motion vector detection units 102 to 109 are circuits. The scale is reduced, for example, feature value extraction comparison and multi-step search. Motion vector detection units 101 to 1
From 09, data input port P1 of the search area and 1
It has an input port P2 of the search area delayed by 5H and an input port P3 of the reference block.

Reference numeral 111 is a search frame memory, and the pixel data in the search range is read from the search frame memory 111. 112 is a reference frame memory,
Pixel data of the reference block is read from the reference frame memory 112.

121 to 123 are 16H delay circuits, 131
˜138 are 256 sample delay circuits. The output of the search frame memory 111 is the 16H delay circuit 131.
˜138 and 256 sample delay circuits 131 to 138 and supplied to the 12:18 multiplexer 140.
The 16H delay circuits 131 to 138, the 256 sample delay circuits 131 to 138, and the multiplexer 40 set the search area. Corresponding to the set search area, the data input port P1 of the search area of each of the motion vector detection units 101 to 109 and the input port P2 of the search area delayed by 15H are connected to the data of the search area and the search area delayed by 15H. Data is supplied.

The output of the reference frame memory 112 is supplied to the input port P3 of the reference block of each motion vector detection unit 101-109.

Motion vector detection units 101-109
Is supplied to the minimum value detection circuit 141. In the minimum value detection circuit 141, the motion vector detection units 101 to 1
The minimum of the residual output from 09 is detected, and the motion vector is obtained from this. The obtained motion vector is output from the output terminal 142 and is also supplied to the motion vector number detection circuit 143. The output of the motion vector number detection circuit 143 is supplied to the control determination circuit 144.

The control determination circuit 144 determines an area in which the highly accurate motion vector detection unit 101 is located based on the control shown in FIG. The output of the control judgment circuit 144 is the multiplexer 14
0, the search area of the highly accurate motion vector detection unit 101 is located at a position where there is a high probability that a correct motion vector exists.

[0058]

According to the present invention, the motion vector detection unit is superposed at the position where the correct motion vector is likely to exist, or by using the motion vector detection unit having high detection accuracy, The detection performance is improved. In the area other than the area where the correct motion vector exists, the detection accuracy does not decrease even if the motion vector detection unit has a simple configuration, and the overall circuit scale can be reduced. In addition, the performance of the motion vector detection unit can be extracted without waste depending on the size of the required search area.

[Brief description of drawings]

FIG. 1 is a schematic diagram used to describe an embodiment of the present invention.

FIG. 2 is a graph used to explain one embodiment of the present invention.

FIG. 3 is a graph used to explain one embodiment of the present invention.

FIG. 4 is a schematic diagram used to describe a modified example of the present invention.

FIG. 5 is a schematic diagram used to describe a modified example of the present invention.

FIG. 6 is a block diagram of an embodiment of the present invention.

FIG. 7 is a schematic diagram used to describe a modified example of the present invention.

FIG. 8 is a schematic diagram used to describe a modified example of the present invention.

FIG. 9 is a schematic diagram used to describe another embodiment of the present invention.

FIG. 10 is a flowchart used for explaining another embodiment of the present invention.

FIG. 11 is a schematic diagram used to describe another embodiment of the present invention.

FIG. 12 is a block diagram of another embodiment of the present invention.

FIG. 13 is a block diagram of an example of a conventional encoding device.

FIG. 14 is a schematic diagram used to describe a conventional motion vector detection method. 1-5 Search area of motion vector detection unit 11-15 Motion vector detection unit 30 Variable delay circuit

Claims (5)

[Claims] 1. A plurality of motion vector detection units for the first search area are provided, and a second vector wider than the first search area is provided.
Motion vector detecting apparatus capable of detecting a motion vector in the second search area, the motion vector detecting apparatus increasing detection performance in an area in which a correct motion vector is likely to exist in the second search area. apparatus. 2. A motion vector detecting apparatus comprising a plurality of motion vector detection units for multi-step search of a first search area, wherein the motion vector can be detected in a second search area wider than the first search area. A motion vector detecting device in which the search areas of the motion vector detecting unit of the multi-step search are overlapped in an area in which a correct motion vector is likely to exist in the second search area. 3. The motion vector detection device according to claim 2, wherein the search areas are overlapped so that search points are increased. 4. A motion vector detecting device, wherein a plurality of motion vector detecting units for detecting a motion vector in a first search area are provided, and the motion vector can be detected in a second search area wider than the first search area. In the motion vector detection device, a motion vector detection unit having high detection accuracy is arranged in a region where a correct motion vector is likely to exist in the second search region. 5. The motion according to claim 1, wherein the region in which the correct motion vector is likely to exist is determined by determining the number of motion vectors found in a plurality of regions. Vector detector.