JP2001222549A - Moving image retrieving device - Google Patents

Abstract

(57) [Summary] [Purpose] Using a key image representing a target object to be searched, a frame image showing the target object is searched from a moving image. As shown in FIG. 1, the moving image search device has a computer system 1, a video player 2, and a visual device 3. As shown in FIG. 2, the visual device 3 uses the key image and the area designation image to generate a normalized image 145 of the target object.
And a normalized image 145 of the moving object is generated using the moving image.
Is generated, and the normalized image 145 is compared, so that the moving image search device can search the moving image for a frame image showing the target object. As shown in FIG. 3, the visual device 3 can also search using only the key image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image retrieval apparatus for retrieving an object in a still image designated by a user from a moving image recorded in a video reproducing machine, a moving image database or the like. A normalized image is generated from the still image specified by the user using the device, a normalized image of the object in the moving image is generated using the visual device, and the two normalized images are compared. To determine the presence or absence of

[0002]

2. Description of the Related Art Hitherto, a large number of devices for searching for a frame image and a sequence of frame images in a moving image have been studied (for example, Japanese Patent Application Laid-Open Nos. 5-20366 and 5-304).
64). By extracting feature data from the entire frame in the moving image and comparing these feature data with the key feature data of the still image and the moving image, these search devices can detect the frame image and the sequence of frame images. Is to search. Therefore, depending on how the feature data is created, the feature data is affected by the background, etc.
These search devices cannot find the desired frame image and the sequence of frame images. In order to avoid such a problem, a device for searching for a frame image showing the moving object by extracting the moving object in the moving image and comparing the feature amount data has been developed (for example, Japanese Patent Laid-Open No.
0-207897, JP-A-10-210473, JP-A-10-214347 and JP-A-2000-010997). However, these search apparatuses use MPEG (Moving Picture E) to extract moving objects.
In order to use the xpert group data, the means for extracting the moving object cannot not only extract the moving object itself to be actually searched, but also the means for calculating the feature amount data sets the average value of the luminance values of the extraction area. In order to calculate such as feature amount data, the means for executing matching cannot accurately compare the target object to be searched with the moving object.

On the other hand, the present inventor has developed a visual device as a device for recognizing the type of an object in a moving image (Japanese Patent Application No. Hei 11 (1999) -107).
253634). This vision device generates edge information for moving and stationary objects regardless of their shapes, separates the object region from the background, and normalizes the separated object region to the image size. And recognize the type of the object. Moreover, the visual device can realize high parallelism and real-time performance by performing local parallel image processing.

[0004] In view of these things, the visual device,
By generating a normalized image of a target object to be searched and a normalized image of a moving object and a still object in a moving image, and comparing these normalized images, the moving image search apparatus can generate a moving image similar to the target object. It is expected that moving and stationary objects inside can be searched.

[0005]

SUMMARY OF THE INVENTION Therefore, according to the present invention, a visual device uses a key image representing a target object to be searched and a moving image to generate a normalized image of the target object in the key image and a moving image. An object of the present invention is to generate a normalized image of a moving object and a stationary object in an image and compare these normalized images to search for a frame image of the target object from the moving image.

[0006]

According to a first aspect of the present invention, there is provided a moving image search apparatus for searching a moving image for a frame image showing a target object, wherein: a means for obtaining the frame image; Means for sequentially storing as a digital image, means for generating a coarse edge information image from the digital image, means for forming the coarse edge information image into a first formed edge information image using the digital image, and Means for generating a first object region image from one formed edge information image, means for generating a duplicated information image from the first formed edge information image, and selecting one moving object using the duplicated information image Means for extracting only the moving object selected from the first object area image by mask processing, and the digital image and the masked first object Means for generating a first area normalized image from the area image, means for holding the first area normalized image, means for storing a key image specifying the target object, and storing an area designation image Means for forming the area designation image into a second formed edge information image using the key image, means for generating a second object area image from the second formed edge information image, Means for generating a second area normalized image from an image and the second object area image, means for comparing the first area normalized image with the second area normalized image, and holding a comparison result And a means for searching for a moving image. The present invention provides a M
PEG (Moving Picture Expert)
Without using a special image format such as Group data, the frame image that shows the moving object similar to the target object is searched from the moving image.
The visual device generates the second normalized image from the key image specifying the target object and the region designation image, and generates the first normalized image from the moving object in the moving image , And the first normalized image is compared with the second normalized image. The visual device performs local parallel image processing. Therefore, according to the present invention, the frame image showing the target object can be searched in real time without using the special image format, and various problems relating to moving image search can be suitably solved.

According to a second aspect of the present invention, there is provided a moving image search device for searching a moving image for a frame image showing a target object, wherein the means for acquiring the frame image and the frame image are sequentially stored as a digital image. Means, means for generating a first coarse edge information image from the digital image, means for forming the first coarse edge information image into a first formed edge information image using the digital image, and Means for generating a first object region image from one formed edge information image, means for generating a duplicated information image from the first formed edge information image, and selecting one moving object using the duplicated information image Means for extracting only the moving object selected from the first object area image by mask processing, and means for extracting the digital image and the masked first object area. Means for generating a first area normalized image from an image, means for holding the first area normalized image, means for storing a key image specifying the target object, and means for vibrating the key image Means for generating a second coarse edge information image from the vibrated key image, and means for forming the second coarse edge information image into a second formed edge information image using the key image, Means for generating a second object area image from the second formed edge information image, means for generating a second area normalized image from the key image and the second object area image, and A moving image search device comprising: means for comparing an area-normalized image with the second area-normalized image; and means for holding a comparison result. The present invention uses a visual device to provide an MPEG (Moving Pic).
The moving image is searched for the frame image showing the moving object similar to the target object, without using a special image format such as “Two Expert Group” data. The visual device generates the second coarse edge information image from the key image identifying the target object, generates the second normalized image from the key image and the second coarse edge information image Generating the first normalized image from the moving object in the moving image, and comparing the first normalized image with the second normalized image. The present invention is particularly effective when the background of the key image is monotone. The visual device performs local parallel image processing. Therefore, according to the present invention, the frame image showing the target object can be searched in real time without using the special image format, and various problems relating to moving image search can be suitably solved.

According to a third aspect of the present invention, there is provided a moving image search apparatus according to the first and second aspects, further comprising means for vibrating the digital image. The present invention uses a visual device to create an MPEG
(Moving Picture Expert Gr
(out) Searching the moving image for the frame image showing the moving object and the stationary object similar to the target object without using a special image format such as data. The visual device generates the second normalized image from the key image specifying the target object, the area designation image, and the second coarse edge information image, and vibrates the frame image of the moving image Thereby, the first normalized image is generated from the moving object and the stationary object in the moving image, and the first normalized image is compared with the second normalized image. The visual device performs local parallel image processing. Therefore, the present invention can search for the frame image showing the moving or stationary target object in real time without using the special image format, and suitably solve various problems related to moving image search. Is done.

[0009]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a moving picture search apparatus according to the present invention.

First, the invention according to claim 1 is represented by a configuration as shown in FIG. Here, the manufacturer of the present invention can use a commercially available product for the computer system 1 and the video player 2, or can easily manufacture it by an existing technology. In the present invention, a commercially available moving image database may be used in place of the video player 2, and the moving image database may be incorporated in the computer system 1. Further, the computer system 1 can connect drawing tools such as a tablet and a touch panel in addition to the mouse. Hereinafter, description will be made with reference to FIG.

The computer system 1 transmits to the visual device 3 a key image representing a target object to be searched and a key designation signal for controlling transmission of the key image. If necessary, the user designates an outline of an area including the target object in the key image using a mouse while viewing the key image displayed on the display, and the computer system 1 converts the image into an area designation image. To the visual device 3. Since this contour is formed by the visual device 3, it may be a broken line. The video player 2 transmits the moving image being played back to the visual device 3 according to the control signal from the computer system 1 and also sends the computer system 1 the frame number of the frame image currently being played back or the playback time from the beginning of the recording medium. Send The visual device 3 receives the moving image transmitted from the video player 2 and the key image, the key designation signal and the region designation image transmitted from the computer system 1, and outputs the target object and the frame image specified by the key image. Are compared, and the result of the comparison is returned to the computer system 1. Therefore, the computer system 1 determines whether or not the target object exists in the frame image using the comparison result.

Finally, the visual device 3 for general use
Each algorithm has already been developed, but needs to be added and changed so as to be suitable for a moving image search device. Therefore, the visual device 3 will be mainly described below.

As shown in FIG. 2, an embodiment of the visual device 3 receives a moving image (frame image) from the video player 2 and converts the moving image (frame image) into a digital image 111 of an appropriate format and size.
Image acquisition means 11 for converting the digital image 111 into a predetermined period, edge information generation means 14 for generating coarse edge information 112 of the moving object from the two digital images 111, and coarse edge information 112 Information forming means 15 for forming a more accurate and clear formed edge information 114, an object / background separating means 16 for separating an area divided by the formed edge information 114, and each area divided by the formed edge information 114 Position / size detecting means 17 for detecting the position and size of the object, position selecting means 22 for selecting one of a plurality of moving objects detected by the position / size detecting means 17, and object / background separating means A masking means 40 for masking a region other than the moving object selected by the position selecting means 22 with respect to the area separated by 16; An area normalization means 27 for normalizing the frequency to the image size, the normalized image 14 to be output to the output destination
5, an image storage unit 12 that receives a key image from the computer system 1 and stores it for a certain period of time, and an image storage unit that receives an area designation image from the computer system 1 and stores it for a certain period of time 12
And more precisely and clearly formed edge information 1
14, an object / background separating unit 16 for separating an area divided by the formed edge information 114, and an area normalizing unit for normalizing a separated area of the key image to an image size. 27, a pattern comparing means 41 for comparing the normalized image 145 generated from the frame image with the normalized image 145 generated from the key image.
And a comparison result holding means 42 for holding the comparison result until output to the output destination.

When the image acquiring means 11 inputs a frame image of a moving image from the video reproducing device 2, if the moving image is an analog signal, the frame image is converted into a digital signal by A / D conversion and the digital image 111 To If the moving image is a digital signal, it is expanded if it is compressed, and is input as it is if it is not compressed. With these, any frame image can be cut out from the moving image,
This frame image is cut out to be a digital image 111. Since the converted digital image 111 has an arbitrary image size while following an appropriate format, the image acquisition unit 11 converts the image data into a format in which image data can be referred to in pixel units, and The minute is cut out and output as a digital image 111.
If the image acquisition means 11 can output all the pixels of the digital image 111 in parallel,
Can be performed in parallel for each pixel.

When the image storage unit 12 inputs the digital image 111 from the image acquisition unit 11, the digital image 111 is stored for a certain period of time in accordance with the time resolution of the visual device 3 or the calculation capability of each unit. That is, even if the digital image 111 is input during this fixed time, the image storage unit 12 does not change the stored image, so that each subsequent unit can input the same digital image 111 at a different timing. Moreover, the image storage means 12 stores the digital image 1
11 is not subjected to image processing, the digital image 1
All two pixels are stored while maintaining a two-dimensional phase relationship. If the image storage means 12 can output all the pixels of the digital image 111 in parallel,
Communication from the image storage unit 12 to the edge information generation unit 14 can be performed in parallel for each pixel. Since the key image and the area designation image are digital images 111, the image storage means 12 can directly input the key image and the area designation image from the computer system 1.

The edge information generating means 14 is used by the image storing means 1
When the digital image 111 is input from Step 2, a coarse edge information image 113 of the moving object is generated by comparing the digital image 111 with the digital image 111 input immediately before. The edge information generation means 14 performs only the neighborhood processing for each pixel and executes the coarse edge information image 1
13 can be generated, which is suitable for parallelization. If the edge information generating means 14 determines that the coarse edge information image 1
If all 13 pixels can be output in parallel, the communication from the edge information generating means 14 to the edge information forming means 15 can be performed in parallel for each pixel.

When the edge information forming means 15 inputs the coarse edge information image 113 from the edge information generating means 14, the rough edge information image 113 is accurately and clearly referred by referring to the digital image 111 stored in the image storage means 12. Is formed on the formed edge information image 115 of the moving object. Since the edge information forming unit 15 can form the formed edge information image 115 by only the neighborhood processing for each pixel, it is suitable for parallelization. If the edge information forming means 15 can output all the pixels of the formed edge information image 115 in parallel, the communication from the edge information forming means 15 to the object / background separating means 16 and the position / size detecting means 17 is as follows. It can be performed in parallel for each pixel. Edge information forming means 1
No. 5 regards the area designation image as the rough edge information image 113 and forms the formed edge information 114.

The object / background separating means 16 receives the formed edge information image 115 from the edge information forming means 15,
Pixels included in the object region 141 of the moving object and pixels included in the background are separated into different groups, and the separation results are sequentially output in group units. When the object areas 141 are clearly distinguished by the formed edge information 114 even though they are adjacent to each other, the object / background separation unit 16 can separate these object areas 141 into different groups. Therefore, the number of groups may be three or more. The object / background separation means 16 is suitable for parallelization because the object region 141 and the background can be separated by only neighborhood processing for each pixel. If the object / background separation means 16 can output all the pixels of the object area image 142 in parallel, the communication from the object / background separation means 16 to the mask means 40 and the area normalization means 27 is performed in parallel for each pixel. Can be done.

When the position / size detecting means 17 inputs the formed edge information image 115 from the edge information forming means 15, it detects the position and size of the area of the moving object indicated by the formed edge information 114. The position / size detecting means 17 performs only neighborhood processing for each pixel, and the overlapping information image 132 representing the detection result of the position and size of the area of the moving object.
Is suitable for parallelization. If the position / size detecting means 17 can output all the pixels of the overlapping information image 132 in parallel, the communication from the position / size detecting means 17 to the position selecting means 22 should be performed in parallel for each pixel. Can be.

The position selecting means 22 determines that the total number of positions of the moving object region in the frame image represented by the overlapping information image 132 is 2
If more than one, one is selected from a plurality of positions.
The position selecting means 22 selects one position according to a certain criterion. The criteria used here are mainly as follows. First, the one with the largest moving object is selected. If the moving object is small, there is a possibility that the edge information generating means 14 has generated noise. Therefore, an object with a high probability that the moving object is present is selected from a plurality of positions. Second, when a plurality of positions are concentrated within a certain range, one of them is selected. There are two possibilities for this situation. One is the possibility that the edge information generating means 14 has generated the coarse edge information 112 dispersed for one moving object, and the other is the possibility that a plurality of moving objects actually exist. Third, if many positions have substantially the same size, the position closest to the center of the frame image is selected. By appropriately combining these in accordance with the application and the situation, the position selecting means 22 can select one from a plurality of positions on the environment coordinate system. In particular, since a moving object is reflected over a large number of frame images, it is possible to select a plurality of moving objects in the frame image by using different judgment criteria for each frame image.

The mask means 40 includes the object / background separation means 1
When the object area image 142 and the overlapping information 131 of the selected moving object are input from the position selecting unit 22 and the position selecting unit 22, respectively, the position and the size of the moving object are calculated from the overlapping information 131, and a circular or rectangular area is set. Object area image 14
2 is masked except for this area. This allows the object /
Even if the background separating unit 16 separates a plurality of moving object regions into the same group, these regions can be handled individually. If the masking means 40 can output all the pixels of the masked object area image 142 in parallel, the communication from the masking means 40 to the area normalizing means 27 can be performed in parallel for each pixel.

The area normalizing means 27 is an object area image 142 masked by the mask means 40 and the image acquiring means 11.
And the digital image 111 are input, the separated object region 143 of the moving object is cut out from the digital image 111, and the separated object region 143 is deformed and complemented and enlarged as much as possible in accordance with the image size of the digital image 111. The generated image 145 is generated. The region normalizing means 27 can normalize the separated object region 143 only by the neighborhood processing for each pixel, and is suitable for parallelization. If the area normalizing means 27 can output all the pixels of the normalized image 145 in parallel, the area normalizing means 27
From the normalized image holding means 28 and the pattern comparing means 41
Can be performed in parallel for each pixel. In the case of a key image, if there is only one target object to be searched, it is not necessary to use the masking means 40 in particular.

When the normalized image 145 is input from the area normalizing means 27 to the normalized image holding means 28, if the output destination of the normalized image 145 is the normalized image 1 having an appropriate format,
If a request is made for the normalized image 145, the normalized image 145 is converted into a format required by the output destination of the normalized image 145. Thereafter, the normalized image holding unit 28 stores the normalized image 145 for a certain period of time until the normalized image 145 is reliably transmitted to the output destination of the normalized image 145. If the format to be converted is limited, the normalized image holding unit 28 can convert the normalized image 145 by only the neighborhood processing for each pixel, and is suitable for parallelization. If the normalized image holding unit 28 can output all the pixels of the normalized image 145 in parallel, the communication from the normalized image holding unit 28 to the output destination of the normalized image 145 is performed in parallel for each pixel. be able to.

The pattern comparing means 41 inputs and stores the normalized image 145 generated from the key image according to the key designation signal, and then inputs the normalized image 145 generated from the frame image, and inputs the two normalized images. 145 is compared in pixel units and neighborhood units, converted to a scalar value using a general method such as mean square error and inner product, and output as a comparison result.

When the comparison result holding unit 42 receives the comparison result of the normalized image 145 from the pattern comparison unit 41, the comparison result is stored for a certain period of time until the recognition result is reliably transmitted to the output destination of the comparison result.

When the digital image 111 vibrates finely, the object in the digital image 111 looks as if it is moving. That is, when the visual device 3 vibrates the digital image 111 finely, the visual device 3 can create a moving image in which an object in the digital image 111 is moving.

The image vibration means 13 receives the digital image 111 from the image storage means 12,
The object moves simultaneously in units of images or individually in units of pixels such that the object vibrates up and down and left and right within a range of about 3 pixels in 1. If the image oscillating unit 13 can output all the pixels of the digital image 111 in parallel, the communication from the image oscillating unit 13 to the edge information generating unit 14 can be performed in parallel for each pixel.

Therefore, as shown in FIG. 3, the visual device 3 according to the second aspect uses the edge information generating means 14 to convert the key image vibrated by the image vibrating means 13 into a coarse edge information image 113. Generate the specified image. In particular, if the key image shows only one target object to be searched and the background is monotone, the edge information generating means 14 can clearly outline the outline of the target object in the key image.

Next, as shown in FIGS. 4 and 5, the visual device 3 according to the third aspect of the invention vibrates the digital image 111 by the image
The stationary object inside can be regarded as a moving object. Therefore, the visual device 3 can generate the normalized image 145 even for a stationary object.

The image storage means 12, the image vibration means 13, the edge information generation means 14, the edge information formation means 15, the object / background separation means used in the visual device 3 according to any one of claims 1 to 3. 16, the position / size detecting means 17, the area normalizing means 27, and the normalized image holding means 28 include an array operation unit 100 (ARRAY OPERATIONUNI).
This can be implemented by using the data processing device 110 configured from T). Therefore, an embodiment of the data processing device 110 using the array operation unit 100 will be described below, and these units will be described with reference to the drawings.

First, the array operation unit 100 generates one pixel of the output image by using one pixel of the input image and its neighboring pixels. Therefore, as shown in FIG. 6, by using the data processing device 110 in which the array operation units 100 are arranged in a lattice in accordance with the size of the input image, the data processing device 110 can generate an output image from the input image. it can. In FIG. 6, the array operation unit 100 is abbreviated as AOU. Next, the array operation unit 100 may be implemented by dedicated hardware, or may be implemented by software on a general-purpose computer. That is, as long as the output image can be generated from the input image, the mounting means is not limited. Therefore, the image processing of the data processing device 110 can be indicated by indicating the algorithm of the array operation unit 100. Therefore, in order to show the algorithm of the array operation unit 100, the image storage unit 12, the image vibration unit 13, the edge information generation unit 14, and the image storage unit 12 shown in FIGS.
Edge information forming means 15, position / size detecting means 17,
The mathematical formulas used in the area normalizing means 27 and the normalized image holding means 28 will be described.

If an arbitrary ²ⁿ gradation image having a width w, a height h, and the number of bands b is x , y , and w , x , y , and w are respectively located at positions p.
(I, j, k) band pixel values x _ijk , y _ijk , w
Expressions 1, 2, and 3 are given by using _ijk . The underlined characters indicate vectors. N is a non-negative integer, w, h, b, i, j, and k are natural numbers.

[0033]

(Equation 1)

[0034]

(Equation 2)

[0035]

(Equation 3)

First, a function relating to point processing for each band pixel value of the image will be described below.

When converting the image x into a binary image, the following expression 4 is used.
, The band pixel value is binarized.

[0038]

(Equation 4)

ImagexIs converted to a band maximum value image,
The maximum value among the values of each band of the pixel in the i-th row and the j-th column in accordance with Expression 5
Select a value. The band maximum value image is a single band image.
Therefore, for convenience, it can be handled as the image with 1 band.
And Therefore function B _ij1(x) Is the third subscript
It is 1.

[0040]

(Equation 5)

[0041] As the image x is a binary image, when reversing the image x, is calculated according to Equation 6.

[0042]

(Equation 6)

The logarithmic transformation at the position p (i, j, k) of the image x is performed according to the following equation (7). Here, e is an offset, and is used to make the value output by the natural logarithmic function fall within the effective range, so that e = 1 is generally sufficient. By this logarithmic conversion, division between band pixel values can be subtracted. If the image x is a digital image 111 of ²ⁿ gradations, it is necessary to calculate the natural logarithmic function every time if the memory 102 has a look-up table including ²ⁿ elements regardless of the number of bands. No need to have a standard log table.

[0044]

(Equation 7)

Now, at the position p (i, j, k) of the image,
Set P near q_ijk(Q) is given by Equation 8.
expressed. Where q is 4, 8, 24, 48, 80, 12
0, (2r + 1)²Is a sequence that follows -1, where r is a natural number
It is. The position beyond the image size is set P
_ijkWhen included in (q), unless otherwise specified
The position p (i, j, k) shall be substituted. Also this
Otherwise, according to the specification, the pixel value corresponds to 0,
Also, a fictitious position not included in the image is substituted. This
Edge processing is performed automatically. Therefore the set P_ijk
The number N of elements of (q) _ijkIs always q.

[0046]

(Equation 8)

Next, functions and operators related to neighborhood processing of up to eight neighborhoods for each band pixel value of an image will be described below.

The vibration at the position p (i, j, k) of the image x is performed in accordance with the equation (9). Here, the position p (i, j,
By virtue of the method of selecting only one position from the vicinity of q in k), it is possible to determine whether to vibrate in image units or in pixel units. If at all positions of the image x, by selecting one of the q vicinity by exactly the same method, the image x vibrates in image units. On the other hand in each position of the image x in, selecting one of the q vicinity randomly using a pseudo-random number or the like, the image x vibrates in pixel units.

[0049]

(Equation 9)

The smoothing at the position p (i, j, k) of the image x is performed according to the equation (10). Where int (v)
Means truncation of the real number v below the decimal point.
If the band pixel value of the image x is an integer value, a right shift instruction is performed twice for the sum of x _lmk when N _ijk = 4 and x _l when N _ijk = 8 when hardware is implemented.
By changing the circuit to execute the right shift instruction three times with respect to the sum of _mk , the circuit for executing the division can be omitted.

[0051]

(Equation 10)

The calculation of Laplacian is a simple second order difference operator as shown in Expression 11. 8
The vicinity is more suitable for the present invention because the number of zero points and zero crossings is increased by catching subtle changes in noise. Where N
_{Since ijk} is 4 or 8, the left shift instruction is _executed twice for x _ijk when N _ijk = 4, and the left shift instruction is _executed three times for x _ijk when N _ijk = 8 when the hardware is implemented. By changing to a circuit that performs the multiplication, a circuit that performs multiplication of real numbers can be omitted.

[0053]

[Equation 11]

As a method of finding a zero point from the value obtained by Laplacian, a pixel that changes from positive to negative has been conventionally found. Try to find a pixel that goes through a zero point, such as going from negative to zero or from zero to positive, or continuing zero. In the present invention, the zero point found by Expression 12 is not a place where an edge exists, but a place where there is noise, that is, a place where there is no edge. In addition, binarization of the real value is performed at the same time according to Expression 12.

[0055]

(Equation 12)

Assuming that the image x is an arbitrary binary image, when filling a pixel having a hole in the image x , calculation is performed according to Expression 13. Here, f is a parameter indicating the size of the hole to be filled, and generally f = 1 is sufficient. Note that a diagonal line cannot be detected due to its nature in the vicinity of 4; therefore, it is better to set it as close to 8 as possible.

[0057]

(Equation 13)

Assuming that the image x is an arbitrary binary image, when an isolated point or an isolated hole is to be deleted from the image x , the calculation is performed in accordance with Expression 14. Note that a diagonal line cannot be detected due to its nature in the vicinity of 4; therefore, it is better to set it as close to 8 as possible.

[0059]

[Equation 14]

Assuming that the image x is an arbitrary binary image, in order to detect a pixel having a line width of 1 in the image x , calculation is performed according to equation 15 using four neighboring pixels.

[0061]

(Equation 15)

[0062] Two image x, y is any binary image. When the image y is assumed to be an image obtained by detecting the pixel out line width is 1 image x, the inner line width of the image x is 1 In order to extend the line width of a certain pixel, Equation 1 is calculated using four neighboring pixels.
Calculate according to 6.

[0063]

(Equation 16)

Therefore, if the line width detection of Expression 15 and the line width expansion of Expression 16 are used, complementation of the line width of the binary image can be easily described according to Expression 17.

[0065]

[Equation 17]

Next, a function and an operator relating to the neighborhood processing for each band pixel value of an image will be described below.

If there are two images x and y , the maximum value image of these images is calculated according to equation (18).

[0068]

(Equation 18)

If there are two images x and y , the difference between these images is calculated according to equation (19).

[0070]

[Equation 19]

Here, the Laplacian of Expression 11 and Expression 19
, The sharpening of the image can be simply described according to equation (20).

[0072]

(Equation 20)

If there are two images x and y , and the image y is a single-band binary image, each band pixel value of the image x can be masked using the band pixel value of the image y according to Equation 21. .

[0074]

(Equation 21)

[0075] Two image x, there is y, if the image x and y are binary images, in accordance with Equation 22, the image based on the image x
y can be shaped.

[0076]

(Equation 22)

[0077] Two image x, there is y, if the image y is binary image in accordance with Equation 23, a band-pixel value of the image x not specified by the image y, in the vicinity of the band-pixel value of the image x The interpolation is performed using the average value of the band pixel values of the image x specified by the image y . However, int (v) means that the real number v is truncated below the decimal point.

[0078]

(Equation 23)

In the present invention, the processing is simplified by treating the position and the movement amount of the pixel like image data. This is called position imaging. In the following, some functions and operators relating to imaging will be described.

First, the operator for converting each value of l, m, and o at the position p (l, m, o) into band pixel values as image data is #, and the converted band pixel values are #p (l, m). ,
o). Next, consider a case where the band pixel value moves from the position p (i, j, k) to the position p (i + 1, j + m, k + o). At this time, the shift amount of the band pixel value is the position p (l,
m, o). That is, the movement amount can be regarded as a vector from a certain position. Finally, the operator that extracts the position from the band pixel value is # ^-1 . Therefore, # ^-1 #p (l, m, o) = p (l,
m, o).

Therefore, according to Equation 24, the movement amount p (i,
j, k) in a plane represented by the width direction and the height direction by 180
Can be turned in the opposite direction.

[0082]

(Equation 24)

[0083] There are image x, when the image x is a single band binary image, the moving amount of the center of gravity position in the position of the image x p (i, j, 1 ) is calculated in accordance with Equation 25. Note that division must be performed when calculating the center of gravity, but the division is canceled when calculating the amount of movement into the vicinity of 8, so division is omitted in Equation 25.

[0084]

(Equation 25)

From the amount of movement p (i, j, k), the amount of movement into the vicinity of 8 can be calculated according to equations 26 and 27, and can be converted into an image of the amount of movement. Expression 27 is used only when expression 26 cannot cope with the discretization of the image.

[0086]

(Equation 26)

[0087]

[Equation 27]

Therefore, using Equations 25, 26 and 27, the band pixel value of the moving amount image of the single-band binary image x in the direction of the center of gravity can be simply described according to Equations 28 and 29. Note that the number of bands of the moving amount image is one.

[0089]

[Equation 28]

[0090]

(Equation 29)

On the other hand, since the position opposite to the position of the center of gravity can be obtained by using Expression 24, the band pixel value of the moving amount image in the direction opposite to the center of gravity of the single-band binary image x is simply described according to Expression 30. can do. Note that the number of bands of the moving amount image is one.

[0092]

[Equation 30]

When there are two images x and y , and the image y is a moving amount image, after moving the band pixel value of the image x to the moving position indicated by the image y according to Equation 31, the same band pixel The sum of the band pixel values that have been moved to the grayscale image can be obtained.

[0094]

(Equation 31)

Then, by using equations (4), (28), (29) and (31), according to equation (32) or (33), after moving the single-band grayscale image x in the vicinity of the center of gravity, the sum of the band pixel values moved to the same band pixel Can be easily described.

[0096]

(Equation 32)

[0097]

[Equation 33]

If there are two images x and y , and the image x is a binary image and the image y is a moving amount image, the position of the moving destination of each band pixel value of the image x can be obtained. Band pixel values with overlapping destinations can be found. Therefore, the destination of each band pixel value of the image x does not overlap,
In addition, the band pixel value of the movable image indicating that there is a moving band pixel value is generated according to Expression 34.
Note that the number of bands of the movable image is one.

[0099]

(Equation 34)

When there are three images x , y , and w , the image y is a movable image, and the image w is a movement amount image, the band pixel value of the image x can be moved according to Expression 35.

[0101]

(Equation 35)

Therefore, using Equations 30, 34 and 35, the band pixels of the image x obtained by moving the band pixels of the image x in the direction opposite to the position of the center of gravity calculated from the binary image y according to Equation 36 Value can be described easily.

[0103]

[Equation 36]

Therefore, by using Equations 1 to 36, the image storage means 1 shown in FIGS.
2. All of the data processing device 110 that implements the image vibration means 13, the edge information generation means 14, the edge information formation means 15, the position / size detection means 17, the area normalization means 27, and the normalized image holding means 28 Array operation unit 10
0 algorithm can be described. Below,
Arbitrary array operation unit 10 in data processing device 110
Using an algorithm of 0, the image storage means 12, the image vibration means 13, the edge information generation means 14, the edge information formation means 15, the position / size detection means 17, the area normalization means 27, and the normalized image holding means 28 Will be described.

Implemented by data processing device 110
The image storage means 12 stores the digital image 111
In addition, the array operation units 100 arranged in a lattice
And work in parallel. Array arranged at i row and j column on the grid
Operation unit 100 is AOU _ijThen, AOU_ij
Is as shown in FIG.

At step 1201, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

At step 1202, the neighborhood of AOU _ij and initial values of variables are set.

In step 1203, it is determined whether or not the digital image 111 to be sequentially input has disappeared. If there is no digital image 111 (step 1203: YE
S), end the algorithm. If digital image 11
If there is 1 (step 1203: NO), step 12
Move to 04. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.

At step 1204, the digital image 111
Wait for input until is ready.

At step 1205, the digital image 111
Of pixels in the i-th row and the j-th column are input for the number of bands. For this reason, AOU
_ij requires the memory 102 to store at least image data for the number of bands.

In step 1206, the pixels in the i-th row and the j-th column of the digital image 111 are stored so as to be able to output while waiting for input.

At step 1207, the digital image 111
Is output. Thereafter, the flow returns to step 1203.

As a result, the image storage means 12 can store the digital image 111 using the data processing device 110 composed of the array operation unit 100.

In order for the image vibration means 13 realized by the data processing device 110 to vibrate the digital image 111, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. If the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AOU _ij , AOU _ij
The algorithm of _ij is as shown in FIG.

In step 1301, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

In step 1302, the neighborhood of AOU _ij and initial values of variables are set.

At step 1303, it is determined whether or not the digital image 111 to be sequentially inputted has disappeared. If there is no digital image 111 (step 1303: YE
S), end the algorithm. If digital image 11
If there is 1 (step 1303: NO), step 13
Move to 04. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.

At step 1304, the digital image 111
Of pixels in the i-th row and the j-th column are input for the number of bands. For this reason, AOU
_ij requires the memory 102 to store at least image data for the number of bands.

In step 1305, the function Ξ _ijk ( x )
, The pixel in the i-th row and the j-th column of the digital image 111 is moved to one of the neighboring pixels.

At step 1306, the digital image 111
Is output. Thereafter, the flow returns to step 1303.

Thus, the image vibrating means 13 can vibrate the digital image 111 using the data processing device 110 including the array operation unit 100.

In order for the edge information generating means 14 realized by the data processing device 110 to generate the coarse edge information image 113 from the digital image 111, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. . Array operation unit 10 arranged at i row and j column on the lattice
If 0 is AOU _ij , the algorithm of AOU _ij for the edge information generating means 14 is as shown in FIG.

At step 1401, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

In step 1402, the neighborhood of AOU _ij and initial values of variables are set. In setting the neighborhood, the neighborhood size q used in each of the functions may be individually determined to be 4 or 8, or the whole may be unified to 4 or 8. In order to increase the accuracy of the rough edge information 112 generated by the edge information generating means 14 of the present invention, it is desirable to set all the neighborhood sizes q to 8. However, the edge information generating means 14 can cope with this by appropriately changing the neighborhood size as needed, depending on the calculation time for generating the rough edge information 112, the number of bands of the digital image 111, and the like.

At step 1403, the digital image 111
Judge whether or not is completed. If the digital image 111
If there is no (step 1403: YES), the algorithm ends. If there is a digital image 111 (step 1403: NO), the algorithm ends. However, when the array operation unit 100 is implemented for a specific number of bands and an image size, an infinite loop may be used.

At step 1404, the digital image 111
Of pixels in the i-th row and the j-th column are input for the number of bands. This is AOU
_ij is to collectively process the pixels in the i-th row and the j-th column of the digital image 111. For this reason, AOU _ij requires a memory 102 for storing at least image data for the number of bands.

In step 1405, AOU _ij communicates with the array operation unit 100 in the vicinity, so that a function S is applied to each band pixel value of the input digital image 111.
Perform smoothing according to _ijk ( x ). The smoothed band pixel values are treated as band pixel values of the smoothed image. Here, the function S _ijk ( x ) may be repeated several times as needed. In the case of a general color image, two times is sufficient.

In step 1406, logarithmic conversion is performed on each band pixel value of the smoothed image in accordance with the function _Lijk ( x ). The logarithmically converted band pixel value is treated as a band pixel value of the logarithmically converted image.

In step 1407, AOU _ij communicates with the neighboring array operation unit 100 to _sharpen each band pixel value of the logarithmically converted image in accordance with the function E _ijk ( x ). The sharpened band pixel value is treated as a band pixel value of the sharpened image.

In step 1408, each band pixel value of the one-input pre-sharpened image is subtracted from each band pixel value of the sharpened image according to the function D _ijk ( x , y ). The band pixel value for which the difference has been calculated is treated as a band pixel value of the time difference image.

In step 1409, each band pixel value of the sharpened image before one input is replaced with a corresponding band pixel value of the sharpened image.

At step 1410, the AOU _ij communicates with the neighboring array operation unit 100, and the operator ∇ ² _ijk x is applied to each band pixel value of the time difference image.
Laplacian is calculated according to the following equation. The band pixel value for which the Laplacian has been calculated is treated as the band pixel value of the time difference Laplacian image.

At step 1411, AOU _ij communicates with the neighboring array operation unit 100 to obtain a function Z for each band pixel value of the time difference Laplacian image.
_The zero point is extracted according to _ijk ( x ). The band pixel value from which the zero point has been extracted is treated as the band pixel value of the time difference zero point image.

In step 1412, the maximum value among the band pixel values of each band pixel value of the time difference Laplacian image is detected according to the function B _ij1 ( x ). The detected maximum value band pixel value is treated as a band pixel value of the maximum value time difference zero point image. Note that the number of bands is 1 for convenience.

In step 1413, AOU _ij communicates with the neighboring array operation unit 100 to calculate Laplacian for each band pixel value of the sharpened image according to the operator ∇ ² _ijk x . The band pixel value for which the Laplacian has been calculated is treated as the band pixel value of the Laplacian image.

In step 1414, AOU _ij communicates with the neighboring array operation unit 100, and the function Z _ijk ( x ) is applied to each band pixel value of the Laplacian image.
The zero point is extracted according to. The band pixel value from which the zero point is extracted is treated as the band pixel value of the zero point image.

In step 1415, the maximum value among the band pixel values of each band pixel of the Laplacian image is detected according to the function B _ij1 ( x ). The detected maximum band pixel value is treated as a band pixel value of the maximum zero point image.
Note that the number of bands is 1 for convenience.

In step 1416, the maximum value of the band pixel values at the same position in each image is calculated for each band pixel value of the Laplacian image and each band pixel value of the time difference Laplacian image according to the function M _ijk ( x , y ). Find the value. The detected maximum band pixel value is treated as a band pixel value of the hybrid zero point image. Note that the number of bands is 1 for convenience.

In step 1417, AOU _ij communicates with the neighboring array operation unit 100 to remove holes according to the function F _ijk ( x ) for the band pixel value of the hybrid zero point image. The band pixel value from which the hole has been removed is treated as the band pixel value of the hole-removed hybrid zero-point image. Note that the number of bands is 1 for convenience. Here, the function F _ijk ( x ) may be repeated several times as necessary. In the case of a general color image, one time is sufficient.

At step 1418, AOU _ij communicates with the neighboring array operation unit 100, and the AOU _ij performs a function A on the band pixel value of the hole-removed hybrid zero-point image
Isolated points and isolated holes are removed according to _ijk ( x ). The band pixel value from which the isolated points and the isolated holes have been removed is treated as the band pixel value of the noise-removed hybrid zero-point image. Note that the number of bands is 1 for convenience.

At step 1419, noise removal hybrid zero
Function I for the band pixel value of the point image _ijk(x)in accordance with
Invert 0 and 1. Inverted band pixel value is coarse edge
It is treated as a band pixel value of the information image 113.

At step 1420, coarse edge information image 1
13 band pixel values are output. Then step 1403
Return to

Thus, the edge information generating means 14 can generate the rough edge information image 113 from the digital image 111 by using the data processing device 110 including the array operation unit 100.

As shown in FIG. 10, the data processing device 11
0, a coarse edge information image 113 composed of coarse edge information 112;
In order to generate the formed edge information image 115 composed of the formed edge information 114 from the digital image 111 and the digital image 111, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. If the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AOU _ij , AOU _ij
Is as shown in FIG.

At step 1501, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

In step 1502, the neighborhood of AOU _ij and the initial values of variables are set. In setting the neighborhood, the neighborhood size q used in each of the functions may be individually determined to be 4 or 8, or the whole may be unified to 4 or 8. In order to increase the accuracy of the formed edge information 114 formed by the edge information forming means 15 of the present invention, it is desirable to set all the neighborhood sizes q to 8. However, the edge information forming means 15 can cope with the situation by appropriately changing the neighborhood size as necessary, due to restrictions on the calculation time for forming the rough edge information 112, the number of bands of the input digital image 111, and the like. .

At step 1503, it is determined whether or not the digital image 111 or the coarse edge information image 113 which is sequentially inputted has disappeared. If there is no digital image 111 or coarse edge information image 113 (step 1503: YES), the algorithm ends. If there is either the digital image 111 or the rough edge information image 113 (step 1503: NO), the process proceeds to step 1504. However, when the array operation unit 100 is implemented for a specific number of bands and an image size, an infinite loop may be used.

At step 1504, the digital image 111
Then, the pixels of the i-th row and the j-th column of the rough edge information image 113 are inputted for the number of bands. This is because AOU _ij is a digital image 111
This is because the pixels in the i-th row and the j-th column of the rough edge information image 113 are collectively processed. For this reason, AOU _ij requires a memory 102 for storing at least image data for the number of bands.

At step 1505, the digital image 111
And the pixel at the i-th row and the j-th column of the coarse edge information image 113 are separated. This is because AOU _ij is the pixel of the i-th row and the j-th column of the digital image 111 and the i of the coarse edge information image 113.
This is because the pixels in the row j column are processed as pixels of an independent image. If the pixel at the i-th row and the j-th column of the digital image 111 and the pixel at the i-th row and the j-th column of the rough edge information image 113 are separated and input from the beginning, nothing is performed.

In step 1506, the AOU _ij communicates with the neighboring array operation unit 100 to obtain a function S for each band pixel value of the input digital image 111.
Perform smoothing according to _ijk ( x ). The smoothed band pixel values are treated as band pixel values of the smoothed image. Here, the function S _ijk ( x ) may be repeated several times as needed. In the case of a general color image, two times is sufficient.

In step 1507, logarithmic conversion is performed on each band pixel of the smoothed image in accordance with the function _Lijk ( x ). The logarithmically converted band pixel value is treated as a band pixel value of the logarithmically converted image.

In step 1508, AOU _ij communicates with the neighboring array operation unit 100, thereby sharpening each band pixel value of the logarithmically converted image according to the function E _ijk ( x ). The sharpened band pixel value is treated as a band pixel value of the sharpened image.

In step 1509, AOU _ij communicates with the neighboring array operation unit 100 to calculate Laplacian for each band pixel value of the sharpened image in accordance with the operator ∇ ² _ijk x . The band pixel value for which the Laplacian has been calculated is treated as the band pixel value of the Laplacian image.

In step 1510, the AOU _ij communicates with the neighboring array operation unit 100, and the function Z _ijk ( x ) is applied to each band pixel value of the Laplacian image.
The zero point is extracted according to. The band pixel value from which the zero point is extracted is treated as the band pixel value of the zero point image.

In step 1511, the maximum value among the band pixel values is detected according to the function B _ij1 ( x ) for each band pixel value of the zero point image. The detected maximum band pixel value is treated as a band pixel value of the maximum zero point image. Note that the number of bands is 1 for convenience.

In step 1512, 0 and 1 are inverted with respect to the band pixel value of the maximum-value zero-point image according to the function I _ijk ( x ). The inverted band pixel value is treated as a band pixel value of the basic edge information image.

In step 1513, the input band pixel value of the coarse edge information image 113 is first treated as the band pixel value of the shaped coarse edge information image, and the AOU _ij communicates with the neighboring array operation unit 100 to form the basic edge image. Using the band pixel value of the information image, the band pixel value of the shaped rough edge information image is shaped according to the function Q _ijk ( x , y ). The shaped band pixel value is treated again as a band pixel value of the shaped rough edge information image. Where the function Q _ijk
( X , y ) is repeated until the band pixel value of the shaped rough edge information image no longer changes. However, the shaping process may be terminated at an appropriate number of repetitions depending on the calculation time constraints, the quality of the input rough edge information image 113, the quality required for the formed edge information image 115 to be formed, and the like.

At step 1514, AOU _ij communicates with the neighboring array operation unit 100, and thereby a function C is applied to the band pixel value of the shaped coarse edge information image.
Perform line width complementation according to _ijk ( x ). The complemented band pixel values are treated as band pixel values of the formed edge information image 115.

At step 1515, the band pixel value of the formed edge information image 115 is output. Then step 150
Return to 3.

Thus, the edge information forming means 15 can form the rough edge information image 113 on the formed edge information image 115 using the data processing device 110 including the array operation unit 100.

As shown in FIG. 12, the data processing device 11
0, the formed edge information image 11 in which the position / size detecting means 17 realized by 0 uses the formed edge information 114 as pixels.
5 to the overlapping information image 132 having the overlapping information 131 as a pixel.
Is generated, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. When the lattice i and a row j array operation unit 100 at a column and _{AOU ij,} Algorithm of _{AOU ij} is shown in Figure 13.

At step 1701, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

In step 1702, the neighborhood of AOU _ij and the initial values of variables are set. In the setting of the neighborhood, the neighborhood size q used in each of the functions may be determined individually, or all may be unified. In order to increase the accuracy of the duplicated information image 132 generated by the data processing device 110 of the present invention, it is desirable to set all the neighborhood sizes q to large values. However, the position / size detection unit 17 appropriately changes the neighborhood size as necessary depending on the calculation time for calculating the center of gravity of the formed edge information 114 of the object, the size of the formed formed edge information image 115, and the like. That can be dealt with.

At step 1703, it is determined whether or not there is no formed edge information image 115 inputted sequentially.
If there is no formed edge information image 115 (step 1
703: YES), the algorithm ends. If there is a formed edge information image 115 (step 1703: N
O), proceed to step 1704. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.

In step 1704, the pixels in the i-th row and the j-th column of the formed edge information image 115 are inputted for one band. Therefore, AOU _ij requires a memory 102 for storing at least one band of image data.

At step 1705, the formed edge information 114 of the formed edge information image 115 is converted into the overlap information 131 of the overlap information image 132. The overlap information 131 is a band pixel value corresponding to 1 or 0.

In step 1706, AOU _ij communicates with the array operation unit 100 in the vicinity so that the function Δ
_The movement amount is calculated according to _ij1 ( x ). The band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.

At step 1707, AOU _ij communicates with the array operation unit 100 in the vicinity, so that a function に 対 し て is applied to each band pixel value of the overlap information image 132.
_ij1 ( x ). The shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.

At step 1708, it is determined whether or not the number of movements representing the number of repetitions from step 1705 to step 1707 has reached the designated number. If the number of times of movement has not reached the specified number of times (step 1708: N
O), returning to step 1705; If the number of times of movement has reached the specified number of times (step 1708: YES), the process moves to step 1709. Note that the designated number is determined by the size of the formed edge information image 115, the size of the object represented by the formed edge information 114, and the size q of the neighborhood. If appropriate parameters are set according to the purpose of use, it does not matter if the designated number is overestimated, but if the designated number is too large, the time required to detect the position and size increases.

At step 1709, AOU _ij communicates with the neighboring array operation unit 100 to obtain a function Δ ′ for each band pixel value of the overlap information image 132.
_The movement amount is calculated according to _ij1 ( x ). The band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.

At step 1710, AOU _ij communicates with the array operation unit 100 in the vicinity to obtain a function Λ ′ for each band pixel value of the overlapping information image 132.
_ij1 ( x ). The shifted band pixel value is newly treated as a band pixel value of the overlapping information image 132.

At step 1711, the duplicate information image 132
Is output. Thereafter, the flow returns to step 1703.

Note that each of the duplication information 13 of the duplication information image 132
Reference numeral 1 denotes formed edge information 1 around the position.
Since it represents the total number of 14, it means the size of the object centered on its position as a result.

Thus, the position / size detecting means 17 can generate the overlapping information image 132 from the formed edge information image 115 by using the data processing device 110 including the array operation unit 100.

As shown in FIG. 14, the data processing device 11
0, the region normalizing means 27 realizes the object region 1
In order to generate a normalized image 145 including a normalized region 144 from an object region image 142 including the object region 41 and a digital image 111 including a separated object region 143 overlapping the object region 141, the array operation units 100 arranged in a grid pattern Work synchronously and in parallel. If the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AOU _ij , AOU _ij
The algorithm of _ij is as shown in FIG.

At step 2701, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

At step 2702, the neighborhood of AOU _ij and initial values of variables are set. In the setting of the neighborhood, the neighborhood size q used in each of the functions may be determined individually, or all may be unified. In order to increase the accuracy of the normalized image 145 generated by the area normalizing means 27 of the present invention, it is desirable to set all the neighborhood sizes q to large values. However, depending on the restriction of the calculation time for normalizing the separated object region 143, the size of the input digital image 111, and the like, the region normalizing means 27 can cope by changing the neighborhood size as needed. .

At step 2703, it is determined whether or not the sequentially input object region image 142 or digital image 111 has disappeared. If there is no object area image 142 or digital image 111 (step 2703: YES),
End the algorithm. If there is the object area image 142 or the digital image 111 (step 2703: N
O), and proceed to step 2704. However, for only a specific number of bands and image size, the array operation unit 100
May be implemented as an infinite loop.

At step 2704, the object area image 142
Of the i-th row and j-th column for one band, and the pixels of the i-th row and j-th column of the digital image 111 for the number of bands. This is AOU
_ij is to collectively process the pixels in the i-th row and the j-th column of the object area image 142 and the pixels in the i-th row and the j-th column of the digital image 111. For this reason, AOU _ij requires a memory 102 for storing at least image data for the total number of bands.

At step 2705, the object area image 142
And the pixel at the i-th row and the j-th column of the digital image 111 are separated. This is because AOU _ij is the object area image 14
This is because the pixels in the i-th row and the j-th column of the digital image 111 and the pixels in the i-th row and the j-th column of the digital image 111 are processed as independent image pixels. If the pixel of the i-th row and the j-th column of the object area image 142 and the pixel of the i-th row and the j-th column of the digital image 111 are separated and input from the beginning, nothing is performed. Object area image 142
And the digital image 111 are copied to the updated object area image and the updated digital image, respectively.

In step 2706, the AOU _ij communicates with the neighboring array operation unit 100, and the function R _ij1 ( x ) is _applied to each band pixel value of the updated object region image.
Is calculated according to the following equation. The band pixel value obtained by imaging the movement amount is treated as a band pixel value of the movement amount image.

In step 2707, the AOU _ij communicates with the neighboring array operation unit 100, and the function H _ijk ( x ,
y ), a destination band pixel value that can be moved can be found. The value indicating whether the destination is a movable destination is treated as a band pixel value of the movable image.

In step 2708, the AOU _ij communicates with the neighboring array operation unit 100 to obtain a function U _ijk ( x , x) for each band pixel value of the updated object region image.
Move to a movable destination according to y ). The shifted band pixel value is newly treated as a band pixel value of the updated object region image.

At step 2709, the AOU _ij communicates with the neighboring array operation unit 100, and the function U _ijk ( x , x) is obtained for each band pixel value of the updated digital image.
Move to a movable destination according to y ). The shifted band pixel value is newly treated as a band pixel value of the updated digital image.

At step 2710, it is determined whether or not the number of movements representing the number of repetitions from step 2706 to step 2709 has reached the designated number. If the number of times of movement has not reached the specified number of times (step 2710: N
O), returning to step 2706; If the number of times of movement has reached the specified number of times (step 2710: YES), the process moves to step 2711. The designated number is determined by the size of the digital image 111, the size of the separated object region 143 of the digital image 111, and the size q of the neighborhood. If appropriate parameters are set according to the purpose of use, it does not matter if the specified number is overestimated, but if the specified number is too large, the time required for normalization becomes longer.

[0186] In step 2711, _{by, AOU ij} to communicate with array operation unit 100 in the vicinity of the function _{V i} jk for each band-pixel value of the updated object-area image completing movement (x, y) in accordance with the proximity Interpolate with the average value. Note that both x and y are updated object region images. The band pixel value filled with the average value is treated as a band pixel value of the normalized updated object region image.

In step 2712, the AOU _ij communicates with the neighboring array operation unit 100 to obtain the average of the neighborhood according to the function V _{i jk} ( x , y ) for each band pixel value of the updated updated digital image. Fill with values. Note that
x is the updated digital image, and y is the updated object area image. The band pixel value filled with the average value is treated as a band pixel value of the normalized updated digital image.

At step 2713, it is determined whether or not the number of interpolations representing the number of repetitions from step 2711 to step 2712 has reached the designated number. If the number of interpolations has not reached the designated number (step 2713: N
O), and return to step 2711. If the number of times of interpolation has reached the designated number of times (step 2713: YES), the process moves to step 2714. Generally, the number of times of interpolation is about half of the neighborhood size q is sufficient.

At step 2714, it is determined whether or not the number of continuations representing the number of repetitions from step 2706 to step 2713 has reached the designated number. If the continuation number has not reached the specified number (step 2714: N
O), returning to step 2706; If the continuation number has reached the specified number (step 2714: YES), the process moves to step 2715. The designated number is determined by the size of the digital image 111, the size of the separated object region 143 of the digital image 111, and the size q of the neighborhood. If appropriate parameters are set according to the purpose of use, it does not matter if the specified number is overestimated, but if the specified number is too large, the time required for normalization becomes longer.

At step 2715, the band pixel value of the updated digital image is output as the band pixel value of the normalized image 145. Thereafter, the flow returns to step 2703.

As a result, using the data processing device 110 composed of the array operation unit 100, the area normalizing means 27 converts the object area image 142 and the digital image 111
Can generate a normalized image 145.

Since the normalized image holding means 28 realized by the data processing device 110 stores the normalized image 145, the array operation units 100 arranged in a lattice form operate synchronously and in parallel. If the array operation unit 100 arranged at the i-th row and the j-th column on the lattice is AOU _ij , AOU _ij
The algorithm of _ij is as shown in FIG.

At step 2801, AOU _ij is arranged at the i-th row and the j-th column on the lattice. This is necessary to determine the neighborhood of AOU _ij , whether logical or physical.

At step 2802, the neighborhood of AOU _ij and initial values of variables are set.

At step 2803, it is determined whether or not the normalized image 145 sequentially inputted has disappeared. If there is no normalized image 145 (step 2803: YE
S), end the algorithm. If the normalized image 145
If there is (Step 2803: NO), Step 280
Move to 4. However, when the array operation unit 100 is implemented only for a specific image size, an infinite loop may be used.

At step 2804, the pixels of the i-th row and the j-th column of the normalized image 145 are inputted for the number of bands. For this reason, AOU
_ij requires the memory 102 to store at least image data for the number of bands.

At step 2805, the format of the normalized image 145 is converted if the output destination device requires it.
In particular, when the number of bands of the normalized image 145 is set to 1 or the number of bands of the digital image 111 is 4 or more, the normalized image 145
This is convenient when the number of bands is set to 3 to make it easier to generate an analog signal. Otherwise do nothing.

In step 2806, the pixels in the i-th row and the j-th column of the normalized image 145 are stored so that the image data can be reliably transmitted to the output device having a different processing speed.

At step 2807, the band pixel value of the normalized image 145 is output. Thereafter, the flow returns to step 2803.

As a result, the normalized image holding means 28 can output the normalized image 145 using the data processing device 110 including the array operation unit 100.

Up to this point, a method of performing image processing including only neighborhood processing using the data processing device 110 including the array operation unit 100 has been described. Hereinafter, the object / background separation unit 16 will be described using only the neighborhood processing using the data processing device 110 including the array operation unit 100.

First, a nonlinear oscillator generally causes a pull-in phenomenon. This pull-in phenomenon is a phenomenon in which nonlinear oscillators having different periods interact with each other and oscillate at a period of a simple constant ratio in a periodic behavior such as a limit cycle or an attractor. At this time, when the vibration of one nonlinear vibrator is changed, the vibrations of the other nonlinear vibrators also change, so that these nonlinear vibrators are synchronized. In addition, by adjusting the interaction of the nonlinear oscillators, the phase difference between the vibrations can be made as small or large as possible. By manipulating this interaction, a group of nonlinear oscillators can be divided into a plurality of groups having different phases. Object / background separation means 1
6 generates an object area image 142 representing the object area 141 by utilizing the pull-in phenomenon of the nonlinear oscillator to separate the object and the background so that the edge information in the edge information image is used as a boundary. Here, a case where Van der Pol is used as the nonlinear oscillator will be described as an example.

First, in a non-linear oscillator network composed of non-linear oscillators arranged in a lattice, i-th row j
When a nonlinear oscillator in the column and omega _ij, a set Omega _ij of nonlinear oscillators in a q near the nonlinear oscillator ω _{i j (q)}
Is represented by Expression 37. Where q is 4, 8, 2
It is a sequence of numbers following 4, 48, 80, 120, (2r + 1) ² -1, and r is a natural number. When a nonlinear oscillator that exceeds the network size is included in the neighborhood set Ω _ij (q), the nonlinear oscillator ω _ij is used instead. Thereby, the edge processing is automatically performed. Thus the number of elements of the neighbor set Ω _{i j (q)} is always q.
As can be seen from this, the nonlinear oscillator network is treated the same as a single-band image. For simplicity of expression, in the nonlinear oscillator network, only two subscripts are used: the width direction and the height direction.

[0204]

(37)

[0205] Next, the nonlinear oscillator is coupled by a coupling value tau _ijkl which are calculated in accordance with Equation 38 with the nonlinear oscillators in a neighbor set Ω _{ij (q a)} contained in the vicinity of _{q a.} When the logarithmic table is not used, approximation by Expression 39 is also possible. Μ and ν are appropriate positive constants.

[0206]

(38)

[0207]

[Equation 39]

If all the nonlinear oscillators of the nonlinear oscillator network are completely in phase and synchronized, the processor 10
As long as the calculation is performed by 1, the nonlinear oscillator ω _ij continues to operate forever with the same phase. Therefore, if a disturbance ρ _ij is given, such a state can be avoided. Although a pseudo random number can be used as the disturbance, a simple expression such as Expression 40 is sufficient. Note that ζ _ij represents the presence / absence of edge information in row i and column j of the edge information image. If there is edge information, it is set to 1, otherwise 0. Κ is an appropriate positive constant.

[0209]

(Equation 40)

The nonlinear oscillator ω _ij has a neighborhood set Ω _ij (q
_a ) In order to synchronize with the nonlinear oscillator ω _kl , Equation 41
_Is calculated according to the following equation.

[0211]

[Equation 41]

The van der Pol nonlinear oscillator ω _ij
The two parameters φ _ij and ψ _ij that form
And 43. Note that γ and ε are appropriate positive constants.

[0213]

(Equation 42)

[0214]

[Equation 43]

In order to separate the nonlinear oscillator into the object region 141 and the background region, it is necessary to calculate the phase shift of all the nonlinear oscillators, but it is simply separated into the object region 141 and the background region. 、 _Ij
Is calculated based on whether or not is equal to or larger than the threshold value θ. An output λ that outputs a result obtained by separating the object area 141 from the background area
_ij is obtained by Expression 44. Θ is an appropriate positive constant.

[0216]

[Equation 44]

If the edge information is not enough to separate the object from the background, the edge information must be interpolated. For this purpose, it is necessary to find out how many nonlinear oscillators are out of phase in a set Ω _ij (q _b ) of nonlinear oscillators near q _b of the nonlinear oscillator ω _ij .
Therefore, the contour parameter η _ij is calculated by Expression 45.

[0218]

[Equation 45]

Based on the result, the boundary parameter ｉ _ij indicating the interpolation ratio of the edge information is calculated by the equation (46).
Note that α, β, η _min , and η _max are appropriate positive constants.

[0220]

[Equation 46]

Here, the case of van der Pol as a nonlinear oscillator has been described. In addition, a nonlinear oscillator that is stable in a limit cycle such as a brasserator,
Any non-linear oscillator that causes a pull-in phenomenon, such as a chaotic oscillator that generates a Lorenz attractor or a wrestler equation attractor, can be operated. In this case, the parameters φ _ij and ψ _ij may be replaced or added with the parameters of the respective nonlinear oscillators. In that case, it is only _necessary to add the neighborhood input sum σ _ij and the disturbance ρ _ij to appropriate parameters. However, in the case of a chaotic oscillator, no disturbance ρ _ij is required.

By using equations (37) to (46), all array operation units 100 of the data processing apparatus 110 on which the object / background separation means 16 can be mounted.
Algorithm can be described. Hereinafter, an arbitrary array operation unit 100 in the data processing device 110 will be described.
The object / background separation means 16 will be described by using the above algorithm.

[0223] As shown in FIG.
0, the object / background separation means 16 uses the triangle edge information 151
The array operation units 100 arranged in a lattice form operate in parallel and in parallel in order to separate the data into the outer region 153 of the triangle. When the lattice i and a row j array operation unit 100 at a column and _{AOU ij,} Algorithm of _{AOU ij} is shown in Figure 18.

At step 1601, AOU _ij is arranged at the i-th row and the j-th column on the lattice.

In step 1602, the neighbors ω _ij and ω _kl are connected by a joint value τ _ijkl based on equations (38) and (39).

In step 1603, appropriate initial values are set for the parameters φ _ij and ψ _ij of the nonlinear oscillator.

In step 1604, it is determined whether or not there is no formed edge information image 115 that is sequentially input.
If there is no formed edge information image 115 (step 1
604: YES), end the algorithm. If the formed edge information image 115 exists (step 1604: N
O), proceed to step 1605; However, the array operation unit 100 for only a specific number of bands and image size
May be implemented as an infinite loop.

At step 1605, the formed edge information 11
Input ζ _ij of 4.

At step 1606, the disturbance ρ is _calculated from ζ _ij of the formed edge information 114 input immediately before according to the equation (40).
_ij is calculated.

At step 1607, the neighborhood set Ω _ij (q
_a ) Array operation unit 1 with nonlinear oscillator ω _{kl in}
Ζ _kl , ξ _kl , ψ _kl are input from the AOU _{kl of} 00, and the total value σ _ij is calculated according to Equation 41.

In step 1608, the parameters φ _ij and ψ _ij of the nonlinear oscillator are calculated according to the equations (42) and (43). That is, the differential equations shown in these equations are expressed by Runge
Solve with the Kutta method.

In step 1609, the output λ _ij of the non-linear oscillator is calculated according to equation (44). Here, ｉ _ij ≧
If θ, λ _ij = 1, otherwise λ _ij =
Set to 0.

At step 1610, the neighborhood set Ω_ij(Q
_bNonlinear oscillator ω in)_klArray operation unit 1 with
00 AOU_klTo λ_klEnter the contour parameter
Η _ijIs calculated according to Equation 45.

At step 1611, the boundary parameter ξ
_ij is calculated according to Equation 46. That is, the differential equation represented by this equation is solved by the difference method or the Runge-Kutta method.

In step 1612, it is determined whether the number of separations indicating the number of repetitions from step 1606 to step 1611 has reached the designated number. If the number of separations has not reached the specified number of times (step 1612: N
O), returning to step 1606; If the number of separations has reached the specified number of times (step 1612: YES), the flow shifts to step 1613.

At step 1613, the object area image 142
The output λ _ij of the non-linear oscillator having the pixel value of the band is output. Thereafter, the flow returns to step 1604.

The following method can be used to determine the number of separations in step 1612. object/
In the background separating means 16, if the image size is constant, the separation ends in a certain fixed time in almost all formed edge information 114 regardless of the initial state of the nonlinear oscillator. The number of repetitions from 1606 to step 1611 may be obtained. This is because if the initial state of the nonlinear oscillator is within a certain range, there is not much difference in the time until the nonlinear oscillator is synchronized by the pull-in phenomenon.

It is a property of the nonlinear oscillator that it is possible to separate the inner region 152 of the triangle and the outer region 153 of the triangle using the edge information 151 of the triangle only by calculating the nonlinear oscillator in this way. This is because the pull-in phenomenon is used. That is, when two nonlinear oscillators are coupled with a positive coupling value, they tend to be in phase, and when coupled with a negative coupling value, the phase difference tends to be as large as possible. When this property is used, nonlinear oscillators that are not directly coupled to each other have the same phase by coupling the nonlinear oscillators arranged in a lattice with neighboring components with a positive coupling value. Further, when the nonlinear oscillators located at the pixels sandwiching the formed edge information 114 are connected with each other with a negative connection value, both sides of the edge information are shifted from each other in phase as much as possible. In this way, different sets of phases can be formed on the inside and outside of the triangular edge information 151 without connecting all the nonlinear oscillators. Therefore, the object / background separation means 16 separates the object into a triangular inner area 152 and a triangular outer area 153 as shown in FIG. At this time, the phase difference between the inner region 152 of the triangle and the outer region 153 of the triangle exceeds 90 degrees and is as small as possible 1
As it approaches 80 degrees, the triangle and the background area can be separated.

What is important here is that, in the present embodiment, every time the formed edge information 114 is obtained, the joint value is changed in a pseudo manner by the following method. First, as determined by Expressions 38 and 39, a coupling value for coupling the nonlinear oscillator ω _kl to the nonlinear oscillator ω _ij is τ _ijkl
(See step 1602). Formation edge information 11
Of the four, ζ _ij and ｋ _kl are both 1 when there is an edge and 0 when there is no edge. When ζ _ij and ζ _kl of the formed edge information 114 are input (see step 1605), AOU _kl to AOU
_ij , the formed edge information 114 @ _kl is transferred to AOU
_{In ij} , the combined value τ _ijkl (1-ζ _kl ) is calculated and used as a substitute for the combined value τ _ijkl (see step 1607). This substituted coupling value τ _ijkl (1-ζ _kl )
, The boundary parameter ｉ _ij acts as a scaling factor from 0 to 1 (see step 1607).

As shown in FIG. 19, formed edge information 114
Becomes the triangle edge information 154 in a broken line state, it is necessary to interpolate the broken line. First, when the system is operated using the edge information 154 of the dashed triangle (see step 1605), the phase difference between the inside and the outside of the edge information 154 of the dashed triangle exceeds about 90 degrees. The boundary between the inside and outside of the triangle is unclear. Therefore, each AOU _ij calculates the output λ _ij of the nonlinear oscillator (see step 1609). When the output λ _ij is 1, λ _kl among the nearby nonlinear oscillators
Is ω _kl , both the parameters ψ _ij and ψ _kl are greater than or equal to θ. That is, λ _ij and λ
_kl are approximately in phase, and the phase difference does not exceed 90 degrees at worst if θ is a positive value. If the maximum value of the phase difference is determined by the value of theta, lambda _{i j} and lambda _kl is gradually increased theta in a range both become 1, the phase difference approaches 0 degrees. Therefore, using λ _ij and λ _kl ,
A contour parameter η _ij representing the number of neighboring non-linear oscillators that are approximately in phase is calculated according to Equation 45 (see step 1610). Subsequently, if the contour parameter η _ij is approximately half of the entire neighborhood, the boundary parameter ξ _ij which is the magnification of the combined value is decreased according to the mathematical formula 46, otherwise, it is increased according to the mathematical formula 46 (see step 1611). . For example, in the case of the vicinity of 8, if it is between 3 and 5, the boundary parameter may be reduced in accordance with Equation 46. If this process is repeatedly performed, the edge information 15 of the triangle indicated by the broken line shown in FIG.
If 4 is given, it is separated into a dashed triangle inner region 155 and a dashed triangle outer region 156.

As shown in FIG. 20, when two triangles overlap each other, edge information 157 of the front triangle and edge information 158 of the rear triangle are obtained. At this time, the inside area 159 of the front triangle and the inside area 1 of the back triangle
The non-linear oscillators in the three regions 60 and the double triangular background region 161 are separated from each other by shifting the phase of each other. Also, as shown in FIG. 21, even if the two overlapping circular edge information 162 are broken lines, they are separated into three parts: a front circular inner area 163, a rear circular inner area 164, and a double circular background area 165. You.

As a result, the object / background separation means 16 separates the object area 141 from the background using the formed edge information 114 of the formed edge information image 115 as a boundary by using the data processing device 110 composed of the array operation unit 100. be able to.

The visual device 3 has been described so far. Naturally, these visual devices 3 can be implemented by a general-purpose computer. However, when a moving object is to be searched, it is necessary to execute each of the means at high speed depending on the moving speed of the moving object. . In particular, when the image size or the resolution of the frame image is increased, the image storage unit 12, the image vibration unit 13, the edge information generation unit 14, the edge information formation unit 15, the object / background separation unit 16, / Size detecting means 17, area normalizing means 27, and normalized image holding means 2
8, the amount of calculation increases in each of the width direction and the height direction in proportion to the image size or the resolution. Therefore, the visual device 3 may not be able to achieve the desired performance depending on the application.

Therefore, using digital technology, image storage means 12, image vibration means 13, edge information generation means 14, edge information formation means 15, object / background separation means 16, position / size detection means 17, area normalization means 27 and the normalized image holding means 28, the array operation units 100 are arranged in a grid pattern in the data processing device 110 as shown in FIG. Are wired so that they can communicate with each other only with the array operation unit 100. That is, the four neighborhoods are directly wired. As a result, as compared with the case where the eight neighborhoods are wired, it is possible to operate at the same high speed with a small number of electronic components and the amount of wiring, and to easily have expandability even when the neighborhood size is expanded in the future.

As shown in FIG. 22, the array operation unit 100 includes a processor (PROCESSOR) 101 for calculating a mathematical expression in image processing, and a memory for storing all parameters, constants, functions and operators used in the mathematical expression. MEMORY) 102 and nearby array operation unit 100
A controller (CONTROLER) 103 for communicating with the
Inputs a clock signal (CLOCK), and the processor 101
Controls communication by transmitting a read (READ) and a write (WRITE) to the memory 102 and the controller 103. The processor 101 has an address bus 5
Memory 102 according to the address (ADDRESS) specified in 1
And an arbitrary memory element and register of the controller 103 can be selected. The processor 101 is connected to the memory 102 and the controller 1 via the data bus 52.
03 and the address bus 51.
Can access the data (DATA) of an arbitrary memory element and register specified by. When the array operation unit 100 inputs a previous input data group (FRONT INPUT DATA SET) composed of one or more input pixels, the controller 103 causes the memory 102 to store the previous input data group. Further, the controller 103 transmits the calculation data in the memory 102 created by the function to the adjacent array operation unit 100, stores the calculation data received from the adjacent array operation unit 100 in the memory 102, and further, if necessary, Are transferred to the array operation unit 100 other than the input. Finally, the controller 103
Converts the image data of the output image to the result data (RESULT DAT
Output as A).

The reason why the controller 103 is mounted on each array operation unit 100 is that the processor 101 can operate while the array operation units 100 are communicating with each other, so that the processor 101 can perform the calculation even during the communication waiting time. The controller 103 performs the edge processing of the image, that is, the processing in the image, that is, the high-speed processing can be realized and the hardware does not need to be changed even if the number of the array operation units 100 in the vicinity is changed. Since the exception processing for the edge pixels can be automatically performed, the processor 10
This is because the program No. 1 does not need to perform the edge processing and is extremely simple.

For the processor 101 and the memory 102, general-purpose digital circuits can be used. Controller 1
The specific circuit diagram of 03 is as shown in FIG. An address buffer (ADDRESS BUFFER) 53 receives an address (ADDRESS) from the processor 101 via an address bus (ADDRESS BUS) 51, and receives an address decoder (ADDRE).
The SS register 54 selects each register and other electronic circuit blocks. Data buffer (DATA BUFFE)
The R) 55 receives data (DATA) from the processor 101 via the data bus (DATA BUS) 52 and communicates exclusively with the register selected by the address decoder 54 via the internal data bus 56. Read the communication direction (READ)
Specified by When data is transmitted from the controller 103 to the processor 101, the reading becomes active. From the processor 101 to the controller 1
When the data is sent toward 03, write (WRIT
E) becomes active. The address is in the flag register (F
When LAG REGISTER) 57 is specified, the data is stored in the flag register 57 and the flag decoder (FLAG DECODE
R) Decoded by 58, clock signal (CLOCK)
Is transmitted to the adjacent array operation unit 100 as a plurality of signals (SIGNALS). A plurality of signals are received by a flag encoder (FLAG ENCODER) 59, and after being analyzed, a status register (STATUS REGISTER) 60
And is returned to the source array operation unit 100 as a reception (RECEIVE). The reception is received by the flag encoder 59 of the transmission source of the plurality of signals, and as a result, completion of transmission of the plurality of signals is confirmed. When the status register 60 is selected by the address, the contents of the status register 60 are transmitted to the processor 101 as data via the data bus 52. One or more input images (IN
PUT IMAGE) one or more pre-input delivery (FRONT I)
NPUT SEND) received by the flag encoder 59, a front input data group (FRONT INPUT) including one or more input images.
DATA SET) is read into the front input data register (FRONT INPUT DATA REGISTER) 61 prepared for the required storage capacity. When the previous input data register 61 is selected by the address, the contents of the previous input data register 61 are transmitted to the processor 101 as data. Processor 1
When 01 completes the calculation, the result data register (RESULT DATA REGISTER) 62 is selected according to the address, and the result data register 62 reads the image data of the output image as the result data (RESULT DATA). At the same time,
The flag encoder 59 transmits a result transmission (RESULT SEND).

When the data necessary for calculation is obtained from the nearby array operation unit 100, the output data register (OUTPUT DATA REGISTER) 63 is selected as an address, and the data to be transmitted to the nearby array operation unit 100 is calculated. (CALCURATION DATA) into the output data register 63. Then, it is transmitted as calculation data to all adjacent array operation units 100. When a plurality of signals (SIGNALS) are received from the upper array operation unit 100, the calculation data is transferred to the upper input data register (UPPERINPUT D).
(ATA REGISTER) 64. Thereafter, when the upper input data register 64 is selected by the address, the contents of the upper input data register 64 are transmitted as calculation data. The same applies to the case where a plurality of signals are received from the lower, left, and right array operation units 100, and the lower input data register 65, the left input data register 66, and the right input data register 67 operate similarly.

Each block of various buffers, various registers, and the address decoder 54 is a general-purpose electronic circuit. Specifically, the flag decoder 58 and the flag encoder 59 have input / output signals as shown in FIGS. Type is OUTPUT DATA REGISTE
R) The type of content read into 63 is represented by 5 bits.
This number of bits is a value sufficient to distinguish all calculation data to be transmitted and received by the array operation unit 100. Count-X (COUNT-X) and Count-Y (COUNT-Y) each represent a 4-bit unsigned integer, and the array operation unit 1
Indicates the number of transfers during 00. When the array operation unit 100 transmits the calculation data, each count becomes 0. When the calculation data transmitted from the left and right array operation units 100 is transmitted again, 1 is added to the count -X of the flag encoder 59. When the calculation data transmitted from the upper and lower array operation units 100 is transmitted again, the value becomes a value obtained by adding 1 to the count-Y of the flag encoder 59. After the processor 101 designates in which direction the contents of the output data register 63 are to be transmitted, up, down, left and right, in the transmission flag (SEND FLAG) of the flag register 57, the central data of the address decoder 54 for designating the output data register 63 Coding (CENTRAL DECODING)
Is received by the flag decoder 58, the flag decoder 5
8 outputs the transmission (SEND) in accordance with the direction specified by the transmission flag. The delivery flag is represented by 4 bits, and the calculation data of the array operation unit 100 is stored in the four sides of the array operation unit 100.
When the processor 101 sets to 1111 when transmitting the calculation data transmitted from the right array operation unit 100 to the upper and lower left sides, the processor 101 sets to 1110 when transmitting the calculation data transmitted from the right side array operation unit 100, and when transferring from the left to the upper and lower right sides. Is set to 1101, and when transferring from the bottom to the top, 10
Set to 00 and 0100 to transfer from top to bottom
Set as As a result, the transfer can be efficiently performed without duplication, and the rules for determining the transfer direction are clear, so that the type, count-X and count-Y
, The flag encoder 59 can determine which type of calculation data has been transmitted from which array operation unit 100. At the same time that the calculation data is read into the result data register 62 as the result data, the flag decoder 58 receives the result decoding (RESULT DECODING) and sends the result transmission (RESULT SEN).
D) Send.

When the flag encoder 59 receives a delivery in any one of the four directions, the flag encoder 59 determines the type and count of the reception direction.
X and count-Y are received, and the contents of the status register 60 of that part are updated. At the same time as this update, the reception is set to 1 in the receiving direction and transmitted. The flag encoder 59 of the array operation unit 100 at the transmission source receives the data at the moment when the reception becomes 1, and updates the reception status (RECEIVE STATUS) of the status register 60. Thus, in each array operation unit 100, the processor 101 can determine which input data register stores valid calculation data only by checking the reception status of the status register 60. Therefore, for example, if the calculation data is read into the upper input data register 64, the processor 101 can read the data from the upper input data register 64 by designating the address, but at the same time, the upper decoder (UPPE) from the address decoder 54.
RDECODING) is transmitted to the flag encoder 59, the upper part of the reception status is returned to 0, and the reception directed upward is transmitted as 0. The same operation is performed on the lower left and right sides. When at least one of the flag encoders 59 receives the previous input delivery for the input image, the status register 60 sets the front input delivery status (FRONT INPUT SEND STATUS) for the input image corresponding to the received previous input delivery to 1.
When the processor 101 reads data from the previous input data register 61 for an input image, the address decoder 54 outputs the predecoding (FRONT) to the flag encoder 59.
T DECODING), and sets the pre-input delivery status corresponding to the received pre-input delivery to 0. The processor 101 reads the contents of the status register 60,
It can be determined whether the latest input image is stored in the previous input data register 61.

FIG. 26 shows an algorithm when the processor 101 transmits calculation data to the array operation units 100 on the four sides via the controller 103. FIG.
FIG. 9 shows a process based on a mixture of program control by the processor 101 and hardware logic by the flag decoder 58 and the flag encoder 59. FIG.
In step 71, the processor 101 reads the contents of the status register 60. In step 72, it is determined whether or not the reception status of the read contents is all zero. If NO, the process ends. YES
If so, the process proceeds to step 73. In step 73, the processor 101 determines the type, counter, and transmission direction of the data to be transmitted to the adjacent array operation unit 100, and writes the content to the flag register 57. In step 74, the processor 101 determines whether the adjacent array operation unit 10
The data to be transmitted to 0 is written to the output data register 63. In step 75, the contents of the output data register 63 are used as calculation data,
Send to In step 76, the transmission is set to 1 only in the direction specified by the transmission flag of the flag register 57 and transmitted. Thereby, one transmission algorithm of the processor 101 ends. The processor 101 starts this transmission algorithm each time data to be transmitted is updated in the memory 102.

FIG. 27 shows an algorithm when the controller 103 receives calculation data from the upper array operation unit 100. FIG. 27 shows processing by hardware logic by the flag decoder 58 and the flag encoder 59. Step 8 is compared with FIG.
At 1, the flag encoder 59 inputs a delivery. In step 82, the flag encoder 59 determines whether the delivery is 1 or not. If NO, the process ends. If YES, the process moves to step 83. In step 83, the upper input data register 64 reads the calculation data transmitted from the upper side. In step 84, the flag encoder 59 sets the reception status for the upper side of the status register 60 to 1 and simultaneously sets the reception to 1 and transmits it to the upper array operation unit 100. The same applies to the lower left and right sides.
Thus, one reception algorithm of the controller 103 ends. The controller 103 always monitors the delivery from the top, bottom, left and right array operation units 100, and starts this reception algorithm each time the delivery is received.

FIG. 28 shows an algorithm when the processor 101 receives data from the upper input data register 64.
Shown in FIG. 28 shows the program control by the processor 101, the flag decoder 58 and the flag encoder 59.
2 shows processing by mixing with hardware logic. In FIG. 28, in step 91, the processor 101 reads the contents of the status register 60. In step 92, it is determined whether or not the reception status for the upper side of the read contents is 1. If NO, the process ends. If YES, the process moves to step 93.
In step 93, the processor 101 reads data from the upper input data register 64. In step 94,
The flag encoder 59 sets the reception status for the upper side of the status register 60 to 0, and at the same time, sets the reception to 0 and transmits it to the array operation unit 100 on the upper side. The same applies to the lower left and right sides. Thus, the one-time reception algorithm of the processor 101 ends. Processor 101
Monitors the contents of the status register 60 at regular intervals,
The receiving algorithm is started each time the receiving status of one of the upper, lower, left and right is 1. Here, the case where the processor 101 constantly monitors the contents of the status register 60 at regular intervals has been described, but by adding an interrupt circuit to the controller 103, the processor 101 can also start this reception algorithm by interruption.

Although the array operation unit 100 has been described mainly on the assumption that one output image is generated from one or more input images, the array operation unit 100 can output calculation data in the middle of calculation depending on the application. The circuit needs to be changed. At this time, the program is simply changed so that the result delivery of the flag decoder 58 is increased by the number of calculation data to be output, and only the result delivery corresponding to the calculation data read into the result data register 62 is set to 1. good.

The present embodiment has been described above. However, the present invention is not limited to the above-described embodiment, and those skilled in the art can implement various aspects, and understand the technical idea of the present invention. It goes without saying that the configuration of the present invention can be appropriately modified without departing from the scope of the present invention, and such modifications also belong to the technical scope of the present invention.

[0256]

According to the first aspect of the present invention, in a video reproducing / editing machine, even if a video is not converted to a specific image format or a human does not add an index,
The moving image search device can easily search for the position of a frame image showing a moving target object. In addition, the present invention can easily search a moving image database for a position of a frame image showing a moving target object and a title of a moving image including the frame image without depending on an image format used for recording. can do. In particular, the present invention is very effective for an object such as a person's face, which can be easily distinguished from other objects, moves frequently, has a limited shooting direction such as front and side, and has a different shooting size. . Further, the user can use the separately developed personal identification software to generate the normalized image 145 and the separated object area 14 generated by the visual device 3.
Recognition of 3 makes it possible to easily search for the position of a frame image showing a specific person.

According to the second aspect of the present invention, the moving image search device can easily search for the position of the frame image showing the moving target object without using the area designation image. By preparing in advance a key image having a monotone background for a frequently searched target object, the user does not need to generate an area designation image every time the target object is searched. Further, by recording such a key image in the database, the user can search the moving image database from the key image.

According to the third aspect of the present invention, the moving image search apparatus can search not only a moving target object but also a stationary target object. Therefore, the present invention can also search for ornaments and buildings such as pots. In addition, by connecting still images into a moving image, the present invention can search for a still image.

[Brief description of the drawings]

FIG. 1 is a block diagram of a moving image search device including a computer system, a video player, and a visual device.

FIG. 2 is a block diagram of a visual device that searches for a moving object in a moving image using a key image and an area designation image.

FIG. 3 is a block diagram of a visual device that searches for a moving object in a moving image using a key image.

FIG. 4 is a block diagram of a visual device that searches for a moving object and a stationary object in a moving image using a key image and an area designation image.

FIG. 5 is a block diagram of a visual device that searches for a moving object and a stationary object in a moving image using a key image.

FIG. 6 is a block diagram of an array operation unit arranged in a lattice.

FIG. 7 is a flowchart illustrating an algorithm of an image storage unit according to the embodiment.

FIG. 8 is a flowchart illustrating an algorithm of an image vibration unit according to the present embodiment.

FIG. 9 is a flowchart illustrating an algorithm of an edge information generating unit according to the embodiment.

FIG. 10 is an explanatory diagram in a case where coarse edge information is formed into formed edge information using a digital image.

FIG. 11 is a flowchart illustrating an algorithm of an edge information forming unit according to the embodiment.

FIG. 12 is an explanatory diagram for detecting the position and size of an object in an edge information image.

FIG. 13 is a flowchart illustrating an algorithm of a position / size detection unit according to the present embodiment.

FIG. 14 is an explanatory diagram in a case where a separated object region of a digital image is normalized.

FIG. 15 is a flowchart illustrating an algorithm of a region normalizing unit of the present embodiment.

FIG. 16 is a flowchart illustrating an algorithm of a normalized image holding unit according to the present embodiment.

FIG. 17 is an explanatory diagram showing a state where edge information of a triangle is separated into an inner area and an outer area of the triangle;

FIG. 18 is a flowchart illustrating an algorithm of an object / background separation unit according to the present embodiment.

FIG. 19 is an explanatory diagram showing a state in which edge information of a triangle in a broken line state is separated into an inner region and an outer region of the broken line triangle;

FIG. 20 is an explanatory diagram showing a state where edge information obtained by overlapping two triangles is separated into two triangle regions and a background region.

FIG. 21 is an explanatory diagram showing a state in which edge information in a broken line state when two circular object areas are superimposed is separated into two circular areas and a background area.

FIG. 22 is a block diagram of the internal structure of the array operation unit.

FIG. 23 is a block diagram of a controller.

FIG. 24 is an explanatory diagram showing input / output signals of a flag decoder.

FIG. 25 is an explanatory diagram showing input / output signals of a flag encoder.

FIG. 26 is a flowchart illustrating an algorithm by which a processor transmits data to an adjacent array operation unit via a controller.

FIG. 27 is a flowchart illustrating an algorithm by which a controller receives data from an adjacent array operation unit.

FIG. 28 is a flowchart showing an algorithm by which a processor receives data from an upper input register.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Computer system 2 Video playback device 3 Visual device 11 Image acquisition means 12 Image storage means 13 Image vibration means 14 Edge information generation means 15 Edge information formation means 16 Object / background separation means 17 Position / size detection means 22 Position selection means 27 Area normalizing means 28 Normalized image holding means 40 Mask means 41 Pattern comparing means 42 Comparison result holding means 51 Address bus 52 Data bus 53 Address buffer 54 Address decoder 55 Data buffer 56 Internal data bus 57 Flag register 58 Flag decoder 59 Flag encoder Reference Signs List 60 Status register 61 Previous input data register 62 Result data register 63 Output data register 64 Upper input data register 65 Lower input data register 66 Left input data register 67 Input data register 100 Array operation unit 101 Processor 102 Memory 103 Controller 110 Data processing device 111 Digital image 112 Coarse edge information 113 Coarse edge information image 114 Formed edge information 115 Formed edge information image 131 Duplicate information 132 Duplicate information image 141 Object area 142 Object Area image 143 Separated object area 144 Normalized area 145 Normalized image 151 Triangular edge information 152 Triangular inner area 153 Triangular outer area 154 Broken triangle state edge information 155 Broken triangular inner area 156 Broken triangular outer area 157 Edge information of the front triangle 158 Edge information of the rear triangle 159 Inner area of the front triangle 160 Inner area of the rear triangle 161 Double triangle background area 162 It became circular edge information 163 front circular inner area 164 behind the circular inner region 165 background region of the double circle

Claims (3)

[Claims] 1. A moving image search device for searching a moving image for a frame image showing a target object, comprising: means for acquiring the frame image; means for sequentially storing the frame image as a digital image; Means for generating a coarse edge information image from an image; means for forming the coarse edge information image into a first formed edge information image using the digital image; and a first object from the first formed edge information image. Means for generating an area image; means for generating an overlapping information image from the first formed edge information image; means for selecting one moving object using the overlapping information image; and the first object area image Means for extracting only the moving object selected from among the selected moving objects by a mask process; Means for generating a normalized image; means for holding the first area normalized image; means for storing a key image specifying the target object; means for storing an area designation image; and using the key image. Means for forming the area designation image into a second formed edge information image, means for generating a second object area image from the second formed edge information image, the key image and the second object area Means for generating a second area normalized image from the image, means for comparing the first area normalized image and the second area normalized image, and means for holding a comparison result, A moving image search device characterized by the following. 2. A moving image search device for searching a moving image for a frame image showing a target object, comprising: means for acquiring the frame image; means for sequentially storing the frame image as a digital image; Means for generating a first rough edge information image from an image; means for forming the first rough edge information image into a first formed edge information image using the digital image; and the first formed edge information. Means for generating a first object area image from an image; means for generating a duplicate information image from the first formed edge information image; means for selecting one moving object using the duplicate information image; Means for extracting only the selected moving object from the first object region image by a mask process; and extracting a second object from the digital image and the masked first object region image. Means for generating one area normalized image; means for holding the first area normalized image; means for storing a key image specifying the target object; means for vibrating the key image; Means for generating a second coarse edge information image from the key image that has been made; means for forming the second coarse edge information image into a second formed edge information image using the key image; and Means for generating a second object area image from the formed edge information image, means for generating a second area normalized image from the key image and the second object area image, and the first area normalization A moving image search device comprising: means for comparing an image with the second area-normalized image; and means for holding a comparison result. 3. The moving image search device according to claim 1, further comprising means for vibrating the digital image.