JPH09327037A - Contour extraction method, image extraction method and device - Google Patents

Abstract

An object of the present invention is to provide image extraction that enables efficient and highly accurate contour line extraction by using a plurality of images with different imaging conditions. An image input unit for inputting a plurality of images, an image memory for storing the plurality of images input by the image input unit, and distance data of pixel values between at least a pair of images of the plurality of images are calculated. An image pickup apparatus comprising: a first calculation unit that performs the above; and a second calculation unit that extracts a contour based on the distance data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup and image processing system having contour extraction and image extraction functions.

[0002]

2. Description of the Related Art Conventionally, as a general method for extracting (extracting) an image, a key signal is obtained by a chroma key using a specific color background or image processing (histogram processing, difference, differentiation processing, contour enhancement, contour tracking, etc.). Generated video mat (Technical Report of the Television Society, vol. 12, p.
p. 29-34, 1988) and the like.

In another method for extracting a specific area from an image, the method disclosed in Japanese Patent Publication No. 6-9062 discloses a method in which a differential value obtained by a spatial filter is binarized to detect a boundary, and the boundary is divided. The connected regions are labeled and the regions having the same label are extracted.

The method of extracting an image based on the difference from the background image is a classical one, and recently, it has been disclosed in Japanese Patent Laid-Open No. 4-216.
Japanese Patent No. 181 discloses a method of extracting or detecting target objects in a plurality of specific regions in an image by setting a mask image (specific processing region) in difference data between a background image and a processing target image. . In addition,
In the method disclosed in Japanese Unexamined Patent Publication No. 250, the existence probability distribution of the extraction target is obtained from the color conversion data of the current image including the background and the brightness difference data between the background image and the current image using the color model of the extraction target.

One of the techniques for extracting the outer contour line of an object from an image to extract a specific object image is the so-called dynamic contour method (M. Kass et al., "Snakes: A.
ctive Contour Models, “Int
international Journal of Com
putter Vision, vol. 1, pp. 321
-331,1987) is known. In this method, as an example, an initial contour that is appropriately set so as to surround the object moves, deforms, and finally converges on the outer shape of the object. The active contour method is a method (active contour method) for obtaining a target contour line by minimizing the evaluation function defined in this way. As a method for setting the initial contour, Japanese Patent Laid-Open No. 6-
138137, JP-A-6-251148,
There is JP-A-6-282652 and the like.

[0006]

However, among the above-mentioned conventional examples, the chroma key method has problems that it cannot be used outdoors due to a large restriction of the background and that the color fades. In addition, the video mat method has a problem in that it is necessary for a person to accurately specify the contour in pixel units, which requires labor and skill.

The method of utilizing the difference from the background image cannot be generally applied when an image of only the background excluding the subject cannot be captured (when the size and weight of the subject are enormous), and also the user. There is also a problem that there is a load on the side. In addition, because the imaging conditions (external conditions such as camera parameters and lighting) are not taken into consideration, if the background image and the image including the subject to be extracted have the same imaging conditions and are not acquired at the same fixed position, the extraction target from the difference data There is a problem that the judgment error of the area becomes very large. In addition, JP-A-7-
No. 16250 is unsuitable for image extraction of an unknown object in that it requires a color model to be extracted.

Among the above-mentioned initial contour setting methods of the active contour method, in Japanese Patent Laid-Open No. 6-138137, detection of a moving object area based on a difference between frames and extraction of a contour in the vicinity of the area (changed area) Maximum gradient edge search)
And is not applicable to stationary objects in any background. In Japanese Patent Laid-Open No. 6-282652, feature points with strong edges are extracted from an image, and points having a high evaluation value are selected from a feature point set based on an evaluation function to set an initial contour. Or, it is necessary that the image data changes gently.

A first object of the present invention is to enable efficient and highly accurate contour line extraction by using a plurality of images with different imaging conditions.

A second object of the present invention is to enable efficient and high-speed contour line extraction based on the first object by repeating the designation of the contour line to be extracted and the tracing of the contour line in the local area. It is to be.

A third object of the present invention is, for example, when processing distance data between a plurality of images having different focus states (for example, absolute difference value, square of difference, etc.), the minimum distance data at each position between the plurality of images. By tracing the point sequence as points on the contour line, it is possible to perform highly accurate contour line extraction in accordance with the first purpose.

A fourth object of the present invention is to improve the efficiency of the contour line tracking processing by setting the scanning direction for searching for the minimum point in the local area based on the tracking result, and to comply with the third object. This is to enable efficient and high-speed contour line extraction.

A fifth object of the present invention is, for example, when processing distance data between a plurality of images having different focus states (for example, absolute difference value, square of difference, etc.), a representative point (for example, a minimum point) on a contour line to be traced. ) Second with improved reliability of extraction accuracy
The purpose is to enable contour line extraction in accordance with the purpose of.

In addition to the fifth object, a sixth object of the present invention is to obtain the peak position of a local peak area even in a region where the representative point position on the contour line cannot be extracted with high reliability, so that the shape accuracy of the contour line can be improved. This is to enable high contour line extraction.

A seventh object of the present invention is to enable contour line extraction using color signals in accordance with the first object of high contour extraction accuracy.

An eighth object of the present invention is to enable efficient and highly accurate image cut-out of a specific subject by using a plurality of images having different imaging conditions.

A ninth object of the present invention is to use the distance data between a plurality of images under different imaging conditions to efficiently detect the boundary line with the background and enable highly accurate image cutting of a specific subject. It is to be.

A tenth object of the present invention is to enable image cutting (extraction) based on the third object.

An eleventh object of the present invention is to enable image cutting (extraction) based on the second object.

A twelfth object of the present invention is to facilitate the generation of a closed curve by concatenating a plurality of cut contour lines even if a plurality of broken contour lines remain as a result of contour tracing.

A thirteenth object of the present invention is to provide a plurality of boundary lines (closed curves) between a specific subject and a background (for example, when the subject image has an opening area including the background), either inside or outside thereof. That is, it is possible to selectively extract an image region based on an instruction.

A fourteenth object of the present invention is that, when there are a plurality of boundary lines (closed curves) between a specific subject and a background, the inside / outside discrimination data of the extracted image area is input all at once, and the batch extraction processing is performed by logical operation (product). It is possible to improve the image extraction efficiency.

A fifteenth object of the present invention is to input discrimination data when there are a plurality of boundary lines (closed curves) between the subject and the background, and when the outside of each boundary line is the extraction area, the data inside is deleted and the discrimination data is input. This is to enable sequential processing of region extraction or deletion each time.

A sixteenth object of the present invention is to provide a contour extraction method based on the first object.

A seventeenth object of the present invention is to provide an image cutting (extracting) method using a closed curve obtained by contour extraction according to the first object.

An eighteenth object of the present invention is to provide an image cutting (extracting) method using a closed curve obtained by performing contour extraction according to the first object from the difference or distance data between a plurality of images having different focus states. It is to be.

A nineteenth object of the present invention is to provide a method for extracting a connected area including an area to be extracted from distance data (for example, absolute difference value) between a plurality of images having different imaging conditions. For example, an area including a predetermined point and having distance data equal to or more than a predetermined threshold is merged to generate a closed area, and an area including the area and an area inside or outside the area is extracted.

A twentieth object of the present invention is to provide an image cutting (extracting) method based on the third object.

A twenty-first object of the present invention is, in the contour extraction conforming to the sixteenth object, capable of performing efficient contour line extraction by local processing and reliability evaluation and having high reliability between contour lines. By providing an interface for generating a curve connecting only those, the final contour shape is not significantly impaired.

A twenty-second object of the present invention is to perform contour line tracing from distance data between a plurality of images having different imaging conditions, and when the contour line has a branch structure or has low reliability, the contour line is located near the point. It is to provide an efficient and highly accurate contour line extraction method by providing an interface mechanism capable of setting control points and the like so as to pass.

A twenty-third object of the present invention is to provide a branching direction selection method in the case where a contour line has a branching structure with a background pattern, and an efficient removal method of a background pattern intersecting the contour line to be extracted. is there.

A twenty-fourth object of the present invention is to provide an automatic selection method of branch directions when a contour has a branch structure.

A twenty-fifth object of the present invention is to provide an interface mechanism by which the user can specify the contour line to be extracted and set the control by displaying the distance data between images having different imaging conditions. The object is to provide a precise and highly accurate outline extraction method.

A twenty-sixth object of the present invention is to provide a highly accurate contour line extraction method with few defects from absolute difference values (distance data) between images having different focus states.

A twenty-seventh object of the present invention is to perform stable image extraction by connecting and extracting adjacent homogeneous (values within a predetermined range) regions from distance data between a plurality of images under different imaging conditions. Is to provide a method.

[0036]

In order to achieve the above object, the invention according to claim 1 of the present application provides an image input means for inputting images under a plurality of image pickup conditions, an image storage means, and an image storage means for storing the plurality of images. It is characterized by comprising a first arithmetic means for calculating distance data of pixel values between at least a pair of images and a second arithmetic means for extracting a contour based on the distance data. As a result, the background pattern can be removed and the subject can be extracted efficiently and highly accurately by utilizing the imaging conditions.

According to a second aspect of the present invention, the second computing means for extracting the contour based on the distance data is a local area setting means centered on a point designated by a predetermined method,
The present invention is characterized by having a contour line tracking means for tracking a contour line in a local area and a recursive local area setting means centering on a contour line end point after tracking. As a result, efficient contour line extraction can be performed.

The invention according to claim 3 according to the present application is characterized in that the second computing means has a detecting means for detecting a minimum point position substantially adjacent to the maximum point of the distance data and a tracking means for tracking the minimum point position. To do. Thereby, for example, contour extraction can be performed with high accuracy from a plurality of images having different focus states.

In the invention according to claim 4 of the present application, the second calculating means is a peak including a scanning direction setting step in the local region, a scanning start point updating step, and a one-dimensional pixel value distribution maximum point in the scanning direction. It is characterized by having a minimum point detection step and a minimum point position tracking step in the region. As a result, the detection efficiency of the contour line to be extracted and its tracking efficiency can be improved.

In the invention according to claim 5 of the present application, the minimum point detecting means detects the minimum point position from the point where the difference from the pixel value of the maximum point is a predetermined threshold value or more, and the contour line tracking means detects the minimum point. It is characterized in that it has a curve generating means for connecting between them. As a result, a contour line is formed by using only points with high reliability of the contour line position, and other points or contour lines are approximated by generating an approximated curve to extract a contour line suitable for subject image extraction. You can

The invention according to claim 6 of the present application is characterized in that the curve generating means sets the maximum point of the peak region as a point on the curve in a region where there is no minimum point between the minimum points. With this, it is possible to obtain the same operation as that of the fifth aspect of the present invention, and it is possible to perform contour line extraction suitable for subject image extraction that can avoid significant distortion of the contour line shape.

According to the invention of claim 9 of the present application, the pixel value at the time of calculating the distance data comprises three kinds of color signals,
The second calculation means is characterized in that the contour is extracted based on the difference data for each color signal. This makes it possible to perform highly reliable contour line extraction using a plurality of images with different imaging conditions.

The invention according to claim 8 of the present application is an image input means for inputting images under a plurality of different image pickup conditions, a control means for the image pickup conditions, an image storage means, an instruction selection means for a point or area in the image. An image display means, a first calculation means for calculating distance data of pixel values between at least a pair of images of a plurality of images, and a second calculation means for extracting image data based on the distance data. . This effectively removes the background from images with multiple imaging conditions,
In addition, highly accurate image extraction is possible.

According to a ninth aspect of the present invention (image extracting apparatus), an image input means for inputting images under a plurality of different image pickup conditions, a control means for the image pickup conditions, an image storage means, a point or area in the image. Instruction selecting means, image display means, first calculating means for calculating distance data of pixel values between at least one pair of images of a plurality of images, second calculating means for extracting a contour line based on the distance data, contour line It is characterized by comprising a third arithmetic means for generating a closed curve from data and a fourth arithmetic means for extracting image data inside or outside the closed curve. As a result, the boundary line with the background can be efficiently extracted as a closed curve, and highly accurate image extraction can be performed. As the precise area extraction method, statistical processing (area growth, clustering), active contour method, or the like is used, but it is not particularly limited.

According to a tenth aspect of the present invention, in the eighth aspect of the invention, the second computing means includes a minimum point position detecting means and a minimum point position tracking means which are substantially adjacent to the maximum point of the distance data. It is characterized by having. This makes the third
It is possible to extract an image having the same operation as the invention of.

In the invention of claim 11 according to the present application, in the invention of the above claim, the second computing means is local area setting means centering on a point designated by a predetermined method, and traces a contour line in the local area. And a recursive local area setting means centered on the contour line end point after tracing. As a result, it is possible to extract an image having the same effect as the invention.

According to a twelfth aspect of the present invention, in the invention shown in the figure, the second computing means is the end point and the designated point on the contour line after the contour line is extracted with the point designated by the instruction selecting means as the center. It is characterized in that a curve connecting the other end points on the contour line that is extracted with a different point as the center is generated.
Accordingly, it is possible to efficiently extract a subject existing region by connecting a plurality of fragmentary contour lines obtained as a result of contour tracing to generate one or a plurality of closed curves.

According to a thirteenth aspect of the present invention, in the above-mentioned ninth aspect of the invention, the second computing means inputs the determination instruction data indicating whether the image region to be extracted is inside or outside the closed curve, and issues an instruction. It is characterized in that the image area is extracted based on the data. As a result, it is possible to specify the extraction target area with the minimum number of specifying steps.

In the invention of FIG. 15 according to claim 14 of the present application, in the invention of claim 9 described above, the fourth computing means inputs a plurality of the closed curve data and discrimination instruction data corresponding to each of them. The logical product data of the plurality of extracted image area data obtained based on each instruction data is calculated. As a result, it is possible to collectively specify and extract the target area with the minimum number of specifying steps.

According to a fifteenth aspect of the present invention, in the invention according to the sixth aspect, the fourth calculating means is based on the determination instruction data corresponding to the plurality of closed curve data input in a predetermined order. The feature is that the extraction processing of the image region is sequentially performed. Thereby, when there are many boundary lines (closed curves) with the background, it is possible to interactively perform the extraction processing in a desired order.

According to a sixteenth aspect of the present invention (contour extraction method), a step of inputting images under a plurality of image pickup conditions, a distance for calculating distance data of pixel values between at least a pair of images of the plurality of images. It is characterized by comprising a data calculation step and a contour extraction step of extracting a contour based on distance data. As a result, the contour extracting step having the same operation as the first aspect of the invention is realized.

According to a seventeenth aspect of the present invention (image extracting method), a step of inputting images under a plurality of image pickup conditions, a distance for calculating distance data of pixel values between at least a pair of images of the plurality of images. A data calculation step, a contour extraction step for extracting a contour based on distance data, a curve generation step for generating a closed curve based on the extracted contour, and a step for extracting image data inside or outside the closed curve. To do. As a result, the image extracting step having the same effect as that of the invention of claim 9 is realized.

The invention (contour extraction method) according to claim 18 of the present application is an image input step of controlling a focus condition and inputting a plurality of images of an image with a maximum focus degree condition and an out-of-focus image, It is characterized by comprising a distance data calculating step of calculating distance data of pixel values between at least a pair of images of a plurality of images and a contour extracting step of extracting a contour based on the distance data. As a result, the image capturing control step for performing highly accurate contour extraction is realized.

According to a nineteenth aspect of the present invention (image extracting method), a step of inputting images under a plurality of image pickup conditions, a distance for calculating distance data of pixel values between at least a pair of images of the plurality of images. It is characterized by comprising a data calculation step, a connected area generation step based on distance data, and an image data extraction step based on the connected area. As a result, if the contour line is complicated, or if the area where the distance data between images is approximately uniform corresponds to the extraction target (for example, when parallax between images with different viewpoint positions is extracted as distance data), By connecting and merging the regions having less data deviation, it is possible to finally improve the efficiency and accuracy of the precise image extraction processing by the statistical processing.

The invention according to claim 20 (image extracting method) according to the present application is the method according to claim 16, wherein the contour extracting step includes a step of detecting a minimum point position which is substantially adjacent to a maximum point of the distance data, and a minimum point. And a point position tracking step. Thereby, for example, a highly accurate contour extraction step from a plurality of images having different focus states is realized.

According to a twenty-first aspect of the present invention (image extracting method), in the invention of the sixteenth aspect, the contour extracting step is a local area setting step centered on a designated point input by a predetermined method, and a local area setting step. A contour tracing step for tracing the contour lines in the area, a step for recursively setting a local area centered on the contour edge points after tracing, and a curve generation step for connecting the remaining contour line endpoints after the tracing step It is characterized by having. As a result, the step of efficiently extracting the boundary line with the background from the images obtained under a plurality of different imaging conditions and extracting the subject is realized.

The invention according to claim 22 (image extracting method) according to the present application is the method according to claim 16, wherein the contour extracting step is a curve generation passing through a control point input by a predetermined method or a characteristic point in the vicinity thereof. It is characterized by having steps. As a result, it is possible to reliably perform the tracking direction selection when the contour line has a branch structure and the contour extraction when the contour extraction is difficult, and to improve the extraction accuracy without performing complicated processing. it can.

The invention according to claim 23 (image extracting method) according to the present application is the method according to claim 16, wherein the contour extracting step is a processing area setting step, or the local peak distribution of distance data in the processing area is maximum or If the existence of a branch structure is determined by the tracking step of the minimal point sequence, the branch structure determination step of the peak distribution, and the determination step, the position of the point sequence end point of one branch and the processing area based on the direction of the point sequence The method is characterized by including a step of determining an intersection and a branch selection step of selecting one of branch structures. Thereby, for example, by limiting the processing to the area surrounding the subject specified by the user and utilizing the intersection determination with the area, the background pattern (contour line) efficiently intersecting the contour of the subject is removed. It can be performed.

According to a twenty-fourth aspect of the present invention (image extracting method), in the invention according to the sixteenth aspect, the contour extracting step includes a step of tracking a local maximum peak or a local minimum point sequence of distance data and a peak. It has a branching structure judgment step of distribution and a branching selection step of selecting one of the branching structures. When the existence of the branching structure is judged by the judgment step in the tracing step, the branching selection step is a branch with high continuity in the point sequence direction. It is characterized by selecting a direction. As a result, contour line tracking can be performed to some extent without performing interactive processing through human hands, and efficient subject extraction is realized.

According to a twenty-fifth aspect of the present invention (image extracting method), in the invention of the sixteenth aspect, the contour extracting step includes a step of displaying a distribution of distance data on a predetermined display means, and a distance on the display means. The method is characterized by including a step of setting a control point based on the data distribution and a step of generating a curve passing through the control point or a characteristic point in the vicinity thereof.
This allows the user to set the control point through which the contour extracted at the region where the boundary with the background is not noticeable or where the branch structure occurs based on the distance data distribution between images, and the reliability of the subject extraction process by the interface mechanism. Is improved.

According to a twenty-sixth aspect of the present invention (image extracting apparatus), in the invention of the ninth aspect, the second computing means passes through a control point set by the instruction selecting means or a characteristic point in the vicinity thereof. A curve generating means, a minimum point of the distance data or a characteristic point extracting means for extracting a representative point in the local peak region when it is determined that the minimum point does not exist according to a predetermined criterion, and the characteristic point sequence tracking means. However, the curve generating means is characterized in that a curve existing at or near the control point and connecting between the adjacent characteristic points extracted by the characteristic point extracting means is generated. As a result, when a boundary line with the background is extracted from the distance data between multiple images with different focus states, if a highly reliable boundary line (local minimum point sequence in the local peak area) exists, it is tracked. If it does not exist, a contour line suitable for subject extraction is generated so as to pass through a representative point (local maximum point or midpoint) or control point of the local peak area. The subject has a complicated pattern shape,
Even if it is difficult to extract the boundary line with the background, the image can be surely extracted.

According to a twenty-seventh aspect of the present invention (image extracting device), in the invention according to the eighth aspect, the second calculating means connects and extracts a plurality of adjacent regions having substantially uniform distance data. It is characterized by Thus, for example, when extracting a region in which parallax (distance data) between images obtained under a plurality of imaging conditions with different viewpoint positions is within a predetermined range, or when the absolute value of difference (distance between images different in focus state It is possible to roughly and efficiently extract a subject region without performing contour tracking when extracting a specific subject from (data). Fine extraction is performed by systematic processing (clustering, region growing method, etc.).

[0063]

BEST MODE FOR CARRYING OUT THE INVENTION

(First Embodiment) An embodiment of the present invention will be described below with reference to the drawings.

In this embodiment, the focus state is controlled, and the specific subject extraction processing from images under different image pickup conditions can be performed stably and at high speed irrespective of the illumination conditions. Extraction of a specific subject means cutting out from the background, but this may be applied to processing such as image classification, recognition, and tracking.

FIG. 2 is a diagram showing the main configuration of the image processing camera 1 having a photographing mode for extracting a specific subject and an extraction processing mechanism used in this embodiment, and a terminal 20, a display 21, and an instruction selecting means 22 connected thereto. It is a figure. However, the processing required for image extraction may be performed on the terminal instead of the camera. The subject extraction shooting mode refers to an image pickup operation mode in which image pickup parameters (lens arrangement, shutter speed, aperture diameter, etc.) are controlled to output a plurality of image data from an image sensor to an image recording (storing) means. .

Reference numeral 1 denotes the entire image pickup means (camera head) which is an image input means, and 2 denotes an image forming optical system for facilitating the discrimination between the main subject and the background. A shallow depth of focus is desirable for clarity. Reference numeral 3 denotes a lens motor drive control unit, which is a focusing lens drive unit (focus motor), a zoom lens drive unit,
And lens information reading (measuring) means such as the type of mounted lens. Reference numeral 4 is an image sensor, typically a solid-state image sensor such as CCD, and 5 is imaging parameter measurement control means, and zoom detection control means 5
1. Focus signal (focusing state) detection control means 52, shutter speed detection control means 53, aperture measurement control means 54, image signal characteristic parameters (gamma, knee, white balance correction, accumulation time (CCD), etc.) It includes a detection means 55 (for example, a correction coefficient for gamma). The shutter speed control means 53 controls a mechanical shutter (not shown). When an electronic shutter is used, the sensor drive circuit 18 described later controls a substantial shutter speed image sensor (charge accumulation time). . The focus state may be detected by either a detection method using a video signal or a measurement method using infrared rays. An image recording / reproducing unit 6 digitally records in a predetermined format on a recording medium (tape medium, disk medium such as optical (magnetic) disk, or IC memory medium such as IC memory card) at the time of image pickup.

The recording medium or the image recording / reproducing means 6 may be detachable from the camera body. Further, the image capturing operation procedure program in the subject extraction mode and necessary data may be written in the recording medium, and the image capturing may be controlled based on the information.

The electronic viewfinder (E
VF) monitor 7 or data transfer means 12 described later
The image data is output to.

Reference numeral 8 denotes an image pickup mode recording means, which is used by the lens motor driving means 3 to obtain lens information, image pickup parameters, image signal characteristic parameters, presence / absence of strobe light emission, intentional camera operation (movement) such as scanning, and camera shake. Information at the time of imaging including the presence or absence is recorded. These pieces of information are recorded in any of external control, manual operation, and autonomous control by a built-in program, and can be used later to reproduce the same imaging condition.

The camera operation (scan, pan, etc.) information may be electronically detected from the motion information in the image signal by video signal processing, or the image pickup means may be provided with an acceleration sensor (not shown), a gyro or the like. It may be built in and physically judged from the output data.

Reference numeral 9 denotes a switch as an image pickup mode setting means for subject extraction, tracking, identification, etc. When this switch is set, the switch is turned on and the image pickup start operation signal from a shutter button or the like (not shown) causes the same. An image is captured in the image capturing mode. The imaging start switch may also be configured to double as a subject extraction mode setting switch.

Reference numeral 10 is a command data communication control means for controlling communication of command data from the outside via the terminal 20, and 11 is an output image generation means for generating an analog / digital image signal according to an output destination.

For example, an NTSC signal or PAL signal is generated for a TV receiver. Further, the image signal which has been subjected to the gradation correction of the light and shade according to the output device, the error diffusion processing, and the color component correction processing may be output to an externally connected display, a printer or the like.

It is assumed that such an output signal generation and processing instruction is given externally via the command / data communication control means 10 or by using a switch (not shown) on the operation panel of the camera.

Reference numeral 12 is a data transfer means, which includes a communication interface means (not shown), an adapter terminal, etc.
Used when outputting image data to an external display device or computer.

Reference numeral 13 denotes a subject extracting means, which performs a process such as discrimination between a background region and a subject region, extraction of contour lines, and clipping of an image on an image obtained under a plurality of image pickup conditions by a method described later. The detailed internal structure will be described later.

Reference numeral 14 is an image capture signal generation circuit,
Appropriate capture of images by automatically checking the exposure conditions, control conditions of image capturing conditions, charge storage / transfer time in the image sensor (CCD), and timing of image writing control signals in the memory 6 Is for doing.

Reference numeral 15 denotes a system control section as a means for setting a plurality of image pickup conditions, and the command data communication control means 1
0, a signal from the imaging mode setting switch 9 or information of a control program written in the recording medium of the image recording / reproducing means 6 is read to generate a series of imaging operation control signals in the subject extraction mode.

Reference numeral 16 is a diaphragm, and 17 is a video signal processing section, which includes a preprocessing circuit (17a; a sample and hold circuit, an auto gain control circuit, an A / D conversion circuit, etc.), a gamma correction circuit (17b), and other video. It is composed of a signal processing circuit (17c; including a white balance correction circuit, a camera shake correction circuit, etc.). Note that these circuit configurations are not limited to those schematically shown in FIG. A drive circuit 18 for the image sensor 4 controls the charge storage time, the transfer timing, and the like. Further, in the figure, the flow of the control signal particularly related to the present embodiment is shown by a thick arrow.

In the present embodiment, in the subject extraction image pickup mode, the focusing lens motor (focus motor) is driven on the basis of the output from the focusing level detecting means to obtain the optimum focusing level, that is, the subject is most focused. In this condition, after capturing a high-resolution image from the image sensor, the lens is then driven so that the focus shifts in the same direction as the driving direction of the focusing lens motor immediately before, and the low-level at an appropriate non-focus level. The resolution image is continuously captured.

Here, the appropriate out-of-focus level is a value set in advance and stored in the storage means or a value set by a command signal from an external terminal, for example, a ratio from the in-focus level. (A numerical value like 90% of the focus value)
Given in.

Still further, the image may be picked up at different out-of-focus levels and used for extracting the initial data for image cutting (roughly cut image data).

When the above continuous image pickup operation is completed, the subject extraction mode is automatically canceled, or the image pickup mode setting switch 9 is used to set the return to the standard mode. The above-described operations such as setting and canceling the imaging mode may be performed by command input from an external device via a terminal (not shown).

FIG. 4 schematically shows the relationship between the focus signal and the image pickup position of the focus lens in the case of performing image pickup under a plurality of image pickup conditions under focus control. The figure shows that the focus lens is driven from the initial position to detect the focus level once, and an image is taken, and then the focus lens is driven by a predetermined amount to capture an out-of-focus image within an allowable range.

When a plurality of low-resolution images having different out-of-focus degrees are used, high spatial frequency components are cut, and as will be described later, the distance data (difference absolute value data, etc.) is used as a target for identification. When extracting the contour line, it is possible to control the generation of extra isolated feature points or isolated regions.

Therefore, it is possible to obtain stable initial data (contour line) with stable accuracy without being affected by noise, high spatial frequency components in the background, or illumination conditions.

Further, by using the imaging optical system having a shallow depth of focus, especially when the distance between the extraction target and the background is sufficiently larger than the subject distance, the boundary portion between the background and the subject in the difference image data is increased. It is also possible to suppress the branching structure of the contour line appearing in, or to selectively extract the contour line of only the specific subject without performing the tracking process.

FIG. 5 shows a basic control flow in the case where a system in which the camera and the terminal are connected to each other or an operation on the camera is basically used. The terminals include terminals connected by lines or cables and remote controllers.
In the figure, items indicated by a bold frame may be performed on the terminal side, and items indicated by a broken line may be performed by either the camera or the terminal. Others indicate a functional operation performed in the camera or direct operation of camera switches.

First, a communication session with the camera is established from the terminal side (S5a), and a subject extraction imaging mode setting command is transmitted to the camera (S5b). Alternatively, the subject extraction mode is set by manual operation of the operation panel of the camera (S5a ').

At this stage, the basic control program is operated (S5c), or the imaging parameter to be controlled, the control amount of the imaging condition, the number of images to be taken, and the processing range inside the camera (for example, calculation of difference data for clipping processing). (Pre-processing such as is specified only for shooting under multiple shooting conditions) is input from the terminal side (S5d)
Or either of them can be selected. This may be selected from a menu or the like displayed on the display means of the terminal.

It is assumed that these conditions are preset as standard in the basic control program. During image pickup by setting a plurality of image pickup conditions (S5e), control of image pickup conditions based on the set parameters and image pickup parameters (lens data, focal length, magnification, focus signal, focus lens arrangement, gamma control value, aperture) at that time are controlled. Diameter, charge accumulation time, etc.) is recorded.

After that, there are three control flows depending on the processing range inside the camera. It automatically returns to the standard mode (S5f), receives the cutout condition input after the imaging operation in the subject extraction mode (S5g), or performs the cutout process (S5h; described later) under the preset standard condition.

After the processing is completed, the cut-out image may be recorded in the storage means in the camera body, transferred to the external memory or the display device (S5i), or displayed on the finder display (S5i ').

It is assumed that the output form of the cut-out image is designated in advance when the control data is input or in the basic program. After the above processing, the automatic or manual return to the standard imaging mode is finally performed (S5f).

Next, the process of processing each image captured by the image sensor 4 will be described in detail. This processing can also be performed in a terminal connected to the camera.
The processing can be registered as a standard processing mode by the arithmetic means inside the camera, or can be designated from a terminal or the like via the command data communication control means 10. Typically, as standard processing inside the camera, distance data between images under a plurality of imaging conditions, that is, calculation of absolute difference value data, binarization processing, smoothing processing, and the like are set as a basic control program. Here, first, gamma correction, white balance correction and the like are performed in the video signal processing means 17 on the video signals from at least two sensors obtained by controlling the in-focus state (focus). Contour extraction processing means 1 inside the image pickup means
3, the difference between the high-resolution image (focused image) and the low-resolution image (non-focused image) or the difference absolute value data between two different low-resolution images is calculated, and the contour line extraction processing is performed on them. I do. The processing other than the calculation of the absolute difference value may be performed by inputting the processing parameter from the external terminal (not shown) or the like via the command data communication control means 10.

FIG. 3 is a diagram for explaining the principle of contour line extraction in this embodiment. Now, as the distance data between the two-sided images of the step edge intensity distribution (a) of the focused image and the blurred edge intensity distribution (b) of the unfocused image, the absolute difference distribution (c) is a local maximum (peak). Region) has a sharp minimum point. This minimum point position corresponds to the boundary point position of the step edge.

The determination of the minimum point to be extracted may be performed on the condition that the difference (positive or zero) with the maximum level in the peak region is equal to or larger than a predetermined threshold value.

When the image input from the image pickup means is a color image, the absolute difference value is calculated for each color component (R, G, B, etc.), and the distance data of the color component for which the minimum point position is similarly used is used. Good.

The basic method of contour line extraction is to trace this point sequence, but a clear minimum point may not always appear depending on the two-dimensional distribution shape of pixel values. In such a case, a representative point (for example, a maximum point,
It is also possible to extract a midpoint, an end point detected by scanning the peak region in a direction substantially orthogonal to the tracking direction). Further, a curve that smoothly connects the minimal points that appear clearly may be generated.

Next, the cutout processing (contour line tracking processing) will be described with reference to FIG. This processing may be performed either inside the subject extraction means 13 or outside terminal means. In the latter case, the plural images are once recorded in the memory and the terminal data is supplied, or they are transferred to the terminal side via the data communication means.

First, by limiting the processing range, the instruction selecting means 22 such as a mouse sets a minimum-size processing area including a subject to be extracted for the purpose of enhancing the extraction efficiency (Sx0).

Next, a contour tracking start point or a control point described later is set by the instruction selecting means 22 such as a mouse (Sx).
1). At the designated start point, a local area of a fixed size (for example, a block of 16 × 16 pixels) centered on the designated start point is automatically set. This is performed by scanning the inside of the screen to detect the edge portion of the image (Sx2 ). As for this area size, a local area having an arbitrary shape may be set here with a mouse or the like.

The contour line tracing process (Sx3) is performed to trace the edge portion, that is, the contour in the area (needless to say, the local area includes the contour line to be extracted). After the processing is completed, a local area centering on the end point is automatically set (Sx4), and similar processing is repeated to finally obtain a closed curve forming a boundary line with the background.

In the contour tracking process (Sx3), distance data (absolute difference value) between images in the local area is displayed on the display (Sx5; it may be displayed by superimposing it on the input image), and a plurality of images are displayed. If there is an edge or a contour line, a selection instruction (Sx6; designation of extracted contour line) may be made using a means such as a mouse as to which edge to take.

Then, when only a single edge exists or when an edge is selected by the user, contour line tracking processing is performed in a preset direction, and each position of sampling points is sequentially extracted. Here, the sampling point in this embodiment refers to the minimum point or the representative point in the local peak area of the difference absolute value data as described above.

In the tracking process, when the contour line or the local peak region has a branch structure, a branch direction selection process (S
x7). As a method, selection of a branching direction in which image data attributes (color, brightness or dispersion thereof) of the contour line in a direction orthogonal to the tracing direction of the point before the branching is selected, a branching direction in which a control point exists, or May preferentially select a direction in which the direction change before and after branching is small.

In the former case, the image attribute of a minute area centered on each point after the branch is detected. As another method, the user may be allowed to select which branch to take by interrupting the contour tracking and blinking the area around the branch point.

As a result of the contour line tracing, when the divided contour lines remain in the middle, the connection processing (Sx8) is performed. In this case, a predetermined edge connection algorithm (23rd
Annual Meeting of Image Engineering Conference, pp. 67-70,
1992) or by interactive processing. The latter performs blinking display of the end points of the contour line or color display different from the contour line, and the user confirms and indicates the instruction selection means 2.
A connection curve is generated and connected according to the instruction using 2.
As the contour line connecting the connection points, a spline interpolation curve or the like may be generated using a straight line or a representative point on the contour line. In particular, when the end point after contour tracing is on the image frame or the frame of the processing area, it may be connected to the upper end point of the other frame.

As described above, the closed contour line that finally forms the boundary with the background is obtained. The image data inside or outside this curve is the area to be extracted.

If there are a plurality of closed borders with the background similar to the background of the subject to be extracted (closed curve), the area to be extracted for each contour line is inside the border. Indication information is required whether it is external or external. If the user gives an instruction as to whether the extraction area is inside or outside, or is given each time the respective contour lines are extracted (Sx9),
Image extraction (Sx10) progresses sequentially, and finally desired subject extraction is completed. It is also possible to give an instruction of the inside / outside condition each time each boundary line is extracted, and collectively perform the extraction by the logical product of the image areas that meet each condition.

FIG. 6 is a diagram for explaining a process of extracting a specific subject image by using a plurality of screens having different degrees of focus described above.

In the figure, (1) shows an image focused on a subject, and (2) shows an image not focused on the same subject. By subtracting these images,
Based on the principle shown in FIG. 3, the common image portion is canceled between both images, and only the portion representing the contour of the subject is emphasized and appears. In (4), after the subtraction process is performed, the absolute value of the difference is smoothed and further binarized, the specific subject image and the background image are separated, and only the contour image is extracted.

On the other hand, in the focused image of (1), it is possible to extract only the designated image as shown in (5) by designating the image area to be processed as the specific subject. .

In this image cutting process,
For example, a plurality of image data of contour portions of a plurality of data images are obtained by statistical processing such as a least square method to obtain a portion having a minimum error, and a contour image is extracted, or the images are changed on the screen while the images are changed. The displacement can be detected, the contour portion of the image can be accurately obtained, and other processing such as the dynamic contour method can be performed to finally extract only the specific subject image as shown in (5).

(Second Embodiment) In the present embodiment, a specific subject is identified based on a parallax distribution between a plurality of images obtained by capturing images of the same subject at different viewpoint positions using a plurality of cameras or a single camera. Separate and extract from the background.

FIG. 7 shows the processing flow of this embodiment.

As in the case of FIG. 5, items indicated by a bold frame may be performed on the terminal side, and items indicated by a broken line may be performed by either the camera or the terminal. Indicates a functional operation performed by another camera or direct operation of camera switches.

The subject extraction mode is set by manipulating the manual of the operation panel of the camera or inputting a command from the terminal (S7a). At this time, the imaging parameters, the number of shots, and the processing range inside the camera (for example, preprocessing such as difference data calculation for cutout processing, only shooting under multiple shooting conditions are specified), etc. May be input and set from the terminal side (S7
a ').

Next, the basic control program stored in the storage means inside the camera operates (S7c). First, an approximate distance to a subject is calculated by a focus signal or the like in a single camera or one camera of a compound eye imaging system (S7).
c) Next, based on that value, an appropriate value of the viewpoint position control parameter (eg, the baseline for a compound eye camera, the viewpoint moving distance for a single camera) is estimated (S7d).

At the time of photographing (S7e), the setting of the imaging condition based on the set parameters (viewpoint movement amount or base line length) and the imaging parameters at that time (lens data, focal length, magnification, focus signal, focus lens arrangement) , Gamma control value, aperture diameter, charge storage time, etc. are recorded.

After that, the processing range within the camera (S7
There are three types of control flows according to (already set in a ').
It automatically returns to the standard mode (S7f), receives the cutout condition input after the image pickup operation in the subject extraction mode (S7g), or performs the cutout process (S7h; described later) under preset standard conditions. Whether to do it. After finishing the process, store the clipped image in the storage means in the camera body,
It may be transferred to an external memory or a display device (S7i), or may be displayed on the finder display (S7i ').

It is assumed that the output form of the cutout image is designated in advance when the control data is input or in the basic program. After the above processing, the automatic or manual return to the standard imaging mode is finally performed (S7f).

The image cutout process (S7h) will be described below. Here, based on the parallax distribution between corresponding points as the distance data between images, the region division at the discontinuous boundary portion or the region extraction having the parallax value within a certain range is performed.
FIG. 8 is a diagram for explaining the principle of initial data extraction based on the parallax distribution.

Corresponding point extraction (S7h-1) is typically performed by a correlation method (a block of a predetermined size is set for each pixel,
Extraction of the parallax distribution as inter-image distance data (S7h-2) is performed in the horizontal direction between the corresponding points (viewpoint If the position is shifted horizontally) Extract as a distance.

Next, the parallax distribution is displayed on the display for calculating initial data for subject extraction (S7h-
3) The user specifies the subject to be extracted while confirming the real image and the parallax distribution, and divides the image area including the representative parallax value and extracts the connected area (S7h-4).

FIG. 8 illustrates the above-mentioned processing for extracting a specific image from the parallax distribution. In FIG. 8A, the horizontal axis represents the parallax value (baseline length, viewpoint distance), and the vertical axis represents the parallax value. This is the frequency at which the value appears and is the parallax value corresponding to the position where the subject is present, and the detection frequency is high.
It can be seen from the figure that the frequency is high in the background and the extraction target to be extracted. That is, it is possible to detect the contour of the subject image.

8 (b)-(1) show the original image, and FIG.
(B)-(2) represents the parallax distribution display image when the original image is viewed.

That is, looking at the parallax distribution (FIGS. 8 (b)-(2)) displayed on the display in FIG. 8 (b), the user designates and selects a part of the area of the extracted subject similar to the previous embodiment. Instructed by means, merge the connected regions whose disparity value difference with that point or region is within a predetermined range, and extract as initial data for cutting out (FIGS. 8 (b)-(2)).

Further, unnecessary data (background, etc.) locally present in the initial data is removed by using an instruction means such as a mouse, or merged as a part of the extraction target (S7h).
-5). The initial data obtained in this way is not sufficient for cutting out accuracy due to the presence of occluded areas (areas that can be seen only on one side of the camera), corresponding points and parallax extraction errors, but this is used as the mask Statistical processing (region growth, clustering, etc.) or the active contour method is applied to the image (S7h-6), only the contour portion of the extracted image is extracted, and finally, precise image extraction is performed to specify. The subject is extracted (FIGS. 8B to 8D).

As a method of dividing the parallax distribution into regions, a method of setting an appropriate threshold value from a histogram may be used. In this case, of the two adjacent peaks in FIG. 8A (one has a background with a small parallax value and the other has an object with a large parallax value), for example, the width of the object-side peak is given as 2
Of the two parallax values, the parallax value on the background side may be used as the threshold value.

[0131]

According to the invention described in claim 1 of the present application, the background pattern can be removed and the subject can be extracted efficiently and highly accurately by utilizing the imaging conditions.

According to the invention described in claim 2 of the present application, efficient and high-speed contour line extraction can be performed by repeatedly designating the contour line and tracing the contour line in the local area.

According to the invention of claim 3 of the present application, the contour can be extracted with high accuracy from a plurality of images having different focus states.

According to the invention described in claim 4 of the present application, the detection efficiency of the contour line to be extracted and the tracking efficiency thereof can be improved.

According to the invention described in claim 5 of the present application, a contour line (boundary with the background) suitable for subject image extraction by using a sequence of points with high reliability of contour line position and generation of an approximate curve together
Was able to be extracted.

According to the invention of claim 6 of the present application, it is possible to perform contour line extraction suitable for subject image extraction with stable accuracy without being significantly affected by the distribution of image data near the contour line. It was

According to the invention of claim 7 of the present application, the contour line extraction accuracy is improved by using the color signal in the method of contour extraction from the image of the plurality of image pickup conditions.

According to the invention of claim 8 of the present application, the background is efficiently removed from the images of a plurality of imaging conditions,
And it became possible to extract images with high accuracy.

According to the invention described in claim 9 of the present application, it is possible to efficiently extract the boundary line with the background as a closed curve and to extract an image with high accuracy.

According to the invention of claim 10 of the present application, it is possible to extract an image having the same effect as that of the invention of claim 3.

According to the eleventh aspect of the present invention, it is possible to extract an image having the same effect as that of the second aspect of the invention.

According to the twelfth aspect of the present invention, a plurality of fragmentary contour lines obtained as a result of contour tracing are concatenated to generate one or more closed curves, and the subject existing region is efficiently formed. Can be extracted.

According to the thirteenth aspect of the present invention, it is possible to specify the extraction target area with a minimum number of specifying steps.

According to the fourteenth aspect of the present invention, when there are a plurality of boundary lines (closed curves) with the background, batch extraction processing can be performed by logical operation (product), and image extraction efficiency is improved. Is possible.

According to the fifteenth aspect of the present invention, when there are many boundary lines (closed curves) with the background, it is possible to interactively perform extraction processing in a desired order.

According to the sixteenth aspect of the present invention, it is possible to extract the contour having the same effect as that of the first aspect of the invention.

According to the seventeenth aspect of the present invention, it is possible to perform contour extraction having the same effect as that of the ninth aspect of the invention.

According to the eighteenth aspect of the present invention, the imaging control step for performing highly accurate contour extraction is realized.

According to the invention described in claim 19 of the present application, the efficiency and accuracy of the precise image extraction processing by the statistical processing are finally improved by connecting and merging the areas having small deviations in the distance data. It is possible to

According to the procedure of the invention described in claim 20 of the present application, it is possible to cut out (extract) an image based on the invention of claim 3.

According to the twenty-first aspect of the present invention, the step of efficiently extracting the boundary between the background and the subject is realized.

According to the twenty-second aspect of the present invention, even if contour extraction or contour tracing is difficult, a curve passing through the control point designated by the user is generated, so that the boundary line with the background is surely obtained. And it is possible to perform highly accurate image extraction with a small number of calculation processing steps.

According to the twenty-third aspect of the present invention, by utilizing the intersection determination with the designated processing area, it is possible to efficiently remove the background pattern or contour line that intersects with the boundary line of the subject. It is possible to

According to the twenty-fourth aspect of the present invention, automatic contour line tracking can be performed in subject extraction utilizing the imaging conditions, and efficient subject extraction is possible.

According to the twenty-fifth aspect of the present invention, by the interface mechanism that enables the user to interactively set the control point through which the contour should pass based on the display of the distance data distribution between the images. The reliability of subject extraction processing has been improved.

According to the twenty-sixth aspect of the present invention, an image with a blurred boundary with the background, a background or a subject having a complicated pattern shape, and reliable extraction of the boundary line with the background are reliable. Image extraction is possible.

According to the twenty-seventh aspect of the present invention, the efficiency of the rough extraction of the subject region and the background removal can be improved by subjecting the subject having a complicated shape to region-based extraction processing utilizing different imaging conditions. Has improved.

[Brief description of drawings]

FIG. 1 is a basic processing flow chart of image cutout processing by contour tracking according to the present invention.

FIG. 2 is a main part configuration diagram of an imaging unit having a subject extraction function, which is connected to a terminal device or the like.

FIG. 3 is an explanatory diagram of a boundary line detection principle by calculating distance data from a plurality of images obtained by controlling a focus state.

FIG. 4 is an explanatory diagram of a focus motor position control range based on a focus state at the time of shooting for subject extraction.

FIG. 5 is a control flow chart from an image capturing operation to image cutting according to the present invention.

FIG. 6 is an explanatory diagram of a process of extracting a specific subject from a plurality of images having different focus states.

FIG. 7 is an explanatory diagram of a subject extraction processing flow based on a parallax distribution obtained by capturing images at different viewpoint positions.

FIG. 8 is an explanatory diagram of an initial data extraction principle based on a parallax distribution.

[Explanation of symbols]

 1 Image Input Means Main Unit 2 Imaging Optical System 3 Lens Motor Drive Control Unit 4 Image Sensor 5 Imaging Parameter Measurement Control Means 6 Image Recording / Reproducing Means 7 Viewfinder 8 Imaging Parameter Recording Means 9 Subject Extraction Imaging Mode Setting Switch 10 Command Data Communication Control means 11 Output image generation means 12 Data transfer means 13 Subject extraction means 14 Image capture signal generation means 15 System control section of image input means body 16 Aperture 17 Video signal processing section 18 Image sensor drive circuit 20 Terminal 21 Display means 22 Instruction Means of selection

Claims (27)

[Claims] 1. An image input unit for inputting images under a plurality of image pickup conditions, an image storage unit for storing the input images, and distance data of pixel values between at least a pair of images of the plurality of images. A contour extracting device comprising: a first computing means; and a second computing means for extracting a contour based on the distance data. 2. A local area setting means centering on a point designated by a predetermined method, and a contour line tracking means for tracking a contour line in the local area according to claim 1. And a recursive local area setting means having a contour line end point after tracking as a center. 3. The method according to claim 1, wherein the second calculation means has a minimum point position detecting means which is substantially adjacent to the maximum point of the distance data, and a minimum point position tracking means. Contour extraction device. 4. The method according to claim 3, wherein the second calculation means determines a scanning direction setting step in the local region, a scanning start point updating step, and the local maximum point of the one-dimensional pixel value distribution in the scanning direction. An outline extracting apparatus comprising: a step of detecting the minimum point and a step of tracking the position of the minimum point in the included peak region. 5. The minimum point detecting means according to claim 2, wherein the minimum point position is detected from a point whose difference from the pixel value of the maximum point is equal to or more than a predetermined threshold value, and the contour line tracking means detects between the minimum points. A contour extraction device having a curve generation unit that connects the contours. 6. The contour extracting device according to claim 5, wherein the curve generating means sets a maximum point of the peak region as a point on the curve in a region where there is no minimum point between the minimum points. . 7. The contour extracting device according to claim 1, wherein the pixel value is composed of three kinds of color signals, and the second computing means extracts a contour based on difference data for each color signal. . 8. An image input unit for inputting images under a plurality of different image pickup conditions, and a control unit for controlling the image pickup conditions.
Image storage means, point or area instruction selection means in the image, image display means, first calculation means for calculating distance data of pixel values between at least a pair of images of the plurality of images, and the distance data An image extracting device, comprising: a second arithmetic means for extracting image data based on the above. 9. An image input unit for inputting images under a plurality of different image pickup conditions, and a control unit for controlling the image pickup conditions.
Image storage means, point or area instruction selection means in the image, image display means, first calculation means for calculating distance data of pixel values between at least a pair of images of the plurality of images, and the distance data A second calculating means for extracting a contour line based on the contour line data, a third calculating means for generating a closed curve from the contour line data, and a fourth calculating means for extracting image data inside or outside the closed curve. An image extraction device characterized by: 10. The method according to claim 9, wherein the second calculation means has a detection means for detecting a minimum point position substantially adjacent to the maximum point of the distance data and a tracking means for tracking the minimum point position. Image extraction device. 11. The method according to claim 9, wherein the second computing means includes a local area setting means having a point designated by a predetermined method as a center, and a contour line tracing means for tracing a contour line in the local area. An image extraction apparatus, comprising: a recursive local area setting means having a contour line end point after tracking as a center. 12. The method according to claim 9, wherein the second computing means is centered on a point different from the designated point and the end point on the contour line after the contour line is extracted with the point designated by the instruction selecting means as the center. An image extraction device characterized by generating a curve that connects with other extracted end points on a contour line. 13. The method according to claim 9, wherein the second computing unit inputs the determination instruction data indicating whether the image area to be extracted is inside or outside the closed curve, and extracts the image area based on the instruction data. An image extraction device characterized by. 14. The fourth arithmetic means inputs a plurality of the closed curve data and the determination instruction data corresponding to each of the closed curve data, and calculates logical product data of a plurality of extracted image area data obtained based on each instruction data. An image extraction device characterized by: 15. The method according to claim 9, wherein the fourth arithmetic means sequentially performs the image area extraction processing based on the determination instruction data corresponding to the closed curve data input in a predetermined order. Image extraction device. 16. A step of inputting images under a plurality of imaging conditions, a distance data calculating step of calculating distance data of pixel values between at least a pair of images of the plurality of images,
A contour extracting method comprising a contour extracting step for extracting a contour based on the distance data. 17. A step of inputting images under a plurality of imaging conditions, a distance data calculation step of calculating distance data of pixel values between at least a pair of images of the plurality of images, and a contour extraction based on the distance data. An image extraction method comprising: a contour extraction step to be performed; a curve generation step of generating a closed curve based on the contour; and a step of extracting image data inside or outside the closed curve. 18. An image input step of controlling a focus condition to input a plurality of images of an image with a maximum degree of focus and an out-of-focus image, and a pixel value distance between at least a pair of images of the plurality of images. A contour extracting method comprising: a distance data calculating step of calculating data; and a contour extracting step of extracting a contour based on the distance data. 19. A step of inputting images under a plurality of imaging conditions, a distance data calculating step of calculating distance data of pixel values between at least a pair of images of the plurality of images,
An image extraction method comprising: a connected area generating step based on the distance data; and a step of extracting image data based on the connected area. 20. The contour extracting method according to claim 16, wherein the contour extracting step includes a step of detecting a minimum point position substantially adjacent to a maximum point of the distance data and a tracking step of the minimum point position. . 21. The contour extraction step according to claim 16, further comprising a local area setting step centered on a designated point input by a predetermined method, and a contour line tracking step of tracing a contour line in the local area. A contour extraction method comprising: a step of recursively setting a local area centered on a contour line end point after tracking; and a curve generation step of connecting remaining end points of the contour line after the tracking step. 22. The contour extracting method according to claim 16, wherein the contour extracting step includes a curve generating step that passes through a control point input by a predetermined method or a characteristic point in the vicinity thereof. 23. The contour extracting step according to claim 16, a processing area setting step, a step of tracking a local maximum or minimum point sequence of a local peak distribution of the distance data in the processing area, and a branching of the peak distribution. A structure determining step, and a step of determining an intersection between the position of the point sequence end point of one branch and the processing region based on the point sequence direction in the tracking step when the presence of a branch structure is determined by the determining step And a branch selection step of selecting one of the branch structures. 24. The contour extracting step according to claim 16, wherein one of a branching structure determining step and a branching structure determining step of the peak distribution of a local peak distribution or a local minimum point sequence of the distance data is selected. A contour having a branch selection step, wherein the branch selection step selects a branch direction having high continuity in the point sequence direction when the presence of the branch structure is determined by the determination step in the tracking step. Extraction method. 25. The contour extraction step according to claim 16, wherein the distribution of the distance data is displayed on a predetermined display means, and a control point is set based on the distance data distribution on the display means. An image extraction method comprising a step of generating a curve passing through the control point or a characteristic point in the vicinity thereof. 26. The curve calculating means according to claim 9, wherein the second computing means passes through a control point set by the instruction selecting means or a characteristic point in the vicinity thereof, and a minimum point of the distance data or the minimum point. Has a feature point extracting means and a feature point sequence tracking means for extracting a representative point in the local peak region when it is determined that the control point or the control point is not present according to a predetermined determination criterion. An image extracting device characterized by generating a curve existing in the vicinity thereof and connecting between adjacent characteristic points extracted by the characteristic point extracting means. 27. The image extraction device according to claim 8, wherein the second calculation means connects and extracts a plurality of substantially uniform adjacent regions of the distance data.