JP2012015674A - Imaging device, operation control method for imaging device, and program for imaging device - Google Patents

Abstract

Image data required for constructing a three-dimensional image model is efficiently obtained.
An occlusion area that appears in a target image and does not appear in another image is obtained. If an image area corresponding to the occlusion area of the image of interest appears in the moving picture S4 obtained by imaging with the digital still camera 1, the occlusion corresponding area ROC42 is displayed in the moving picture S4. It is possible to observe (view) an image that is lacking in the construction of a three-dimensional image model through a moving image.
[Selection] Figure 14

Description

  The present invention relates to an imaging apparatus, an operation control method thereof, and a program thereof.

  In order to construct a three-dimensional image of an object, it is necessary to include an image area common to a plurality of images obtained by imaging the object from different imaging positions (viewpoints).

  Patent Document 1 describes that a shooting position and direction for constructing a three-dimensional image are calculated by using a GPS sensor and an orientation sensor. Relatively expensive sensors such as a GPS sensor and an orientation sensor are required.

  Patent Document 2 describes that a parallax image sequence is generated by photographing an object a plurality of times while moving a camera on a straight rail. It does not provide the photographer with information such as a position where a new image necessary for constructing a three-dimensional image model should be taken.

JP 2008-154027 A JP 2006-154800 A

  To make it possible to observe (view) images that are lacking in the construction of a 3D image through a video displayed on the display screen, and to efficiently acquire the image data necessary for the construction of the 3D image. For the purpose.

  An imaging device according to a first aspect of the present invention is an imaging means for imaging an object and outputting image data obtained by imaging, and a moving image represented by moving image data for determining an angle of view that is output from the imaging means at a constant cycle. Moving image display control means for controlling the display device to display on a display screen, an occlusion area that is not captured in other images for each of a plurality of images that are imaged from different viewpoints by the imaging means and that are at least partially in common An occlusion area detecting means for detecting an image, and detecting one of the plurality of images as a target image, and detecting an image area corresponding to the occlusion area in the target image detected by the occlusion area detecting means from the moving image. By the occlusion correspondence area detection means and the above occlusion correspondence area detection means The occlusion corresponding region issued provided with occlusion corresponding region display control means for controlling the display device to display in said video.

  According to a first aspect of the present invention, there is provided an operation control method for an image pickup apparatus, wherein an image pickup means picks up an object and outputs image data obtained by the image pickup, and a moving image display control means outputs an image output from the image pickup means at a constant cycle. The display device is controlled to display the moving image represented by the moving image data for determining the angle on the display screen, and the occlusion area detecting unit is imaged by the imaging unit from different viewpoints, and at least a part is shared For each of the images, an occlusion area that does not appear in another image is detected, and the occlusion corresponding area detection means sets one of the plurality of images as the attention image, and the attention detected by the occlusion area detection means The image area corresponding to the occlusion area in the image is detected from the above video and the occlusion corresponding area display control is performed. Stage, the occlusion corresponding area detected by the occlusion corresponding area detection means controls the display device to display in said video.

  According to the first invention, an occlusion area (appears in the target image and appears in another image) of the target image that is one of a plurality of images obtained by imaging from different viewpoints (imaging positions). If an image area corresponding to (non-image area) exists in the moving image (through image), the image region (occlusion corresponding region) is displayed in the moving image. The occlusion correspondence area may be displayed by displaying a translucent image of a specific color superimposed on the moving picture, or the occlusion correspondence area in the moving picture may be displayed by surrounding it with a frame. When a moving image in which an occlusion-corresponding region appears is shot (recorded as image data), the image region that has been the occlusion region for the image of interest until now disappears from the occlusion region (appears in the image of interest, and It will be the image area that appears in other images). It can be said that the moving image in which the occlusion correspondence area appears is an image that is insufficient for constructing a three-dimensional image of the object. According to the first invention, it is possible to observe (visually recognize) an image that is insufficient for constructing a three-dimensional image of the object through the moving image by the occlusion corresponding area represented in the moving image. Image data necessary for constructing a three-dimensional image of an object can be efficiently acquired.

  In one embodiment, the imaging apparatus includes a target image determination unit that determines any one of the plurality of images as the target image, and the occlusion area detection unit is determined by the target image determination unit. The occlusion area for the image of interest is detected. An occlusion area for the determined image of interest is detected, and an occlusion corresponding area corresponding to the occlusion area is detected from the moving image.

  Preferably, the image-of-interest determination means sequentially determines each of the plurality of images as the image of interest in the order of the image captured first to the image captured later. For each of the plurality of images, an occlusion area is sequentially detected, and an occlusion corresponding area corresponding to the occlusion area is detected from the moving image.

  In one embodiment, the image-of-interest determination means, when the occlusion area in the image of interest is not detected by the occlusion area detection means, This is the image of interest. The occlusion area can be eliminated sequentially for each of a plurality of images picked up from different viewpoints.

  The occlusion area detecting means has only an image picked up from one viewpoint by the image pickup means (a state in which the image is taken and recorded in the memory card), and other images picked up with different viewpoints. When no image exists, the entire region of the image captured from the one viewpoint may be the occlusion region.

  In one embodiment, the occlusion area detection means sets a logical product area of a plurality of occlusion areas detected using the image of interest and each of the other images as an occlusion area for the image of interest.

  The larger the occlusion-corresponding area, the more the moving image represents an image (an image effective for constructing a three-dimensional image of the object) that is effective in reducing the occlusion area of the image of interest. Preferably, evaluation information display control means for controlling the display device so as to display evaluation information calculated according to the size of the occlusion correspondence area detected by the occlusion correspondence area detection means in the moving image is provided. . The evaluation information may be an image (for example, a bar display) or a numerical value. The size of the occlusion corresponding area in the moving image is displayed in an easy-to-understand manner based on the evaluation information. The photographing position where the occlusion area of the image of interest can be efficiently reduced can be easily shown to the photographer by the evaluation information.

  The image pickup apparatus according to the second invention picks up an image of a subject while moving the position along a predetermined rotation direction, moves the height position in a predetermined height direction after completing one round of image pickup, and again at the height position. Imaging the subject while moving the position along the rotation direction, imaging means for outputting image data representing a plurality of subject images obtained by imaging the subject, and outputting from the imaging means at regular intervals A moving image display control means for displaying the moving image represented by the moving image data for determining the angle of view on the display screen of the display device, a one-round imaging completion judging means for judging whether or not the one-round imaging along the rotation direction has been completed, And when the one-round imaging completion means determines that the one-round imaging has not been completed, a part of the previous frame image moved in the direction opposite to the predetermined rotation direction is placed on the moving image. Assist image display means for transparently displaying and semitransparently displaying on the moving image a part of the one-round start image that has been moved in the direction opposite to the predetermined height direction when it is determined that one-round imaging has been completed. I have.

  According to a second aspect of the present invention, there is provided a method for controlling the operation of an image pickup apparatus, wherein an object is imaged while moving a position along a predetermined rotation direction, and once a round of imaging is finished, the height position is moved in a predetermined height direction, An operation control method for an imaging apparatus that images the subject while moving the position along the rotation direction at a height position, and image data representing a plurality of subject images obtained by the imaging means capturing the subject. And the moving image display control means displays the moving image represented by the moving image data for determining the angle of view outputted from the imaging means at a fixed cycle on the display screen of the display device. It is determined whether or not the one-round imaging along the rotation direction is completed, and when the assist image display means determines that the one-round imaging is not completed by the one-round imaging completion means, the predetermined rotation is performed. A part of the previous frame image moved in the opposite direction is displayed semi-transparently on the moving image, and when it is determined that one-round imaging has been completed, the position is moved in the direction opposite to the predetermined height direction. A part of the image is displayed semi-transparently on the video.

  According to the second aspect of the present invention, when the round-trip imaging is not completed, the direction is opposite to the predetermined rotation direction (left direction if the predetermined rotation direction is clockwise, and right direction if the rotation is counterclockwise). A part of the previous frame image that has been moved to is displayed as an assist image on a moving image (through image) in a translucent manner. The position where the next imaging along the predetermined rotation direction should be performed can be determined using the assist image displayed semi-transparently. When it was determined that the round-trip shooting was completed, the position was moved in the direction opposite to the predetermined height direction (downward if the predetermined height direction is upward, and upward if it is downward) A part of the first circle starting image (the image obtained by the first imaging of the first round imaging) is displayed semitransparently on the moving image as an assist image. Using the assist image displayed semi-transparently, it is possible to determine the position (first imaging position of the next round imaging) where the next imaging along the height direction is to be performed. Since the entire subject can be imaged without omission while varying the imaging position (viewpoint) in the rotation direction and the height direction, it is possible to efficiently acquire image data effective for constructing a three-dimensional image of the object. it can.

  For example, the completion of the round imaging is determined when a moving image matching the image at the beginning of the round is taken.

  Preferably, the imaging apparatus displays a one-round imaging completion assist information indicating that one-round imaging has been completed on the moving image when the one-round imaging completion is determined by the one-round imaging completion determining unit. Information display control means is provided. Since it is displayed that the one-round imaging has been completed, useless imaging is prevented from being performed.

  In another embodiment, the variation determination means for determining whether the distance to the subject or the angle of view has changed, and the change when the distance or angle of view is determined to have changed by the variation determination means. Fluctuation assist information display control means for displaying on the moving image assist information for urging the imaging position to move in the canceling direction is further provided. It is possible to acquire a plurality of pieces of image data obtained by imaging a subject a plurality of times from substantially the same distance and having substantially the same angle of view.

  The present invention also provides a computer program for performing the operation control method of the imaging apparatus. A recording medium (such as a CD-ROM) storing such a program may be provided.

A state where an object is photographed from four different positions is shown. (A) to (D) show four subject images obtained by different imaging positions. This shows how the occlusion area is reduced. 2 is a block diagram showing an electrical configuration of a digital still camera. It is a flowchart which shows the operation | movement procedure of a digital still camera including an occlusion reduction process. It is a flowchart which shows the operation | movement procedure of a digital still camera including an occlusion reduction process. An example of a moving image (through image) is shown. An example of a movie is shown. (A) and (B) show a subject image and its occlusion area, respectively. An example of a movie is shown. (A) and (B) show subject images. A state in which an occlusion area is calculated from two subject images is shown. A state in which a logical product area of two occlusion areas is calculated is shown. An example of a movie is shown. An example of a movie is shown. (A) and (B) show a moving image with a bar display corresponding to the size of the occlusion corresponding area. It is a flowchart which shows the operation | movement procedure of the digital still camera in a modification. A state in which an object is photographed from a different position is shown. It is a flowchart which shows the operation | movement procedure of the digital still camera of 2nd Example. It is a flowchart which shows the operation | movement procedure of the digital still camera of 2nd Example. An example of a subject image is shown. An example of a movie is shown. An example of a movie is shown. An example of a movie is shown. An example of a movie is shown. An example of a movie is shown.

(First embodiment)
In this embodiment, a digital still camera 1 is used to image an object from various positions. FIG. 1 shows a state where two objects OB1 and OB2 are imaged by a digital still camera 1 from each of four different imaging positions (viewpoints) I to IV. 2A to 2D respectively show subject images g1 to g4 represented by image data acquired and acquired from the imaging positions I to IV shown in FIG.

  The objects OB1 and OB2 are both cubes and each have six faces. In FIG. 1, six surfaces (front, left side, back, right side, top, and bottom) of the object OB1 are indicated by alphabetic signs A to F. Similarly, six surfaces of the object OB2 are indicated by alphabetic symbols a to f.

  The subject images g1 to g4 shown in FIGS. 2A to 2D include target images ob1 and ob2 representing the objects OB1 and OB2 and a background image h representing the background H. 2A to 2D, the target images ob1 and ob2 are denoted by the same reference numerals as the above alphabetical codes indicating the specific surfaces of the objects OB1 and OB2 shown in FIG. Further, in FIGS. 2A to 2D, for easy understanding, different patterns are drawn on different surfaces and the same pattern is drawn on the same surface in each of the target images ob1 and ob2.

  The subject images g1 to g4 are all obtained by capturing the objects OB1 and OB2 from different viewpoints. For this reason, for example, although it appears in the subject image g1 (FIG. 2A) obtained by imaging from the first imaging position I, the subject image g2 (FIG. 2A) obtained by imaging from the second imaging position II. 2 (B)) includes an image portion that does not appear. Conversely, there is also an image portion that appears in the subject image g2 but does not appear in the subject image g1.

  In the first embodiment, one of a plurality of images obtained by imaging from different viewpoints is set as a target image, and the same image region as the image region appearing in the target image is different from the target image. As shown in FIG. 5, the user of the digital still camera 1 is assisted in shooting.

  An image area that appears in the target image and does not appear in another image (an image other than the target image (reference image)) is hereinafter referred to as an occlusion area OC. By obtaining a plurality of images by changing the position of the digital still camera 1, the occlusion area of the image of interest can be reduced and eliminated.

  FIG. 3 shows how the occlusion area is reduced when the subject image g1 (FIG. 2A) obtained by imaging from the first imaging position I is used as the image of interest. The upper part of FIG. 3 shows a state in which the occlusion area (shown by alternate long and short dashed lines) of the image of interest g1 is reduced. The lower part of FIG. 3 shows other subject images g2 and g3 used for reducing the occlusion area.

  When only the target image g1 is captured and the subject images g2 and g3 are not yet captured, the entire region of the target image g1 is the occlusion region OC10 (the target image g1 shown on the left in the upper part of FIG. 3).

  The subject image g2 obtained by imaging from the second imaging position II includes an image area corresponding to the target image g1. That is, an image area representing the front A of the target image ob1 (hereinafter referred to as the front area A, other areas are also expressed in the same manner), an upper area E, a front area a, an upper area e, and a background area h of the target image ob2. Appears in the image of interest g1 and also in the subject image g2. On the other hand, the left side region B of the target image ob1 and the left side region b of the target image ob2 in the target image g1 appear in the target image g1, but do not appear in the subject image g2. The image area that appears in the image of interest g1 and also in the subject image g2 is no longer an occlusion area and is therefore deleted from the occlusion area OC10. By capturing the subject image g2, only the left side region B of the target image ob1 and the left side region b of the target image ob2 in the target image g1 become the new occlusion region OC12 of the target image g1 (in the upper center of FIG. 3). The image of interest g1) shown.

  In the subject image g3 obtained by imaging from the third shooting position III, a left side area B of the target image ob1 and a left side area b of the target image ob2 appear. By capturing the subject image g3, the occlusion area OC12 is no longer an occlusion area. In this way, the subject image images g2 and g3 are captured, so that the occlusion area of the target image g1 disappears (target image g1 shown on the right in the upper part of FIG. 3).

  As will be described later, each of the images to be compared with the target image (subject images g2 and g3 in the above description) is also set as the target image, and the above-described processing is repeated. An occlusion area does not exist for a plurality of images captured at different imaging positions (viewpoints).

  The above-described reduction process (annihilation process) of the occlusion area is executed using, for example, the digital still camera 1. FIG. 4 is a block diagram showing an electrical configuration of the digital still camera 1.

  The overall operation of the digital still camera 1 is controlled by the CPU 2.

  The digital still camera 1 includes a solid-state imaging device (CCD, CMOS, etc.) 15, and an imaging lens 11, an aperture 12, an infrared cut filter 13 and an optical low-pass filter (OLPF) 14 are disposed in front of the solid-state imaging device 15. Is provided.

  The digital still camera 1 includes an operation device 3. The operation unit 3 includes a power button, a mode setting dial, a two-stroke type shutter release button, and the like. An operation signal output from the operation device 3 is input to the CPU 2. The mode set by the mode setting dial includes a shooting mode and a playback mode. The shooting modes further include a normal shooting mode and a multi-view shooting mode.

  Further, the digital still camera 1 is provided with a light emitting device 4 for strobe imaging and a light receiving device 5 for receiving reflected light of emitted light emitted from the light emitting device 4.

  When the power of the digital still camera 1 is turned on and the photographing mode is set, the light beam representing the subject image is passed through the imaging lens 11, the diaphragm 12, the infrared cut filter 13, and the optical low-pass filter 14. The light enters the light receiving surface of the solid-state image sensor 15. A subject image is formed on the light receiving surface of the solid-state image sensor 15, and an analog video signal representing the subject image is output from the solid-state image sensor 15. A subject is imaged at a constant cycle by the solid-state imaging device 15, and an analog video signal representing the subject image is output frame by frame at a constant cycle.

  An analog video signal representing the subject image output from the solid-state imaging device 15 is input to the analog signal processing device 16. The analog signal processing device 16 includes a correlated double sampling circuit, a signal amplifier, and the like, where correlated double sampling, signal amplification, and the like are performed. The analog video signal output from the analog signal processing device 16 is input to the analog / digital conversion circuit 17, where it is converted into digital image data. The digital image data is supplied to the memory control circuit 21 via the data bus 32. Under the control of the memory control circuit 21, the digital image data is temporarily recorded in the main memory 22.

  Digital image data is read from the main memory 22 and input to the digital signal processing circuit 23 via the data bus 31 or 32. The digital signal processing circuit 23 performs predetermined digital signal processing such as white balance adjustment and gamma correction. The data subjected to the digital signal processing in the digital signal processing circuit 23 is given to the display control circuit 28. When the display device 29 is controlled by the display control circuit 28, a subject image (moving image) (through image) is displayed on the display screen.

  When the shutter release button is pressed in the first stage, the lens 11 is driven by the lens driving circuit 6 to perform focusing. Luminance data is obtained in the digital signal processing circuit 23 based on the image data read from the main memory 22, and the luminance data is input to the integrating circuit 25 and integrated. Data representing the integrated value is given to the CPU 2 to calculate the exposure amount. The aperture of the aperture 12 is controlled by the aperture drive circuit 7 so that the calculated exposure amount is obtained, and the shutter speed of the solid-state image sensor 15 is controlled by the image sensor drive circuit 8.

  When the shutter release button is pressed in the second stage, the image data output from the analog / digital conversion circuit 17 is recorded in the main memory 22. The digital signal processing circuit 23 performs predetermined digital signal processing on the image data read from the main memory 22 as described above. Image data output from the digital signal processing circuit 23 is compressed in the compression / decompression processing circuit 24. The compressed image data is recorded on the memory card 27 under the control of the external memory control circuit 26.

  When the reproduction mode is set in the operation device 3, the compressed image data recorded in the memory card 27 is read out. The read compressed image data is expanded in the compression / expansion processing circuit 24 and then supplied to the display control circuit 28. The reproduced image is displayed on the display screen of the display device 29.

  5 and 6 are flowcharts showing the operation of the digital still camera 1 including the occlusion area reduction processing described above. The digital still camera when the multi-view shooting mode is set by the mode setting dial described above. The operation of the camera 1 is shown. The processing of the digital still camera 1 described below is basically executed by the CPU 2 or the digital signal processing circuit 23. Of course, dedicated hardware for executing processing by the digital still camera 1 described below, for example, an occlusion detection device for detecting an occlusion region, an occlusion correspondence region detection device for detecting an occlusion correspondence region, etc. The camera 1 may be provided separately. The same applies to the other embodiments.

  The digital still camera 1 includes the solid-state image sensor 15 as described above, and the solid-state image sensor 15 outputs an analog video signal representing a subject image for each frame at a constant period. The analog video signal is converted into digital image data, and the image represented by the image data obtained by imaging at a fixed period is continuously displayed on the display screen of the display device 29 (displaying a moving image for determining the angle of view). .

  FIG. 7 shows the digital still camera 1 from the back, and a moving image displayed on the display screen of the display device 29 when the digital still camera 1 is located at the imaging position I (see FIG. 1). An example of S1 is shown.

  On the display screen of the display device 29, a moving image S1 obtained by imaging from the first imaging position I (the same as the subject image g1 shown in FIG. 2A) is displayed and superimposed on the moving image S1. The occlusion correspondence area ROC10 (indicated by a two-dot chain line hatching) is displayed. As will be described later, the occlusion-corresponding region ROC refers to an image region in the moving image that appears in the occlusion region in the target image and also appears in the moving image. The state in which the acquisition (shooting) of image data has not yet been performed is a state in which the target image does not exist. The entire moving image in this state is handled as the occlusion corresponding region ROC, and therefore the entire moving image S1 displayed in the state before the first (first frame) shooting is handled as the occlusion corresponding region ROC10. The occlusion corresponding area ROC10 is superimposed on a moving image by a translucent image of a specific color (for example, red), for example, and is visually displayed on the display screen.

  Returning to FIG. 5, it is determined whether or not it is the first shooting (whether or not the first frame image has not yet been obtained) (step 41). For example, whether or not it is the first shooting is determined based on whether or not the shooting flag stored in the main memory 22 is turned on. If the photographed flag is off, it is determined that the first photographing is performed, and if the photographed flag is on, it is determined that the photographing is not the first photographing (YES or NO in step 41).

  When the shutter release button is pressed, the first shooting is performed (step 42). The above-mentioned photographed flag is turned on. For example, the image data representing the subject image g1 shown in FIG. 2A is stored in the memory card 27 as the first frame image.

  When the first frame image g1 is shot, two counters i and n are prepared in the main memory 22. The counter i is a counter of the frame number of the image to be treated as the image of interest, and the counter n is a counter representing the total number of shooting frames. When the first shooting is performed, 1 is assigned to each of the counters i and n. The first frame image g1 is the image of interest i (i = 1). The total number of photographed images is 1 (n = 1) (steps 43, 44, 45).

  When the first frame image g1 is set as the target image i, the entire area of the first frame image g1 is set as an occlusion area as an initial state (step 46). This is because there is no other image to be compared in a state where only the first frame image g1 is obtained. For example, an occlusion flag indicating that all pixels of the first frame image g1 are occlusion pixels is set. This indicates that the entire area of the first frame image g1 is the occlusion area OC10.

  The first frame image g1 may be displayed on the display screen of the display device 29, or an image (a translucent image of a predetermined color) showing the occlusion area OC10 may be displayed superimposed on the first frame image g1. Good (see subject image g1 shown on the left in the upper part of FIG. 3). In this case, the same display as the moving image S1 and the occlusion corresponding area ROC10 shown in FIG. 7 is displayed on the display screen of the display device 29.

  When a predetermined time elapses from the shooting of the first frame image g1 or when the shutter release button is pressed halfway, the moving image (through image) is displayed again on the display screen instead of displaying the first frame image g1. Is done.

  When the digital still camera 1 is moved and a moving image is captured at the imaging position after the movement, an image region (appearing in the occlusion region OC of the image of interest i and correspondingly appearing in the moving image) (Corresponding area) is obtained, and the obtained corresponding area is superimposed on the moving image and displayed (steps 52 and 53).

  FIG. 8 shows a moving image S2 (which is the same as the subject image g2 in FIG. 2B) when being imaged from the second imaging position II (see FIG. 1). It shows a state in which an image area that appears in the image of interest i, here the occlusion area OC10 (see FIG. 7) of the first frame image g1, and also appears in the moving image S2 is displayed. . The image area that appears in the occlusion area of the image of interest i and also appears in the moving image is the above-described occlusion correspondence area ROC. In FIG. 8, the occlusion corresponding region ROC21 is indicated by two-dot chain hatching.

  Assume that the image of interest i is the first frame image (subject image g1) (FIG. 2A) and the entire area is the occlusion area OC10. Further, it is assumed that the moving image S2 captured from the second imaging position II is the same as the subject image g2 in FIG. In this case, comparing FIG. 2A and FIG. 2B, the background region h, the upper surface region E of the target image ob1, the front region a and the upper surface region e of the target image b2, are the occlusion region of the target image i. It appears in OC10 (the entire subject image g1) and also appears in the moving image S2. Referring to FIG. 8, the background region h, the upper surface region E of the target image ob1, the front region a and the upper surface region e of the target image ob2 are set as new occlusion correspondence regions ROC21. The occlusion area OC10 in the target image i is compared with the moving picture S2, and the pixel corresponding to the pixel constituting the moving picture S2 is also present in the occlusion area OC10 in the target image i. The area is an occlusion corresponding area ROC21.

  In the front area A of the target image ob1, the area that does not appear in the target image i (subject image g1) (FIG. 2A) and appears in the moving image S2 (subject image g2) (FIG. 2B) Exists. This region (hereinafter referred to as a non-corresponding region) is shown surrounded by a frame indicated by reference numeral N1 in FIG. Regarding the front area A of the target image ob1, the area excluding the non-corresponding area N1 is set as the occlusion corresponding area ROC21. The non-corresponding area N1 is displayed with the color of the front A of the object OB1 imaged in the moving image S2.

  The occlusion correspondence area ROC indicates an image area appearing in common in the occlusion area and the moving image in the image of interest i in the moving image. It is indicated that the larger the occlusion correspondence area ROC, the more the image area corresponding to the occlusion area OC of the image of interest i includes. If an image including many image areas corresponding to the occlusion area OC of the image of interest i can be taken, the effect of reducing the occlusion area OC of the image of interest i is enhanced (see FIG. 3).

  The user tries to acquire the second frame image from the imaging position where the occlusion correspondence area ROC21 is displayed large while viewing the moving image for determining the angle of view displayed on the display screen of the display device 29.

  When the shutter release button is pressed, shooting is performed and image data is stored in the memory card 27 (step 54). The total number of shot frames n is incremented (step 55). n = 2.

  The target image i and the n-th frame image (images taken in step 54 described above) are used to determine the image region appearing in the n-th frame image among the occlusion regions of the target image i, and the remaining An occlusion area is determined (step 56). Further, the occlusion area of the n-th frame image is obtained using the image of interest i (step 57).

  Steps 56 and 57 are exemplified by a case where the subject image g1 shown in FIG. 2A is the image of interest i (first frame image) and the subject image g2 shown in FIG. 2B is the second frame image. The process will be described.

  FIG. 9A shows the remaining occlusion area OC12 except for the image area appearing also in the second frame image image g2 in the occlusion area OC10 (the entire subject image g1) of the first frame image g1. This is clearly shown in the frame image g1.

  Of the image areas included in the occlusion area OC10 of the first frame image g1, areas other than the left side area B of the target image ob1 and the left side area b of the target image ob2 appear in the second frame image g2. Referring to FIG. 9A, the image area appearing in the occlusion area OC10 of the first frame image g1 and also appearing in the second frame image image g2 is excluded from the occlusion area OC10. As a result, the left side area B of the target image ob1 and the left side area b of the target image ob2 are grasped as the remaining occlusion area OC12 of the first frame image g1 (step 56).

  FIG. 9B shows the occlusion area of the second frame image g2, and the occlusion area OC21 obtained in relation to the first frame image g1 is clearly shown in the second frame image g2. Yes.

  A part of the front area A of the target image ob1 in the second frame image g2 is an image area that does not appear in the first frame image g1 among the image areas that appear in the second frame image g2. Referring to FIG. 9B, a part of front area A of target image ob1 in second frame image g2 is obtained as occlusion area OC21 of second frame image g2 (step 57).

  In the occlusion area OC21 (FIG. 9B) for the second frame image g2 obtained by using the first frame image g1 and the second frame image g2 obtained in step 57, the second frame image g2 is the image of interest. It is used in the processing when it is said (details will be described later).

  Returning to FIG. 6, when the above-described processing using the image of interest i and the n-th frame image is completed, an update process of the occlusion area of the image of interest i is performed (step 58). In the update process of the occlusion area, a process is performed in which the occlusion area so far for the image of interest i is replaced with the remaining occlusion area obtained in step 56. That is, the occlusion area OC10 (FIG. 7), which was the entire area of the first frame image g1, is replaced with the occlusion area OC12 shown in FIG. 9 (A).

  Returning to FIG. 5, it is determined whether or not the image of interest i, here the first frame image g1, has an occlusion area (NO in step 41, step 47). Since the first frame image g1 has an occlusion area OC12 (FIG. 9A), there is again a corresponding area (occlusion corresponding area) between the first frame image g1 and a moving image for determining an angle of view (through image). Desired. The obtained occlusion correspondence area is superimposed on the moving image and displayed (YES in step 47, steps 52 and 53).

  FIG. 10 shows a moving image S3 when the digital still camera 1 is moved from the second imaging position II to the third imaging position III (see FIG. 1) and is imaged from the third imaging position III (FIG. 2). (C) is the same as the subject image g3) and is superimposed on the moving image S3 and appears in the current occlusion area OC12 (FIG. 9A) of the target image i, here the first frame image g1. The occlusion correspondence area ROC31 appearing correspondingly in the moving image S3 is displayed. The current occlusion area OC12 of the target image i (first frame image g1) is the left side area B of the target image ob1 and the left side area b of the target image ob2 (FIG. 9A). The left side area B of the target image ob1 and the left side area b of the target image ob2 appear in the moving image S3. In the moving image S3, the left side area B of the target image ob1 and the left side area b of the target image ob2 are displayed as the occlusion corresponding area ROC31.

  When shooting is performed by pressing the shutter release button, the total number of frames n is incremented (steps 54 and 55). Here, n = 3.

  As described above, the target image i and the n-th frame image are used to determine the image region that appears in the n-th frame image among the occlusion regions in the target image i, and the remaining occlusion regions are determined. (Step 56). Further, the occlusion area of the n-th frame image is obtained using the image of interest i (step 57). Here, the first frame image (subject image g1 in FIG. 2A) and the third frame image (subject image g3 in FIG. 2C) are used.

  Referring to FIG. 9A, the current occlusion area OC12 of the first frame image g1 is a left side area B of the target image ob1 and a left side area b of the target image ob2. Here, in the third frame image (subject image g3) (FIG. 2C), a left side region B of the target image ob1 and a left side region b of the target image ob2 appear. Therefore, by photographing the third frame image g3, as shown in FIG. 11A, no remaining occlusion area exists in the first frame image g1 (step 56).

  Referring to FIGS. 2A and 2C, all image areas of the third frame image g3 appear in the first frame image g1. For this reason, when the first frame image g1 is used, the third frame image g3 has no occlusion area. As shown in FIG. 11B, it is determined in step 57 that there is no occlusion area for the third frame image g3.

  In the process of updating the occlusion area of the image of interest i (step 58), the previous occlusion area of the image of interest i is replaced with the remaining occlusion area obtained in step 56. It can be understood that all the occlusion areas OC12 (FIG. 9A) of the first frame image g1 have disappeared (FIG. 11A).

  In determining whether or not the image of interest i has an occlusion area, if it is determined that there is no occlusion area in the first frame image g1 (NO in step 47), the counter i is incremented, and the second frame image, here Then, the process proceeds to the process of using the subject image g2 (FIG. 2B) as the image of interest (step 48, i = 2).

  In the above-described processing for the first frame image g1, the first frame image g1, the second frame image g2, and the third frame image g3 are photographed. In the process using the second frame image g2 as the image of interest, first, a process using an image already obtained by the processes so far is performed.

  Using the image of interest i, here the second frame image g2, and the images of i + 1 to n frames, here the third frame image g3 (i + 1 = 3, n = 3), the image of interest i (second frame image) The occlusion area of g2) is determined (step 50).

  FIG. 12 shows how the occlusion area OC23 of the second frame image g2 is obtained using the second frame image g2 (target image i) and the third frame image g3.

  Of the image areas appearing in the second frame image g2, the front area A of the target image ob1 and the front area a of the target image ob2 do not appear in the third frame image g3. The front area A of the target image ob1 and the front area a of the target image ob2 in the second frame image g2 are obtained by comparing the second frame image g2 and the third frame image g3. The occlusion area OC23. If an image of the fourth frame, the fifth frame, etc. is taken, the occlusion area of the second frame image g2 obtained by comparing the second frame image g2 with these images is also obtained in step 50. .

  A logical product of the occlusion area OC23 obtained in step 50 and the occlusion area already obtained for the target image i is calculated, and the occlusion area is updated (step 51). With respect to the target image i, here the second frame image g2, the occlusion area OC21 of the second frame image g2 is obtained between the first frame image g1 and the first frame image g1 in the above-described processing in the first frame image g1. FIG. 9B). The logical product of the occlusion area OC23 obtained in step 50 and the already calculated occlusion area OC21 is calculated. If a plurality of occlusion areas of the second frame image g2 are obtained in step 50, the logical product is also taken for these occlusion areas OC.

  FIG. 13 shows a state in which a logical product of the occlusion area OC23 of the second frame image g2 and the occlusion area OC21 of the second frame image g2 is obtained.

  By taking the logical product of the two occlusion areas OC23, 21 for the second frame image g2, the occlusion area OC213 appearing in common in both of the two occlusion areas OC23, OC21 is the remaining for the second frame image g2. It becomes an occlusion area. The occlusion area for the second frame image g2 is updated to the occlusion area OC213 (step 51).

  A corresponding area (occlusion corresponding area) between the occlusion area of the image of interest i and a moving image is obtained, and the obtained occlusion corresponding area is displayed (step 52). FIG. 14 shows the subject of the moving image S4 (FIG. 2D) captured from the fourth imaging position IV (see FIG. 1) when the second frame image g2 having the occlusion area OC213 is the image of interest i. The same as image g4). Overlaid with the moving image S4, the image i of interest (here, the image region (occlusion corresponding region ROC42) that appears in the occlusion region OC213 (see FIG. 13) of the second frame image g2 and also appears in the moving image S4) is displayed. (Step 53).

  When shooting from the fourth imaging position IV is performed and the fourth frame image g4 is acquired, the counter n is incremented (n = 4), and in the image of interest i (here, the second frame image g2). The occlusion area OC213 is eliminated (NO in steps 54, 55, 56, 58, and step 47). The counter i is incremented (i = 3), and the process proceeds to processing for setting the third frame image g3 as the image of interest (step 48).

  In this way, each of all photographed images is set as a target image, and the occlusion area of the target image is gradually reduced and deleted (see FIG. 3).

  When there is no occlusion area for all of the captured images, the number of counters i and n is equal (i = n). As a result, the above-described series of processing ends (YES in step 49).

  A focused image without an occlusion area means that all of the image areas appearing in the focused image also appear in other images acquired from an imaging position different from the imaging position of the focused image. Corresponding image areas are included in at least two images having different imaging positions (viewpoints). It is easy to generate a three-dimensional image from such a plurality of images.

  As described above, in the moving image for determining the angle of view displayed on the display screen of the display device 29 of the digital still camera 1, an image area that can reduce the occlusion area OC of the image of interest i is occlusion-compatible. It is explicitly displayed as a region ROC. It can be said that the larger the displayed occlusion correspondence area ROC is, the more effective the reduction or erasure of the occlusion area OC of the image of interest i is, and an image necessary for the 3D image (an image lacking in the 3D image). The occlusion-corresponding region ROC assists the user with a preferable imaging direction for constructing a three-dimensional image. It can be easily recognized by the occlusion corresponding area ROC represented in the moving image that the current imaging direction is appropriate (or not valid).

  In photographing a plurality of objects OB1 and OB2 from different imaging positions using the digital still camera 1 (see FIG. 1), the distance between the digital still camera 1 and the objects OB1 and OB2 is almost the same. Preferably they are the same. When there is a large variation in the distance between the digital still camera 1 and the objects OB1 and OB2 (for example, when a shooting distance that is different from the shooting distance measured when shooting the previous frame image by a predetermined distance is measured) ), As shown in FIG. 15, the assist information m1 related to the distance such as “Please move the camera closer!” (Or “Please move away!”) May be displayed over the moving image S5. The distance (distance variation) between the digital still camera 1 and the objects OB1 and OB2 may be determined by a distance measuring device (not shown) provided in the digital still camera 1 or in the subject image. The determination may be made based on the sizes of the target images ob1 and ob2.

  16A, 16B, and 17 show a modification.

  16A and 16B show the digital still camera 1 from the back.

  As described above, the moving image for determining the angle of view (through image) is displayed on the display screen of the digital still camera 1, and the corresponding region (occlusion) in the moving image corresponding to the occlusion region OC of the image of interest i. Corresponding region) ROC is obtained and displayed superimposed on the moving image. In this modification, the area (size) of the occlusion corresponding region ROC in the moving image is obtained, and the display bar B1 having a length corresponding to the obtained area is superimposed on the moving image and displayed. FIGS. 16 (A) and 16 (B) show how the display bar B1 having a length corresponding to the area of the occlusion region ROC is superimposed on the moving images S6 and S7. When the area of the occlusion corresponding region ROC varies by changing the position of the digital still camera 1, the length of the display bar B1 varies accordingly. As the area of the occlusion correspondence region ROC is larger, the longer display bar 1 is displayed. When the area of the occlusion correspondence area ROC is reduced, the length of the display bar B1 is reduced. The size of the occlusion corresponding area ROC can be easily recognized by the length of the display bar B1, and the position (viewpoint) of the digital still camera 1 can be adjusted to a preferable position by reducing the occlusion area.

  FIG. 17 is a flowchart showing processing of the digital still camera 1 when displaying the display bar B1, and corresponds to the flowchart shown in FIG. The same processes as those in the flowchart shown in FIG.

  First, a memory area for storing the display area corresponding to the occlusion is provided in the main memory 22, and an initial value (0) is stored therein (step 61).

  As described above, the occlusion area ROC is obtained using the occlusion area OC and the moving image of the image of interest i (step 52).

  The area (or the number of pixels or the area ratio) of the obtained occlusion corresponding area ROC is calculated and stored in the memory area. The length of the display bar B1 corresponding to the area of the occlusion corresponding area ROC stored in the memory area is calculated (step 62). The display bar B1 having the calculated length is superimposed on the moving images S5 and S6 and displayed on the display screen (step 63, FIG. 16 (A), FIG. 16 (B)).

  Until there is a photographing operation (pressing the shutter release button) (YES in step 64, step 54), the display of the moving image, the display of the occlusion correspondence area ROC, and the display of the display bar B1 are repeated (NO in step 64).

  In place of or in addition to the display of the display bar B1, a numerical value representing the area ratio of the calculated area (the ratio of the number of pixels constituting the occlusion corresponding region ROC to the total number of pixels of one frame of the moving image) is superimposed on the moving image. You may display together.

(Second embodiment)
18 to 26 show a second embodiment.

  In the second embodiment, when the digital still camera 1 is moved in order to photograph an object from different viewpoints, assist information regarding the moving direction and moving distance is displayed.

  FIG. 18 shows a state in which the object (bottle) OB3 is imaged from various positions p (1) to p (m) using the digital still camera 1.

  In the second embodiment, two moving directions and moving amounts for moving the digital still camera 1 are preset. The amount of movement is determined in advance, and the two movement directions are set using the operation device 3, for example. One of the two movement directions to be set (first movement direction) is whether the object OB3 is photographed clockwise (counterclockwise) or counterclockwise (clockwise) (digital still, Whether to move the camera 1). The other one movement direction (second movement direction) is a direction orthogonal to the first movement direction, and after photographing the object OB3 for one turn in the first movement direction (clockwise or counterclockwise), It defines whether the digital still camera 1 is moved upward or downward. After the movement in the second movement direction, photographing is performed a plurality of times while moving the digital still camera 1 in the first movement direction again. As shown in FIG. 18, for example, the digital still camera 1 is moved clockwise, and the object OB3 of the first frame, the second frame,. When shooting for one round is completed, the digital still camera 1 is moved upward, and right-turn shooting for the second round starting from the (N + 1) th frame is performed again at that height position. When it is determined that the entire object OB3 that can be visually recognized by the M-th frame is completely captured, the processing by the digital still camera 1 ends.

  In the following description, the first moving direction will be described as a clockwise direction, and the second moving direction will be described as an upward direction.

  19 and 20 are flowcharts showing the processing of the digital still camera 1 of the second embodiment.

  First, the controller 3 is used to set two different directions (first moving direction and second moving direction) in which the digital still camera 1 should be moved (step 71). Here, as described above, a case where the clockwise direction is set as the first movement direction and the upward direction is set as the second movement direction will be described.

  It is determined whether or not this is the first shooting (first frame shooting) (step 72).

  In the case of the first shooting, no special processing is performed, and image data including the target image ob3 displayed as a moving image on the display device 29 of the digital still camera 1 is displayed in response to pressing of the shutter release button. It is recorded on the memory card 27 (step 83).

  FIG. 21 shows the digital still camera 1 from the back side, and shows a state in which the first frame image n1 obtained by imaging from the imaging position p (1) is displayed on the display screen of the display device 29. ing. The first frame image n1 and other images to be described later include a target image ob3 representing the target object OB3.

  An image obtained by the first imaging of each lap is stored in the main memory 22 as a lap start image. Since the first frame image n1 is an image obtained by the first imaging of the first round, it is recorded in the main memory 22 as the first picture of the round (step 84).

  A flag for starting the first round is set in the main memory 22. In the initial state, the first cycle start flag is turned off (step 85). As will be described later, the one-round start flag is turned on when photographing for one round is completed. Details of the process using the one-round start flag will be described later.

  If there is no shooting completion operation (NO in step 93), the process proceeds to the next frame (second frame) (NO in step 72). The digital still still 1 is moved in the first movement direction, here in the clockwise direction. With reference to FIG. 18, the imaging position is changed from p (1) to p (2).

  It is determined whether or not the first cycle start flag is turned on (step 73). When only the first frame image n1 has been acquired, the first round start flag is turned off as described above (step 85). In this case, it is determined that the current movement direction of the digital still camera 1 is the first movement direction (clockwise direction) (NO in step 73, step 75).

  The previous frame image that has been moved (shifted) according to the determined current moving direction is displayed as an assist image on a moving image for determining the angle of view (through image) in a translucent manner (step 76).

  FIG. 22 shows the moving image Sn2 displayed on the display screen before the second frame image n2 is acquired from the imaging position p (2).

  Superimposed on the moving picture Sn2, the previous frame image, here the first frame image n1 (see FIG. 21) is displayed in a semi-transparent manner in a state where the position is moved to the left. The front frame image n1 that has been moved and displayed semi-transparently is the assist image An1.

  When the movement direction is clockwise, the assist image moves the position of the previous frame image to the left by a predetermined amount of movement (for example, a movement amount of about 1/3 of the horizontal width of the previous frame image) and displays it translucently. It is what. If the movement direction is counterclockwise, the position movement direction of the assist image is rightward. As shown in FIG. 22, the first frame image An1 as the assist image is displayed in a semi-transparent manner while being shifted in the left direction in the moving image Sn2. A part of the first frame image n1 (about 2/3 from the right end of the first frame image) is displayed translucently in the moving image Sn2.

  It is determined whether the position of the moving image matches the position of the assist image (step 77). Here, it is determined whether or not the position of the moving image Sn2 matches the position of the assist image An1 (first frame image n1 after the position movement). For example, based on a plurality of feature points obtained from a translucently displayed assist image (a part of the first frame image n1) and the positions of corresponding points corresponding to the feature points obtained from the movie, It is determined whether the position matches the position of the assist image. Of course, the match between the position of the moving image and the position of the assist image may be determined according to the position of the main subject image (for example, the target image ob3) appearing in common in the assist image and the moving image. The determination of whether the position of the moving image and the position of the assist image match is made by determining whether or not the image position after movement (a part of the previous frame image appears in the moving image and matches the set moving direction and the predetermined moving amount ( That is, it is determined whether or not imaging from the next imaging position is being performed.

  If the position of the moving image Sn2 and the position of the assist image An1 (the previous frame image n1 after the position movement) do not match, assist information for matching the angle of view and the position is displayed on the display screen (NO in step 78). Step 79).

  Referring to FIG. 22, for example, when the assist image An1 is displayed on the left and the moving image Sn2 is displayed on the right, the right arrow r1 and the character string m2 “Please align the camera position” It is displayed as information. As described above, the direction and magnitude of the position shift of the moving picture Sn2 are the positions of the plurality of feature points obtained from the assist image and the positions of the corresponding points corresponding to the feature points obtained from the moving picture (ie, movement vectors). It is requested from. When the digital still camera 1 is moved in the right direction according to the right-pointing arrow r1, the target image ob3 in the moving picture Sn2 moves relatively in the left direction. By moving the digital still camera 1 in the direction according to the assist information (right arrow r1), the position of the moving image Sn2 and the position of the assist image An1 are matched.

  As described in the first embodiment, when it is detected that there is a large variation in the distance between the digital still camera 1 and the object OB3, “Please bring the camera closer!” (Or The assist information m1 regarding the distance (view angle) such as “Please keep away!” Is displayed so as to overlap the moving image Sn2 (see FIG. 15).

  If it is determined that the position of the moving image Sn2 matches the position of the assist image An1 (YES in step 78), as shown in FIG. 23, the assist information r1, m2 disappears from the display screen. The movie currently displayed on the display screen is compared with the captured image represented by the acquired image data stored in the memory card 27, and the displayed movie is already captured. Is determined (steps 80 and 81).

  Only the first frame image n1 (FIG. 21) has been shot, and the moving image Sn2 (FIG. 23) is captured by imaging from the imaging position p (2) different from the imaging position p (1) of the first frame image. In this case, it is determined that the movie Sn2 has not been shot (not shot) (NO in step 81). The second shooting of the object OB3 is performed on the condition that the first cycle start flag is turned off (NO in step 82, step 86). The second frame image n2 is stored in the memory card 27. Further, when the first round start flag is turned on, it is turned off (step 87).

  Similarly, a plurality of shootings are continued while moving clockwise around the object OB3 (see FIG. 18).

  FIG. 24 shows a moving image SnN that is displayed when N times of clockwise rotations in the first round are performed and the first imaging position p (N) is returned.

  When the digital still camera 1 makes a round while moving clockwise around the object OB3, the moving picture SnN at that time substantially coincides with the photographed image (first frame image n1). It is determined that the moving image SnN has already been photographed (YES in step 80 and step 81).

  It is determined whether or not the moving picture SnN matches the circle start image stored in the main memory 22 described above, here the first frame image n1 (step 88). When it is determined that the moving image SnN and the start-of-round image (first frame image n1) match, assist information m3 indicating completion of the first-round shooting is displayed on the display screen (FIG. 24, YES in step 89, step 90).

  When it is determined that the one-round shooting is completed, the above-mentioned one-round start flag is turned on (step 91). When the first round start flag is turned on, it is recognized that the current movement direction is the second movement direction, that is, the upward direction in the process for the next N + 1 frame (YES in step 73, step 74). .

  FIG. 25 shows the moving image Sn (N + 1) when it is determined that the current movement direction is the upward direction.

  If it is determined that the determined current movement direction is the second movement direction, in this case, the upward direction, not the previous frame image, but the start-of-round image (first frame image n1) is represented in the video as an assist image (step) 76). Here, since the current movement direction is the upward direction, the start-of-round image AnN as the assist image is moved downward by a predetermined amount of movement (for example, about 1/3 of the vertical width of the captured image). It will be a thing. Further, an up arrow r2 appears on the display screen as assist information regarding the moving direction of the digital still camera 1.

  At the time before the first first frame after being moved upward, as described above, the first cycle start flag is turned on. In this case, the image obtained by the first shooting after being moved upward (the first image in the second round) is newly stored in the main memory 22 as the second round-start image. (YES at step 82, steps 83, 84). Proceed to the second round of shooting (shooting multiple times while moving clockwise).

  When a plurality of clockwise shootings, upward movements, and a plurality of further clockwise shootings are repeated, everything except the bottom surface of the object OB3 is shot. If the moving image SnM has already been photographed even though the one-round photographing has not been completed, it means that photographing of all areas of the object OB3 that can be visually recognized has been finished. In this case, assist information m4 indicating to the user that shooting is not required is displayed on the display screen, and a series of processing ends (YES in step 81, NO in step 88, NO in step 89, step 92, FIG. 26).

  Since the assist images An1, An2,... Are translucently displayed on the display screen, the photographer can easily grasp the position and moving distance at which the next photographing should be performed. In addition, arrows A1 and A2 indicating the direction in which the digital still camera 1 should be moved are also displayed, and assist information m2 that prompts the movement is also displayed, so that the photographer can easily assist in the direction of movement. . Assist information m2 indicating that the one-round shooting has been completed is displayed, and assist information m3 indicating that the shooting of all areas has been completed is also displayed, so that unnecessary shooting can be avoided.

  In the second embodiment described above, an example has been described in which the digital still camera 1 is moved around the object OB3 while moving around it, but of course the digital still camera 1 is the center. The second embodiment can be applied even when shooting the surrounding scenery.

1 Digital still camera 2 CPU
3 Controller
15 Solid-state image sensor
22 Main memory
23 Digital signal processing circuit
27 Memory card
28 Display control circuit
29 Display device

Claims (15)

Imaging means for imaging an object and outputting image data obtained by imaging;
Moving image display control means for displaying a moving image represented by moving image data for determining an angle of view output from the imaging means at a fixed period on a display screen of a display device;
An occlusion area detecting means for detecting an occlusion area not appearing in another image for each of a plurality of images taken from different viewpoints by the imaging means and having at least a part in common;
An occlusion corresponding area detecting means for detecting an image area corresponding to an occlusion area in the noted image detected by the occlusion area detecting means as one of the plurality of images as a noticed image; Occlusion correspondence area display control means for displaying the occlusion correspondence area detected by the occlusion correspondence area detection means in the moving image,
An imaging apparatus comprising: A focused image determining means for determining any one of the plurality of images as the focused image;
The occlusion area detection means is:
The occlusion area for the target image determined by the target image determination means is detected.
The imaging device according to claim 1. The focused image determining means is
Each of the plurality of images is sequentially determined as the image of interest in the order of an image captured first to an image captured later.
The imaging device according to claim 2. The focused image determining means is
When the occlusion region in the image of interest is not detected by the occlusion region detection means, the image taken next to the image that is the image of interest is the image of interest.
The imaging device according to claim 3. The occlusion area detection means is:
If there is only an image captured from one viewpoint by the imaging means and there is no other image captured from a different viewpoint, the entire area of the image captured from the one viewpoint is the occlusion area. Is,
The imaging device according to claim 1. The occlusion area detection means is:
A logical product area of a plurality of occlusion areas detected using the image of interest and each of the other images is used as an occlusion area for the image of interest.
The imaging device according to any one of claims 1 to 5. Evaluation information display control means for controlling the display device so as to display the evaluation information calculated according to the size of the occlusion correspondence area detected by the occlusion correspondence area detection means in the moving image,
The imaging device according to any one of claims 1 to 6. The subject is imaged while moving the position along a predetermined rotation direction, and once the round-trip imaging is completed, the height position is moved in a predetermined height direction, and the position movement along the rotation direction is performed again at the height position. However, in an imaging device for imaging the subject,
Imaging means for outputting image data representing a plurality of subject images obtained by imaging the subject;
Moving image display control means for displaying a moving image represented by moving image data for determining an angle of view output from the imaging means at a fixed period on a display screen of a display device;
A round-trip imaging completion judging means for judging whether round-trip imaging along the round direction is completed, and a reverse direction to the predetermined round-trip direction when the round-trip imaging completion means judges that round trip imaging is not completed A part of the previous frame image moved to the semi-transparent position is displayed on the moving image, and when it is determined that one-round imaging has been completed, the first circle starting image moved in the direction opposite to the predetermined height direction. Assist image display means for semi-transparently displaying a part of the image on the video,
An imaging apparatus comprising: The one-round imaging completion judging means is for judging completion of one-round imaging when a moving image matching the image at the start of the first round is taken.
The imaging device according to claim 8. A round-trip imaging completion assist information display control unit for displaying round-trip imaging completion assist information indicating completion of round-trip imaging on the video when the round-trip imaging completion judging section determines completion of round-trip imaging;
The imaging device according to claim 8 or 9. Fluctuation determining means for determining whether or not the distance to the subject or the angle of view has changed, and imaging in a direction that eliminates the fluctuation when the distance or angle of view is determined to have changed by the fluctuation determining means Fluctuation assist information display control means for displaying assist information for prompting position movement on the moving image,
The imaging device according to any one of claims 8 to 10. The imaging means images the object and outputs image data obtained by imaging,
The moving image display control means controls the display device so as to display the moving image represented by the moving image data for determining the angle of view output from the imaging means at a constant cycle on the display screen
The occlusion area detecting means detects an occlusion area that is not captured in other images for each of a plurality of images that are imaged from different viewpoints by the imaging means and at least partially in common,
An occlusion correspondence area detecting means detects one of the plurality of images as a noticed image, and detects an image area corresponding to the occlusion area in the noticed image detected by the occlusion area detecting means from the moving image;
The occlusion correspondence area display control means controls the display device to display the occlusion correspondence area detected by the occlusion correspondence area detection means in the moving image;
An operation control method for an imaging apparatus. The subject is imaged while moving the position along a predetermined rotation direction, and once the round-trip imaging is completed, the height position is moved in a predetermined height direction, and the position movement along the rotation direction is performed again at the height position. However, in the operation control method of the imaging device for imaging the subject,
The imaging means outputs image data representing a plurality of subject images obtained by imaging the subject,
The moving image display control means displays on the display screen of the display device a moving image represented by moving image data for determining an angle of view that is output from the imaging means at a constant cycle.
A round-trip imaging completion judging means judges whether round-trip imaging along the rotation direction is completed,
When the assist image display means determines that the round imaging has not been completed by the round imaging completion means, a part of the previous frame image moved in the direction opposite to the predetermined rotation direction is displayed on the moving image. Semi-transparent display is performed, and when it is determined that one-round imaging has been completed, a part of the first image that has been moved in the direction opposite to the predetermined height direction is displayed semi-transparently on the moving image.
An operation control method for an imaging apparatus. An image pickup means for picking up an image of an object and outputting image data obtained by the image pickup, and a moving picture represented by moving picture data for determining an angle of view outputted from the image pickup means at a fixed cycle is displayed on a display screen of a display A program for controlling a computer of an image pickup apparatus including a moving image display control means,
For each of a plurality of images captured from different viewpoints by the imaging means and having at least a part in common, an occlusion region that does not appear in other images is detected,
One of the plurality of images as a target image, and an image region corresponding to the occlusion region in the detected target image is detected from the moving image,
A program for controlling the computer of the imaging apparatus so that the detected occlusion corresponding area is displayed in the moving image. The subject is imaged while moving the position along a predetermined rotation direction, and once the round-trip imaging is completed, the height position is moved in a predetermined height direction, and the position movement along the rotation direction is performed again at the height position. A program for controlling a computer of an imaging apparatus for imaging the subject,
The imaging device is represented by an imaging unit that outputs image data representing a plurality of subject images obtained by imaging the subject, and moving image data for determining an angle of view that is output from the imaging unit at a constant cycle. Provided with a moving image display control means for displaying a moving image on a display screen of a display device;
Determine whether or not the round imaging along the above direction is complete,
If it is determined that the round imaging has not been completed, a part of the previous frame image moved in the direction opposite to the predetermined rotation direction is displayed semi-transparently on the moving image, and it is determined that the round imaging has been completed. A program for controlling the computer of the imaging apparatus so that a part of the first image that has been moved in the direction opposite to the predetermined height direction is displayed semi-transparently on the moving image.

Images