JP5620871B2 - Panorama image data generation device - Google Patents

Description

  The present invention relates to a panorama image data generation apparatus that generates panorama image data for displaying a panorama image.

  As a technique related to a panoramic image, a technique for capturing a panoramic image by taking an omnidirectional image from a predetermined position inside a building is known (see Patent Document 1).

JP 2009-235696 A

  In this technique, since the omnidirectional image is processed and used, the omnidirectional image cannot be used for construction recording or the like. Therefore, it is desired to generate a panoramic image without processing the original image.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a panoramic image data generation device capable of generating a suitable panoramic image without processing the original image. .

In order to solve the above problems, the panorama image data generation device of the present invention uses the captured image data relating to a plurality of captured images to compare the panorama image with another image different from the panorama image. A panoramic image data generating apparatus for generating panoramic image data relating to a panoramic image data storage unit storing a plurality of captured image data, and reading out the stored plurality of captured image data and arranging them in a virtual space by mapping the plurality of plate model constituting the map includes a panorama image data generator for generating the panoramic image data regarding the panoramic image, and the image data output unit for outputting the generated the panorama image data, the , So that the plate model overlaps the other adjacent plate models Serial is arranged in the virtual space, the panoramic image data and based on other image data related to the other image, the polymerization image data generation that forms a polymerization image data relating to the polymerization image superposed with the panoramic image and the other image The superimposed image data generation unit translucently processes the panoramic image data in which only the viewpoint side is drawn with respect to the overlapped portions of the plurality of captured image data, and the other image data and the translucent process Based on the panorama image data thus generated, superimposing the semi-transparent panorama image in front of the other image, to generate superimposed image data related to the superimposed image for comparing the other image and the panoramic image The image data output unit outputs the generated superimposed image data .

  According to such a configuration, since the adjacent captured images overlap in the panoramic image, even if the captured image is moved with respect to the adjacent captured images, no blank portion is generated between the captured images. Therefore, when there is a deviation in the shooting direction, the continuity of the shooting target can be maintained by moving the shot image without processing the shot image.

Further, the panorama image data generating apparatus further comprises a CG image data storage CG image data relates to a three-dimensional CG image is the other image is stored, the polymerizable image data generating unit, the CG image data and the translucent process Based on the obtained panoramic image data, the superposed image for superimposing the three-dimensional CG image and the panoramic image by superimposing the translucent panoramic image in front of the three-dimensional CG image The structure which produces | generates data may be sufficient.

The panorama image data generation unit includes a first panorama image data related to the first panorama image that is the other image, and a second panorama image data that relates to the second panorama image that is the panorama image . of the panoramic image data to generate the polymerization image data generation unit may semitransparent processing the second panorama image data drawn only toward the viewpoint with respect to the overlapped portion of the plurality of captured image data, Based on the first panorama image data and the second panorama image data subjected to the translucent process, the second panorama image that is translucent is superimposed on the front of the first panorama image . a configuration that forms a polymerization image data relating to the polymerization picture for comparing said second panorama image as one of a panoramic image It may be.

According to the present invention, a suitable panoramic image can be generated without modifying the original image, and the panoramic image and another image different from the panoramic image can be compared with the superimposed image. .

1 is a block diagram illustrating a panoramic image display system according to an embodiment of the present invention. It is a schematic diagram for demonstrating arrangement | positioning of a board model. It is a figure for demonstrating the overlap of a board model, (a) is the figure seen from the panorama map center, (b) is the figure which looked at a part of panorama map from the top, (c) is one board model It is a figure for demonstrating the part which has overlapped in and the part which has not overlapped. It is a figure for demonstrating the 1st operation example of the panorama image display system which concerns on embodiment of this invention, (a) shows the panorama image before correction | amendment, (b) shows the panorama image after correction | amendment. It is a figure for demonstrating the 2nd example of operation | movement of the panoramic image display system which concerns on embodiment of this invention, (a) is a CG image, (b) is a panoramic image, (c) is CG image and semi-transparent. A superposed image composed of a panoramic image is shown. It is a figure for demonstrating the 3rd operation example of the panorama image display system which concerns on embodiment of this invention, (a) is a 1st panorama image, (b) is a 2nd panorama image, (c) is The superimposition image which consists of a 1st panoramic image and a translucent 2nd panoramic image is shown.

  Hereinafter, a case where the panorama image data generation device according to the embodiment of the present invention is applied to a panorama image display inside a building will be described as an example with reference to the drawings as appropriate. Similar parts are denoted by the same reference numerals, and redundant description is omitted.

  As shown in FIG. 1, a panoramic image display system 1 according to an embodiment of the present invention is a system that generates and displays an omnidirectional panoramic image inside a building, and includes a photographing device 10, an input device 20, A panoramic image data generation device 30 and a display device 40 are provided.

<Photographing device 10>
The photographing apparatus 10 includes a photographing unit 11 that is a digital camera, a pan / tilt driving unit 12 that drives the photographing unit 11 in a pan direction and a tilt direction, and a control that controls driving of the photographing unit 11 and the pan / tilt driving unit 12. Unit 13.

The control unit 13 controls the pan / tilt drive unit 12 to drive the shooting unit 11 provided inside the building in a desired direction, and controls the shooting unit 11 to drive and control the shooting unit 11 in such a state. The captured image data related to the captured image is output to the panoramic image data generation device 30. Note that the shooting data of the shooting unit 11 may be stored in a storage medium such as an SD card and read by the panorama image data generation device 30.
Here, the photographing unit 11 is configured so that adjacent photographed images overlap in all the vertical and horizontal directions of the photographing unit 11 (up and down 360 degrees, right and left 360 degrees, except for right under the photographing unit 11). Photographing is performed for each predetermined pitch, and photographed image data regarding the photographed image in all directions is output to the control unit 13.
In addition, the control unit 13 assigns an ID based on the shooting order (that is, the shooting direction) to the shot image data, and outputs the shot image data to which the ID is given to the panorama image data generation device 30.

<Input device 20>
The input device 20 includes a keyboard, a mouse, and the like, and outputs a panorama image viewpoint position instruction, a panorama image correction instruction, a panorama image translucent processing instruction, and the like to the panorama image data generation apparatus 30 in accordance with a user operation. .

<Panorama Image Data Generation Device 30>
The panorama image data generation device 30 is composed of a CPU, a ROM, a RAM, an input / output circuit, and the like. As a functional unit, a captured image data storage unit 31, a panorama image data generation unit 32, a storage unit 33, and a superposition An image data generation unit 34, an image data output unit 35, and a panorama image correction unit 36 are provided.

<< Photographed Image Data Storage Unit 31 >>
The captured image data storage unit 31 receives and stores a plurality of captured image data related to a plurality of captured images output from the control unit 13.

<< Panorama Image Data Generation Unit 32 >>
The panoramic image data generation unit 32 reads a plurality of captured image data output from the captured image data storage unit 31 and generates panoramic image data related to the panoramic image based on the read captured image.
Specifically, the panorama image data generation unit 32 reads a panorama map 50 (see FIG. 2) in which a plurality of plate models 51 are arranged in a virtual space, which is stored in advance in the storage unit 33, and the plate model of the panorama map 50. Panorama image data is generated by mapping the captured image data to 51.

  As shown in FIG. 2, the panoramic map 50 is configured by arranging a plurality of plate models 51 in a spherical shape. The center of the panorama map 50 (a sphere composed of a plurality of plate models 51) corresponds to the installation position of the imaging unit 11, and each plate model 51 corresponds to the imaging range of each captured image. That is, the number of plate models 51 is the same as the number of captured images, and the angle of an angle formed by two line segments respectively connecting the center of the panoramic map 50 and each center of the adjacent plate model 51 is the installation position of the imaging unit 11. Corresponds to the angle difference in the shooting direction by the shooting unit 11 when shooting adjacent images, and the aspect ratio of each plate model 51 corresponds to the aspect ratio of each shot image, and a plurality of plate models 51. The distance from the center of the sphere to the plate model 51 and the vertical and horizontal sizes of the plate model 51 correspond to the focal length of the imaging unit 11 and the vertical and horizontal sizes of the captured image. Each plate model 51 is assigned an ID in the same manner as the photographed image data, and the panorama image data generation unit 32 is based on the ID assigned to the photographed image data and the ID assigned to the plate model 51. Then, panoramic image data is generated by mapping the captured image data to the corresponding plate model 51, and the generated panoramic image data is output to the image data output unit 35 or stored in the storage unit 33. Here, the panoramic image data generation unit 32 can generate panoramic image data of a predetermined range centered on the viewpoint position based on the viewpoint position instruction output from the input unit 20.

≪Storage unit 33≫
Returning to FIG. 1, the storage unit 33 stores the panorama map 50 (see FIG. 2) in advance, and further stores the panorama image data output from the panorama image data generation unit 32. Also, the storage unit 33, other image data for other images for superimposing the panoramic image is stored.

<< Polymerized image data generation unit 34 >>
The superimposed image data generation unit 34 reads the other image data and the panorama image data stored in the storage unit 33, generates the superimposed image data related to the superimposed image in which the panoramic image is superimposed in front of the other image , and generates the generated polymerization. The image data is output to the image data output unit 35.
Here, based on the viewpoint position instruction output from the input unit 20, the superimposed image data generation unit 34 reads other image data and panoramic image data in a predetermined range centered on the viewpoint position from the storage unit 33 and reads them. Superimposition image data is generated based on the output data.

Further, the superimposed image data generation unit 34 also serves as a translucent processing unit , and performs processing for making the panoramic image data read from the storage unit 33 translucent based on the translucent processing instruction output from the input unit 20. Do. Panoramic when semitransparent process, polymerization image data generating unit 34, the overlapping panorama image data portion is removed hidden in the photographic image data and translucent process was translucent process on top of other images Superposition image data is generated by superimposing the image data. Proportion of translucent can be appropriately changed by a semi-transparent processing instruction.
Here, the peripheral portion of the captured image 52 mapped to the plate model indicated by the thick line in FIG. 3A overlaps with the adjacent captured image and is hidden behind the panoramic map 50 as viewed from the center. Yes. That is, in the captured image 52, the central portion 52a is visible from the center of the panorama map 50, and the peripheral portion 52b is not visible (hidden) from the center of the panoramic map 50 ( see FIG. 3B ). Therefore, the superimposed image data generation unit 34 reads the panorama image data from the storage unit 33 when the panorama image data is translucently processed, and relates to the peripheral portion 52b that overlaps and hides the captured image adjacent to each captured image. Display panorama image data not including photographed image data is generated, and then translucent processing is performed. That is, the panorama image data including the captured image data related to the peripheral edge portion 52b that is hidden by overlapping remains in the storage unit 33 as it is.

As an example of the translucent processing , the superimposed image data generation unit 34 reads panorama image data from the storage unit 33, generates display panorama image data related to the actually displayed panorama image, and stores the display panorama image data in advance. Remember. Further, the superimposed image data generation unit 34 outputs the display panorama image data to the image data output unit 35. Such display panoramic image data does not include data of the overlapped and hidden portion of the captured image. Then, polymerization image data generating unit 34, based on the semi-transparent processing instruction output from the input unit 20, the stored display the panorama image data is translucently process.

<< Image Data Output Unit 35 >>
The image data output unit 35 outputs the panorama image data to the display device 40 and causes the display device 40 to display the panorama image. In the present embodiment, the image data output unit 35 outputs superimposed image data obtained by superimposing panoramic image data to other image data (for example, CG image data) output from the superimposed image data generation unit 34 to the display device 40. It can also be output.

<< Panorama Image Correction Unit 36 >>
The panoramic image correcting unit 36 corrects the panoramic image by moving the captured image mapped to the plate model 51 in the latitude direction and the meridian direction based on the correction instruction output from the input unit 20. The panoramic image correcting unit 36 can also correct the panoramic image by rotating and moving the captured image mapped to the plate model 51 or changing the scale (enlarging or reducing).

<Display device 40>
The display device 40 is configured to display a panoramic image using the panoramic image data output from the panoramic image data generation device 30.

<First operation example>
Next, a first operation example of the panoramic image display system 1 according to the embodiment of the present invention will be described with reference to FIG. The first operation example is an operation example for moving a captured image in a virtual space in order to correct a panoramic image.

At the time of photographing by the photographing unit 11, there may be a deviation between adjacent photographed images as shown in FIG. 4A due to an error of the pan / tilt driving unit 12. In the example shown in FIG. 4A, the left photographic image 52L and the right photographic image 52R are deviated in the vertical direction (meridian direction) as a result of the deviating direction of the left photographic image 52L. Of the captured images 52L, 52R, and 52D indicated by the solid rectangular frames, the portions indicated by the dotted rectangular frames are actually used and displayed in the panoramic image.

When correcting such a deviation, the superimposed image data generation unit 34 generates correction panoramic image data (unpolymerized) based on the line-of-sight position output from the input device 20, and the image data output unit 35 Are output to the display device 40. On the display device 40, a panoramic image for correction is displayed.
When the user views the correction panorama image and determines that there is a shift between the adjacent captured images 52L and 52R, the user operates the input device 20. The input device 20 outputs to the panoramic image data correction unit 36 a correction instruction including the selection result of the captured image and the moving direction and moving amount of the selected captured image.
The panoramic image data correction unit 36 acquires the correction instruction output from the input device 20, and corrects the panoramic image data by shifting the position of the captured image in the virtual space from the plate model in accordance with the correction instruction. The corrected panoramic image data is stored in the storage unit 33.
For example, the panoramic image correction unit 36 corrects the position information of the panoramic image data so that the left captured image 52L is moved upward even if there is a deviation between the adjacent captured images 52L and 52R in FIG. By doing so, the panorama image can be corrected as shown in FIG. In addition, in the portion that is vacated by moving the captured image 52L up, there is a portion that overlaps the captured image 52D below the captured image 52L and is hidden, and this portion is displayed. It is possible to prevent a blank portion from being generated in the panoramic image.

<Second operation example>
Next, a second operation example of the panoramic image display system 1 according to the embodiment of the present invention will be described with reference to FIGS. The second operation example is an operation example for comparing a CG model of a building with a panoramic image after completion.

In this operation example, the storage unit 33 stores in advance CG image data related to a three-dimensional CG image representing a CG model of a building, which is generated before the building is built.
After the building of the building is completed, the captured image data acquisition unit 31 acquires a plurality of captured image data related to the plurality of captured images captured by the imaging device 10, and the panoramic image data generation unit 32 acquires the plurality of captured image data. The panorama image data is generated using the data and stored in the storage unit 33.

  Based on the line-of-sight position output from the input device 20, the superimposed image data generation unit 34 uses the CG image data related to the three-dimensional CG image (see FIG. 5A) centered on the line-of-sight position, and the line-of-sight position as the center. Panorama image data related to the panorama image (see FIG. 5B) to be read from the storage unit 33, and based on the read data, a semi-transparent panorama image is superimposed on the front of the three-dimensional CG image Superposed image data relating to the image is generated and output to the image data output unit 35.

Subsequently, the image data output unit 35 outputs the superimposed image data to the display device 40, and causes the display device 40 to display the superimposed image (see FIG. 5C).
The user who has seen the superposition image shown in FIG. 5C can see the translucent panoramic image and the CG model behind it, so that the CG model of the building and the building were actually constructed. Compare panoramic images of buildings at a glance.

<Third operation example>
Next, a third operation example of the panoramic image display system 2 according to the embodiment of the present invention will be described with reference to FIG. The third operation example is an operation example for displaying a panoramic image before the interior of the building (first panoramic image) and a panoramic image after the interior of the building (second panoramic image).

  Before the interior of the building in the middle of construction, the captured image data acquisition unit 31 acquires a plurality of captured image data regarding a plurality of captured images captured by the imaging device 10, and the panoramic image data generation unit 32 acquires a plurality of captured images. First panorama image data is generated using the image data and stored in the storage unit 33.

  Subsequently, after the building of the building is completed, that is, after interior decoration, the captured image data acquisition unit 31 acquires a plurality of captured image data regarding a plurality of captured images captured by the imaging device 10, and a panoramic image data generation unit 32 generates second panoramic image data using a plurality of captured image data, and stores the second panoramic image data in the storage unit 33.

  Subsequently, based on the line-of-sight position output from the input device 20, the superimposed image data generation unit 34 generates second panoramic image data related to the second panoramic image centered on the line-of-sight position (see FIG. 6B). Is displayed from the storage unit 33, display image data relating to the second panoramic image centered on the line-of-sight position is generated based on the read data, and is output to the image data output unit 35.

  Subsequently, the image data output unit 35 outputs the display image data to the display device 40, and causes the display device 40 to display a second panoramic image (see FIG. 6B) centered on the line-of-sight position.

Subsequently, the superimposed image data generation unit 34, based on the translucent processing instruction output from the input device 20, the first panorama regarding the first panorama image (see FIG. 6A) centered on the line-of-sight position. The image data is read out, and the second panorama image data is translucently processed to generate superimposed image data, which is output to the image data output unit 35.

  Subsequently, the image data output unit 35 outputs the superimposed image data to the display device 40, and causes the display device 40 to display the superimposed image (see FIG. 6C). The user who sees the superimposed image shown in FIG. 6C can see the semi-transparent second panoramic image and the first panoramic image behind it, so that the internal state of the building can be seen. I can grasp it.

  In the panoramic image display system 1 according to the embodiment of the present invention, since the adjacent captured images overlap in the panoramic image, even if the captured image is moved with respect to the adjacent captured images, a blank portion is left between the captured images. Does not occur. Therefore, when there is a deviation in the shooting direction, it is possible to maintain the continuity of the shooting target by moving the shot image, so that a suitable panoramic image is generated without processing the original shot image. be able to.

In addition, when the panoramic image display system 1 according to the embodiment of the present invention displays a panoramic image in a transparent manner, the portion of the captured image data that overlaps and is hidden is removed from the panoramic image data, and the portion that is hidden is removed. Since the panoramic image data is translucently processed , it is possible to prevent the portion that has been overlapped and hidden by the translucent display from being displayed.

DESCRIPTION OF SYMBOLS 1 Panorama image display system 30 Panorama image data generation apparatus 31 Captured image data storage part 32 Panorama image data generation part 34 Superimposition image data generation part 35 Image data output part

Claims (3)

In order to compare a panoramic image and another image different from the panoramic image, a panoramic image data generation device that generates panoramic image data related to the panoramic image using captured image data related to a plurality of captured images,
A photographed image data storage unit for storing a plurality of photographed image data;
Reading the stored plurality of captured image data, by mapping the plurality of plate models constituting a panorama map arranged in the virtual space, and panoramic image data generator for generating the panoramic image data regarding the panoramic image,
An image data output unit for outputting the generated panoramic image data;
With
The plate model is arranged in the virtual space so as to overlap with the other adjacent plate models ,
Based on the panoramic image data and other image data related to the other image, further includes a superimposed image data generation unit that generates superimposed image data related to a superimposed image obtained by superimposing the panoramic image and the other image,
The superimposed image data generation unit
For the overlapped portion of the plurality of captured image data, the panoramic image data in which only the viewpoint side is drawn is translucently processed,
Based on the other image data and the translucent panorama image data, the superimposition of the translucent panorama image in front of the other image to compare the other image with the panorama image Generate superposition image data for the image,
The panoramic image data generation device, wherein the image data output unit outputs the generated superimposed image data. Further comprising a CG image data storage CG image data is stored relating to the three-dimensional CG image is the another image,
The polymerization image data generating unit, the CG image data and based on the semi-transparent processing is the panoramic image data by superimposing the translucent the panoramic image in front of the three-dimensional CG image, the three-dimensional CG panoramic image data generating apparatus according to claim 1, characterized in that to produce a polymer image data relating to the polymerization picture for comparing the image with the said panoramic image. The panorama image data generation unit generates first panorama image data related to the first panorama image that is the other image, and second panorama image data related to the second panorama image that is the panorama image ,
The superimposed image data generation unit
With respect to the overlapping portions of the plurality of captured image data, the second panoramic image data in which only the viewpoint side is drawn is translucently processed ,
Based on the first panorama image data and the second panorama image data subjected to the translucent process, the second panorama image that is translucent is superimposed on the front of the first panorama image . panoramic image data generating apparatus according to claim 1, characterized in that to produce a polymer image data relating to the polymerization picture for comparing said second panorama image as one of a panoramic image.

Images