JP2009145061A - Position measuring device - Google Patents

Abstract

It is possible to correctly determine the position of each object from a multi-viewpoint image in which a plurality of objects exist without identifying the identity between the images in advance.
A figure similar to a figure obtained by directly looking at the plane from above using a multi-viewpoint image of a background image, a test image, and an image including a plurality of objects taken from different directions on a plane area in a three-dimensional space. , Plane projective transformation with each image plane taken by the camera, determine the vertical vanishing point of each image, extract the object area for each image including the object, and determine the internal points for each object area Then, a straight line passing through the point and the vertical vanishing point is obtained, and the obtained straight line is projected onto the above similar figure using plane projective transformation, the intersection of the projected straight lines is obtained, and the intersection is determined according to the position on the figure. By dividing into groups, the number of intersections included in each intersection group is obtained, and the correct position of the object on the plane is obtained from the group having a large number of intersections.
[Selection] Figure 1

Description

  The present invention relates to a position measurement apparatus that determines the position of an object from multi-viewpoint images obtained by photographing the object with a plurality of cameras.

As a technique for determining the position of an object from multi-viewpoint images obtained by shooting an object with multiple cameras, an image shot with a camera for a single object and an image shot with a different camera are used. , Determine each vertical line passing through the area of the object in each image, convert the vertical line on one of the images to the other image using plane projective transformation, and convert the converted straight line There is a method in which the intersection of vertical lines on the image is set as the position of the object. (For example, refer nonpatent literature 1).
In addition, in the images taken by three or more cameras, the vertical lines passing through the object regions considered to be identical to each other are determined, and the vertical lines on the image photographed by one camera, and the image, Obtain a straight line obtained by projecting a vertical line on an image taken with another camera by plane projective transformation, calculate the distance between the straight lines, and reject the straight line whose distance between the straight lines exceeds half and exceeds the threshold. Thus, there is a method of determining an accurate object position by using the position of the intersection point only by a straight line derived from the correct object region. (For example, refer nonpatent literature 2).

Kenji Kimura, Hideo Saito “Generation of Player Perspective Image from Multi-viewpoint Tennis Image”, Emotion Magazine, Vol. 58, no. 10, pp. 1503-1510 (2004) Shinya Nagase and Shinji Ozawa, “Judgment of player's play in soccer using multi-view video”, Eiji Magazine, Vol. 60, no. 10, pp. 1664-1671 (2006)

  The conventional position measurement method using multi-viewpoint images is as described above. However, in the case of Non-Patent Document 1, an object through which a vertical line in an image photographed by a camera passes and an image photographed by another camera are used. It is presupposed that the object through which the vertical line passes is the same. Therefore, if there are multiple different objects in the image and each draws a vertical line, the multiple vertical lines are projected onto another image by plane projective transformation to obtain the intersections with the multiple vertical lines on the image. In other words, the intersection of straight lines derived from the same object and the intersection of straight lines derived from different objects are generated, and the position of each object may not be determined correctly. This problem can be solved if the same objects can be identified for a plurality of objects in images taken by different cameras, but this identification has been difficult so far.

  The present invention has been made to solve the above-described problem. From a multi-viewpoint image in which a plurality of objects exist, the position of each object can be determined without previously identifying the identity of the plurality of objects between the images. An object of the present invention is to obtain a position determination device that can correctly determine the position.

  The position determination device according to the present invention includes a background image, a test image, and a plurality of objects obtained by photographing a predetermined region on a plane from different directions by a plurality of cameras installed so as to surround the plane in the three-dimensional space. A position measurement device that calculates the positions of a plurality of objects on a plan view using multi-viewpoint images of each of the images to be included, and is a plan view for creating a similar plan view that resembles a figure on a plane viewed from directly above Based on the plane in each test image and the similar plan view, a plane projective transformation determination unit that determines the plane projective transformation between the image captured by the camera and the similar plan view, and the same angle based on each test image A vertical vanishing point determining unit that determines the vertical vanishing point of each image captured by the camera, an object region extracting unit that extracts each object region by taking a difference between each background image and the image including the plurality of objects, and extraction The An object area internal point determination unit that determines an object area internal point as a reference for each object area, and an object passing vertical line determination unit that determines a straight line passing through each object area internal point and a vertical vanishing point for each image; A straight line determining unit that projects straight lines passing through points in the object area of all images onto a similar plan view based on plane projective transformation, and obtains an intersection of the straight lines projected onto the similar plan view An intersection determination unit, a grouping unit for grouping a plurality of intersections based on positions on the similar plan view, and an intersection group effective as an object position based on the number of intersections included from among the grouped intersection groups And an object position determination unit that calculates a correct position of each object on the similar plan view based on the determined intersection point in the intersection group.

  According to the present invention, in a multi-viewpoint image in which a certain three-dimensional region is photographed by a plurality of cameras from different directions, an object on a plane in a three-dimensional space is photographed in common for each image. In this case, the positions of the objects can be determined even if the identity of the objects is not previously identified between the camera images.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a functional configuration of a position measuring apparatus according to Embodiment 1 of the present invention.
1, the position measuring apparatus according to the first embodiment includes a plurality of cameras 1 ₁ to 1 _N (N ≧ 3), an image data temporary storage unit 2, a floor plan view creation unit (similar plan view creation unit) 3. , Background image storage unit 4, test image storage unit 5, vertical vanishing point determination unit 6, floor surface corresponding plane projection conversion determination unit (image / graphic plane projection conversion determination unit) 7, object region extraction unit 8, in object region A point determining unit 9, an object passing vertical line determining unit 10, a straight line determining unit on the floor plan view (straight line determining unit on the plan view) 11, an intersection determining unit 12, a grouping unit 13, and an object position determining unit 14 are provided.
The plurality of cameras 1 ₁ to 1 _N (N ≧ 3) are installed so as to surround a plane in the three-dimensional space, are synchronized with each other, and are photographed from different directions on the plane, respectively, to obtain a multi-viewpoint image. To get. In the present invention, each camera captures a background image, a test image, and an image including a plurality of objects existing on a plane thereafter from the same viewpoint that is not moved after installation. As an example using eight cameras, a background image capturing method will be described with reference to FIG. 2, a test image capturing method with reference to FIG. 3, and an image including an object with reference to FIG. 2 to 4, considering a floor composed of horizontal planes, eight cameras ₁₁ to ₁₈ mounted on a tripod of approximately the same size are placed on the floor so as to surround the floor. In any shooting, the cameras ₁₁ to ₁₈ do not change their positions and orientations. In addition, when installing the camera, it is possible to capture the “floor area within the field of view of all cameras” from any camera.

In the background image capturing example of FIG. 2, the floor surface on which nothing is placed is used as the background of the capturing target. As another example, a state in which an object other than the object to be imaged shown in FIG. 4 is placed on the floor may be used as the background of the object to be imaged, but the position and orientation of the object includes the object shown in FIG. It is a condition that there is no change at the time of shooting. In the test image shooting example of FIG. 3, a square marked on the floor surface and two bars installed perpendicularly to the floor surface of the square are taken as a shooting target. It is assumed that these squares and two bars can be photographed by all the cameras ₁₁ to ₁₈ . In the example of photographing an image including the object in FIG. 4, a stuffed animal and a vase placed on the floor are taken as objects to be photographed.
After the images obtained by photographing the Figures 2-4 which is temporarily stored in the image data temporary storing section 2 by the camera 1 ₁ to 1 _8, when the processing of the positioning device, the background image is a background image storage unit 4 The test images are transferred to and stored in the test image storage unit 5. Further, the image including each object is given to the object region extraction unit 8.
In the following, in FIGS. 5 to 8, images taking four cameras 1 ₁ , 1 ₃ , 1 ₅ , and ₁₇ as examples will be described.

The vertical vanishing point determination unit 6 determines the vertical vanishing point of the image taken by the camera at the same angle based on each test image stored in the test image storage unit 5. FIG. 5 shows an example in which the vertical vanishing point is calculated from the test images photographed by the cameras 1 ₁ , 1 ₃ , 1 ₅ , and ₁₇ shown in FIG. In each test image, a square marked on the floor and two bars installed perpendicular to the floor are photographed. In addition, the broken line in a figure is an auxiliary line which shows how to obtain | require a vertical vanishing point. For each test image, a straight line including two bars perpendicular to the floor in the image is extended, and the intersection is determined as a vertical vanishing point. In the present invention, a vanishing point that is a point obtained by projecting an infinite point of a straight line in a three-dimensional space onto an image is called a vertical vanishing point. Details of the infinite point and vanishing point are disclosed in, for example, the document “Satoshi Sato“ Geometry of Computer Vision ”, Corona, 1999”. In this example, the vertical vanishing point is determined using two vertical lines, but it may be determined by other methods. The vertical vanishing point is used to determine the direction of the vertical straight line with respect to the horizontal floor surface, but the vertical direction may be determined by other methods, and the vertical axis direction of the image may be changed depending on the installation position of the camera. You may approximate the vertical direction.

The object region extraction unit 8 extracts the object region by taking the difference between each background image stored in the background image storage unit 4 and the image including each of the plurality of objects stored in the image data temporary storage unit 2. FIG. 6 shows images including objects photographed by the photographing cameras 1 ₁ , 1 ₃ , 1 ₅ , and ₁₇ shown in FIG. In FIG. 6, a stuffed animal and a vase are represented in each image. It should be noted that the floor, wall, and other objects in the background are actually shown behind them, but are omitted here for the sake of simplicity. A difference for each pixel value is taken between an image in which two objects are photographed as shown in FIG. 6 and a background image stored in the background image storage unit 4, and the difference value is equal to or greater than a certain threshold value. An area including a pixel is extracted as an object area. An example of an image representing the object region extracted by the object region extraction unit 8 is shown in FIG. In this example, since the object region is extracted by the background subtraction method, the stuffed animal and vase regions that did not exist at the time of photographing the background image are shown in FIG. Since the portion where the pixel value has changed is extracted as the object region, at the time of FIG. 7, it is not possible to determine which of the two regions in each image is a stuffed animal and which is a vase. In this example, the object region is extracted by the background subtraction method. However, the object region may be extracted by the movement or color of the object, and if the region of the object whose position is to be determined can be extracted from the image captured by the camera. Well, for example, some of it may be missing due to shielding or other factors.

The object area inner point determination unit 9 determines an object area inner point serving as a reference for each object area extracted by the object area extraction unit 8. In the image of the object region extracted by the object region extraction unit 8 shown in FIG. 7, for example, the position of the center of gravity of the area of the object region is calculated, and the position is determined as an object region inner point serving as a reference for each object region. .
Next, the object passing vertical line determination unit 10 determines a straight line passing through each object region inner point determined by the object region inner point determination unit 9 and the vertical vanishing point for each image. That is, by connecting the vertical vanishing point and the object area inner point for each camera shown in FIG. 5, a vertical line passing through the object area inner point in each image as shown in FIG. 8 is created. This is a projection of a straight line perpendicular to a plane in a three-dimensional space.

Next, prior to the description of the processing of the straight line determination unit 11 on the floor plan view, the processing by the floor plan creation unit 3 and the floor corresponding plane projection conversion determination unit 7 will be described.
The floor plan view creation unit 3 creates a similar plan view that resembles a figure as seen from directly above the floor surface (plane) in shooting a test image. A similar plan view created by the floor plan view creation unit 3 is shown in FIG. In FIG. 9, squares having different sizes from those marked on the floor surface shown in FIG. 3 are written. Note that the similar plan view here is a figure drawn on an image, but here it is simply treated as a “similar plan view”.
In the floor surface corresponding plane projection conversion determining unit 7, a photograph is taken with a camera based on the plane in each test image stored in the test image storage unit 5 and the similar plan view created in the floor plan view creation unit 3. Planar projective transformation between an image and a similar plan view (planar projective transformation between image and figure) is determined. From the coordinates of the four vertices of the square marked on the floor in the test image (FIG. 5) photographed by each camera and the coordinates of the four vertices of the similar plan view (square) shown in FIG. The plane projective transformation from the floor surface of the image taken in step 1 to the similar plan view is determined. The calculation method of the planar projective transformation is disclosed in Koichiro Deguchi “Basics of Robot Vision”, Corona, 2000, for example. Here, the four vertices of the square marked on the floor surface and the four vertices of the plan view corresponding thereto are used, but the shape may not be a square. Further, it is sufficient that the point is a point on a certain plane in the camera image and a point on the plan view corresponding to the four or more points.

Next, the straight line determining unit 11 on the floor plan view determines the straight line passing through the object area inner point determined by the object passing vertical line determining unit 10 in all images by the floor corresponding plane projective transformation determining unit 7. Projection is performed on the similar plan view based on the projected image / graphic plane projection transformation. FIG. 10 shows a state in which all the vertical lines passing through the object region on each camera image as shown in FIG. 8 are projected onto a similar plan view. In FIG. 10, L <b> 1 to L <b> 16 indicate straight lines on the plan view determined by the straight line determining unit 11 on the floor plan view. In each of the images acquired by the _eight cameras 1 ₁ to 1 ₈ , two objects are extracted and the number of straight lines passing through the objects is also two one image. When straight lines are converted, a total of 16 straight lines are obtained.
The intersection determining unit 12 determines the intersection of the straight lines projected on the similar plan view by the straight line determining unit 11 on the floor plan view. That is, in the case of FIG. 10, all intersections formed by the straight lines L1 to L16 are obtained. Next, the grouping unit 13 groups a plurality of intersections on the similarity plan obtained by the intersection determination unit 12 based on the positions on the similarity plan. In this case, the intersections where the distance between the intersections is within a certain threshold are grouped together as the same group. By doing so, grouping can be performed easily. In the similar plan view of FIG. 10, it is shown that it is divided into six intersection groups A, B, C, D, E, and F surrounded by ○.

Next, the object position determination unit 14 determines an intersection group effective as the position of the object from the intersection groups grouped by the grouping unit 13 based on the number of intersections included, and the intersections in the determined intersection group Based on the above, the correct position of each object on the similar plan view is calculated. Among the intersection groups shown in FIG. 10, for example, the top M intersection groups (M is a predetermined number and M = 2 in this example) having the largest number of intersections included in the group are used as object positions. Determine a valid intersection group. And the average value of the coordinate of the intersection point contained in each said upper group is each taken, and each average value is calculated as a correct position of the object of a corresponding group. In this case, instead of all the intersections in the effective intersection group, some of them may be used. It is only necessary for a person to specify the number of objects in advance to determine the number of intersection groups from which the number of intersections is large as the effective intersection group. By doing so, it is possible to determine the object position even if there are a plurality of objects and the objects extracted for each image in advance cannot be identified. On the other hand, the intersection group effective as the object position may use the number of object regions extracted by the object region extraction unit 8, so that the actual number of objects can be obtained without determining the number of objects in advance as described above. Accordingly, the position of the object can be calculated with higher accuracy.
If the number of extracted object regions differs for each camera, a numerical value obtained by converting the average number into an integer may be used, or the frequency extracted most frequently from the number of objects extracted by each camera. You may use the number of objects. In addition, when groups are arranged in descending order of the number of intersections included in the group and the number of intersections is greatly reduced with a certain group as a boundary, the position up to that group may be set as the object position. The number of intersections may be changed depending on the number of photographing cameras. Alternatively, the determination may be made by using the density or dispersion in a partial area with a smaller number of intersections in the group. Further, the number of objects extracted for each image photographed by each camera is obtained, and among the number of objects, the highest number of objects is set as a predetermined number of objects M. It is good also considering the group of an object as the position of an object.

Further, the object position may be determined using the method shown in the flowchart of FIG. In this case, a re-grouping unit 131 indicated by a broken line in FIG. 1 is provided.
Referring to the group of FIG. 10 as an example, the object position determination unit 14 selects the group with the largest number of intersections from among the six groups A, B, C, D, E, and F that are first grouped by the grouping unit 13. The first intersection group effective as the position of the object is determined. If there are a plurality of groups having the largest number of intersections, an arbitrary group among them may be the first intersection group, and a group having the smallest variance of the intersection coordinates may be the first intersection group. Now, assume that group E is determined as the first intersection group. Next, the re-grouping unit 131 removes the straight line that formed the intersection of the first intersection group E, groups the remaining straight line intersections, and re-inputs them to the object position determination unit 14. In this example, the straight lines L9 to L16 are removed. Then, the group of intersections formed by the remaining straight lines L1 to L8 is only B. Therefore, the object position determination unit 14 calculates the correct position of the object from the intersection point with the group B as the second intersection point group that is effective as the position of the next object.

Here, in the above example, only B remains after removing the straight lines L9 to L16 related to the intersections of the first intersection group, but there may be cases where two or more groups remain. In this case, the group having the largest number of intersections among the remaining groups is set as the second intersection group effective as the position of the next object. Next, after the straight line related to the intersection of the second intersection group is removed by the regrouping unit 131, the group of intersections of the remaining straight lines is reorganized and effective as the position of the third object among them. A third intersection group is determined. Thereafter, when there are remaining groups, this process is repeated between the object position determination unit 14 and the regrouping unit 131. Eventually, if the position of an object is calculated from a group of intersections with a predetermined number of objects, or if the number of intersections included in the group and the variance value of its coordinates become smaller than a certain threshold, no more objects will be The object position determination is terminated as it does not exist or the position cannot be determined. In FIG. 11, only a predetermined number of object positions are found, but the number of object positions may be determined by other methods.
According to the method described with reference to FIG. 11, the position of an object can be determined with higher accuracy even when the number of objects increases.

  As described above, according to the first embodiment, a plurality of background images, test images, and images including a plurality of objects, each of which is obtained by photographing a region on a plane in a three-dimensional space from different directions using a plurality of cameras. Using a viewpoint image, determine the plane projective transformation between the figure similar to the figure seen from directly above the plane and the plane in each image taken by the camera, and determine the vertical vanishing point of each image. The object region is extracted for each image including a plurality of objects, internal points are determined for each object region, a straight line passing through these points and the vertical vanishing point is determined for each object, and the obtained straight line Projecting onto a figure similar to the figure seen from directly above using plane projective transformation, obtaining the intersection of the projected straight lines, grouping the intersection according to the position on the figure, and grouping the grouped intersection Find the number of intersections included in each , So that the correct position of the object on the plane to choose several from the group with many number of intersections. Therefore, the intersections of the straight lines are determined on the plan view, the intersection points are grouped, and the positions of the objects are determined. Therefore, it is not possible to identify the objects extracted in advance for each image with multiple objects. Even the position of the object can be determined.

Embodiment 2. FIG.
FIG. 12 is a block diagram showing a functional configuration of a position measuring apparatus according to Embodiment 2 of the present invention. In this figure, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted in principle.
In the second embodiment, an inter-image plane projection conversion determining unit 17 is provided in addition to the floor-surface corresponding plane projection conversion determining unit 7, and the inter-image plane projective conversion determining unit 17 is arranged on a plane in each test image. The plane projection conversion (inter-image plane projection conversion) between the images photographed by the camera is determined based on the figure. That is, the plane projective transformation for performing projection between the floor surfaces (planar regions) of the camera images represented by the coordinates of the four vertices of the square marked on the floor surface in the test image (FIG. 5) is determined. . As described in the first embodiment, the object passing vertical line determination unit 10 generates a straight line that passes through the point in the object region for each image. The captured image conversion straight line determination unit 21 determines for each of these images. The straight lines passing through the object point in the object area are projected onto the images photographed by the cameras using the inter-image plane projective transformation determined by the inter-image plane projective transformation determining unit 17. For example shows an example of the projected straight line in the photographed image of the camera 1 ₁ in Figure 13. Linear camera 1 ₁ itself other cameras that have been projected as linearly related to the image of 1 _{2-1 8} images are represented in this FIG. In this way, the images in which the converted straight lines of the images of the other cameras are collected together on the images of the cameras ₁₂ to ₁₈ are created.

Next, the intra-image intersection determination unit 22 obtains the intersections of the straight lines collected on each image on each image in the same manner as the intersection determination unit 12 of the first embodiment. In the primary grouping unit 23, the distance between the intersections in the image is within a certain threshold for a plurality of intersections on each image obtained by the in-image intersection determination unit 22, as in the grouping unit 13 of the first embodiment. Collect the intersections that are in the same group. In the primary object position determination unit 24, in the same manner as the object position determination unit 14 of the first embodiment, the position of the object is determined on the basis of the number of intersection points included in the group of intersection points on each image. The effective intersection group is determined, and the position of the object on each image is calculated based on the intersection in the determined intersection group. Referring to FIG. 13, for example, the intersection groups grouped by the primary grouping unit 23 on the image of the camera 1 ₁ (the same applies to each of the cameras 1 ₂ to ₁₈ ) are surrounded by circles. The primary object position determination unit 24 sets, from among these groups, the top M groups (M = 2) having the largest number of intersections included in each group as intersection groups representing the positions of the objects, and the intersections included in the groups. The average of the coordinates is determined as the coordinates of the object position. In this case, instead of obtaining the average of the coordinates of all the intersections, the coordinates of some intersections of the group may be used.

  Next, in the floor plan view position determination unit (plan view position determination unit) 25, the position of the object on each image determined by the primary object position determination unit 24 is converted by the floor surface corresponding plane projection conversion determination unit 7. The image is projected onto the similar plan view using the determined plane / surface projection transformation corresponding to the floor surface, and the object position candidates on the similar plan view are collected. In the secondary grouping unit 33, the position candidates of the objects grouped on the similar plan view by the floor plan view position determination unit 25 are grouped. This state is shown in FIG.

  Next, the secondary object position determination unit 34 selects a group effective as the correct object position from the group of object position candidates grouped by the secondary grouping unit 33 based on the number of position candidates included. The correct position of each object on the similar plan view is calculated based on the determined position candidate of the object in the group. In this case, among the group of object position candidates shown in FIG. 14, for example, the top m objects having a large number of object position candidates included in the group (m is a predetermined number. In this example, m = 2 ) Is determined as a valid group as the correct object position. And the average value of the coordinate of the object position candidate contained in each said upper group is each taken, and each average value is calculated as a correct position of the object of a corresponding group. In this case, not all the object position candidates in the effective group but some of them may be used.

As described above, according to the second embodiment, since grouping and part of object position determination are performed on the image, grouping and object position determination can be performed with a simpler procedure. In addition, by performing the grouping twice, the position of the object can be determined with high accuracy.
In this example, the floor plan view position determination unit 25, the secondary grouping unit 33, and the secondary object position determination unit 34 determine the correct position of the object on the plan view as shown in FIG. The position of the object may be determined by performing these operations on a certain image as shown in FIG.

It is a block diagram which shows the function structure of the position measuring device which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the imaging | photography method of the background image which concerns on each embodiment of this invention. It is explanatory drawing which shows the imaging | photography method of the test image for determining the perpendicular direction which concerns on each embodiment of this invention. It is a figure which shows the imaging | photography method of the image containing the object of the imaging | photography object which concerns on each embodiment of this invention. It is explanatory drawing which shows the determination method of the vertical vanishing point using the test image which concerns on each embodiment of this invention. It is explanatory drawing which shows the example of the image containing the object which image | photographed the situation shown in FIG. 4 with the camera. It is explanatory drawing which shows the example of an image showing the object area | region extracted by the object area | region extraction part which concerns on each embodiment of this invention. It is explanatory drawing which shows the straight line which passes the point in an object area | region determined by the object passage perpendicular | vertical line determination part which concerns on each embodiment of this invention. It is a top view which shows the similar plan view similar to the figure of the floor created by the floor plan creation part concerning each embodiment of this invention. It is explanatory drawing which shows the straight line on the similar plane view converted by the straight line determination part on the floor plan view concerning Embodiment 1 of this invention, and the grouped intersection. It is a flowchart which shows the other process sequence of the position determination method which concerns on Embodiment 1 of this invention. It is a block diagram which shows the function structure of the position measuring device by Embodiment 2 of this invention. It is explanatory drawing which shows the example of the straight line converted on one image which concerns on Embodiment 2 of this invention, and its grouped intersection. It is explanatory drawing which shows the position candidate of the object grouped on the similar top view which concerns on Embodiment 2 of this invention.

Explanation of symbols

1 ₁ to 1 _N camera, 2 image data temporary storage unit, 3 floor plan creation unit, 4 background image storage unit, 5 test image storage unit, 6 vertical vanishing point determination unit, 7 floor plane corresponding plane projection conversion determination unit , 8 Object region extraction unit, 9 Object region internal point determination unit, 10 Object passing vertical line determination unit, 11 Plane line straight line determination unit, 12 Intersection point determination unit, 13 Grouping unit, 131 Re-grouping unit, 14 Object position determination , 17 Inter-image plane projection conversion determination unit, 21 On-photographed image conversion line determination unit, 22 Primary intersection determination unit, 23 Primary grouping unit, 24 Primary object position determination unit, 25 Floor plan view determination unit, 33 Secondary grouping unit, 34 Secondary object position determination unit.

Claims (7)

Using a multi-viewpoint image, each of a background image, a test image, and an image including a plurality of objects, obtained by photographing a predetermined area on the plane from different directions by a plurality of cameras installed so as to surround a plane in a three-dimensional space A position measuring device for calculating the positions of a plurality of objects on a plan view,
A similar plan view creating unit for creating a similar plan view similar to the figure seen from directly above the figure on the plane;
Based on the plane in each test image and the similar plan view, an image / graphic plane projection conversion determining unit for determining an image / figure plane projection conversion between the image taken by the camera and the similar plan view;
A vertical vanishing point determining unit that determines a vertical vanishing point of an image captured by a camera at the same angle based on each test image;
An object region extraction unit that extracts the object region by taking the difference between the background image and the image including the plurality of objects,
An object area inner point determination unit for determining an object area inner point as a reference for each of the extracted object areas;
An object passing vertical line determining unit that determines a straight line passing through each object area internal point and the vertical vanishing point for each image;
A straight line determination unit on a plan view that projects straight lines passing through the points in the object area of all images onto the similar plan view based on the plane projective transformation;
An intersection determination unit for obtaining an intersection of straight lines projected on the similar plan view;
A grouping unit that groups the plurality of intersections based on positions on the similar plan view;
From the grouped intersection groups, an effective intersection group is determined as the position of the object based on the number of intersection points included, and the correctness of each object on the similarity plan is determined based on the intersection in the determined intersection group. A position measurement apparatus comprising an object position determination unit for calculating a position.   The position measuring device according to claim 1, wherein the grouping unit groups the intersections obtained by the intersection determination unit on the condition that a distance between adjacent intersections is equal to or less than a threshold value.   The object position determination unit determines a predetermined number of upper groups having a large number of intersections from the group of intersection points divided into groups as effective intersection groups as object positions, and the number of intersections included in each of the upper groups. The position measuring device according to claim 1, wherein the correct position of each corresponding object is calculated based on.   4. The position measuring apparatus according to claim 3, wherein the object position determination unit sets the number of objects having the highest frequency among the number of objects extracted by the camera as a predetermined number of the upper group. A regrouping unit for grouping the intersections of the remaining straight lines, except for the straight line related to the intersection of the intersection group, in which the object position determination unit sequentially calculates the correct position of the object in units of 1;
The object position determination unit determines a group having the largest number of intersection points as an effective intersection group from among the grouped intersection groups input from the grouping unit at first and then from the re-grouping unit. The position determination device according to claim 1, wherein the correct position of the corresponding object is calculated based on the number of intersections included in the effective intersection group. Using a multi-viewpoint image, each of a background image, a test image, and an image including a plurality of objects, obtained by photographing a predetermined area on the plane from different directions by a plurality of cameras installed so as to surround a plane in a three-dimensional space A position measuring device for calculating the positions of a plurality of objects on a plan view,
A plan view creation unit for creating a similar plan view similar to the figure seen from directly above the figure on the plane;
Based on the plane in each test image and the similar plan view, an image / graphic plane projection conversion determining unit for determining an image / figure plane projection conversion between the image taken by the camera and the similar plan view;
An inter-image plane projective transformation determining unit that determines an inter-image plane projective transformation between each image captured by a camera based on a figure on a plane in each test image;
A vertical vanishing point determining unit that determines a vertical vanishing point of an image captured by a camera at the same angle based on each test image;
An object region extraction unit that extracts the object region by taking the difference between the background image and the image including the plurality of objects,
An object area inner point determination unit for determining an object area inner point as a reference for each of the extracted object areas;
An object passing vertical line determining unit that determines a straight line passing through each object area internal point and the vertical vanishing point for each image;
A straight line passing through a point in the object area for each image, and a projected image conversion straight line determination unit that projects each line on each image based on the inter-plane planar projection conversion;
An in-image intersection determination unit for obtaining an intersection between the straight lines converted on each image on each image;
A primary grouping unit for grouping intersections obtained on each image based on positions on the image;
An effective intersection group is determined as the position of the object based on the number of intersection points included from the group of intersection points, and the position of the object on each image is calculated based on the intersection point in the determined intersection group. A primary object position determination unit that
A plan view position determining unit that projects the position of the object on each image onto the similar plan view using the image-figure plane projective transformation,
A secondary grouping unit that groups the position candidates of the objects collected on the similar plan view based on the positions on the similar plan view;
An effective group is determined as a correct object position based on the number of position candidates included from among the grouped object position candidate groups, and the similarity is determined based on the object position candidates in the determined group. A position measurement apparatus comprising a secondary object position determination unit that calculates a correct position of each object on a plan view.   The position measuring device according to claim 6, wherein the primary grouping unit groups the intersections obtained by the intra-image intersection determination unit on condition that a distance between adjacent intersections is equal to or less than a threshold value.

Images