JP2019186858A - Selection device, selection method, and selection program - Google Patents

Abstract

An image pickup apparatus that picks up an image capable of accurately detecting an object is selected. A selection device (2) is configured to, when a three-dimensional model imitating an object is virtually arranged at a predetermined position in a predetermined area, include a three-dimensional model included in a first image captured by a first imaging device. An area specifying unit 221 that specifies a first area indicating an area of the image to be displayed, a second area indicating an area of the image indicating the three-dimensional model included in the second image captured by the second imaging device, A confidence value that specifies a first confidence value that indicates reliability when an object is detected based on a region and a second confidence value that indicates a reliability value when an object is detected based on a second region. A specifying unit 222, and a selecting unit that selects an imaging device to be used for detecting an object at a predetermined position from the first imaging device and the second imaging device based on a ratio between the first reliability value and the second reliability value. 223 and Equipped with a. [Selection diagram] FIG.

Description

  The present invention relates to a selection device, a selection method, and a selection program that can select an imaging device that generates an image used for object detection.

  Conventionally, when generating a flow line indicating a movement trajectory of an object in a predetermined area, an object is detected based on a plurality of images acquired from a plurality of imaging devices (see, for example, Patent Document 1). .

Japanese Patent Laid-Open No. 2006-139949

  When the imaging ranges of a plurality of imaging devices are installed so as to overlap, one object may be reflected in an image captured by the plurality of imaging devices. In detecting the object with high accuracy, it is preferable to select an imaging device that picks up an image capable of detecting the object with high accuracy from among a plurality of imaging devices.

  Therefore, the present invention has been made in view of these points, and provides a selection device, a selection method, and a selection program that can select an imaging device that captures an image that can accurately detect an object. With the goal.

  In the selection device according to the first aspect of the present invention, a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in the three-dimensional space indicating the predetermined area. A first area indicating an area of the image indicating the three-dimensional model included in the first image captured by the first image capturing apparatus capturing the predetermined area, and a second image capturing performed by the second image capturing apparatus capturing the predetermined area. An area specifying unit that specifies a second area indicating an area of the image indicating the three-dimensional model included in two images, and the reliability of the detection when the object is detected based on the first area A confidence value identifying unit that identifies a first confidence value and a second confidence value indicating a reliability value of the detection when the object is detected based on the second region; and the first confidence value, Serial based on the ratio between the second confidence value, among the first imaging device and the second imaging device, and a selection unit for selecting an imaging device used to detect the object in the predetermined position.

The reliability value specifying unit may use the area of the first region for specifying the first reliability value and use the area of the second region for specifying the second reliability value.
The first imaging device or the second imaging device is an imaging device that captures an omnidirectional image, and the reliability value specifying unit calculates a distance between the position of the first region and the center of the first image. The first confidence value may be used for specifying the second confidence value by using a distance between the position of the second region and the center of the second image.

  In the case where the three-dimensional model and the second three-dimensional model imitating the second object that is not the target to be detected are virtually arranged, the first value in the confidence value specifying unit is the first area. A ratio hidden by an image region indicating the second three-dimensional model included in one image is used for specifying the first confidence value, and the second region is included in the second image. The ratio hidden by the image area indicating the model may be used for specifying the second confidence value.

  The selection device includes information associating a position in the image of the first image with a first confidence value corresponding to the position, a position in the image of the second image, and a second corresponding to the position. A storage unit that stores information associated with a confidence value; and the confidence value specifying unit refers to the storage unit and is associated with a position of an image indicating the three-dimensional model included in the first image. Obtaining the first confidence value, and obtaining the second confidence value associated with a position of an image indicating the three-dimensional model included in the second image, and The second confidence value may be specified.

  The selection unit is an imaging device used for detecting the object at the predetermined position when a ratio of the first reliability value to the second reliability value corresponding to the predetermined position is greater than or equal to a first threshold value. Used to detect the object at the predetermined position when the first imaging device is selected and the ratio of the second reliability value to the first reliability value corresponding to the predetermined position is greater than or equal to a second threshold value. The second imaging device may be selected as the imaging device to perform.

  The selection unit is configured such that a ratio of the first reliability value to the second reliability value corresponding to the predetermined position is less than the first threshold, and the ratio to the first reliability value corresponding to the predetermined position. When the ratio of the second confidence values is less than the second threshold value, the first imaging device and the second imaging device may be selected as the imaging device used for detecting the object at the predetermined position.

  When the predetermined position is within a first distance from the first imaging device, the selection unit is configured to detect the object at the predetermined position regardless of a ratio between the first reliability value and the second reliability value. When the first imaging device is selected as an imaging device to be used for detection of and the predetermined position is within a second distance from the second imaging device, the first confidence value and the second confidence value Regardless of the ratio, the second imaging device may be selected as the imaging device used to detect the object at the predetermined position.

  The reliability value specifying unit specifies the first reliability value and the second reliability value when the three-dimensional model is virtually arranged for each of a plurality of positions in the three-dimensional space, and the selection unit For each of a plurality of positions in the three-dimensional space, an imaging device used for detection of the object may be selected based on a ratio between the first confidence value and the second confidence value.

  The selection device further includes a storage control unit that associates device identification information for identifying the imaging device with location information indicating a location where the imaging device is selected to be used for detection of the object, and stores the information in the storage unit. You may prepare.

  The reliability value specifying unit, for each of a plurality of shapes corresponding to the object at the predetermined position, the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position and the A second confidence value is specified, and the selection unit determines the object based on a ratio between the first confidence value and the second confidence value for each of a plurality of shapes corresponding to the object at the predetermined position. You may select the imaging device used for detection.

  The reliability value specifying unit, for each of one or more shapes that can be taken by the object at the predetermined position, the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position The second confidence value may be specified.

  The selection device stores, in a storage unit, device identification information for identifying an imaging device, a shape of the object, and position information indicating a position where the imaging device is selected to be used for detection of the object. A storage control unit may be further provided.

  The selection method according to the second aspect of the present invention is a virtual method in which a computer executes a three-dimensional model imitating an object to be detected in a predetermined area at a predetermined position in a three-dimensional space indicating the predetermined area. A first area indicating an area of an image indicating the three-dimensional model included in the first image captured by the first imaging device that captures the predetermined area, and a second imaging for capturing the predetermined area. A step of identifying a second area indicating an area of the image indicating the three-dimensional model included in the second image captured by the apparatus; and the reliability of the detection when detecting the object based on the first area Identifying a first confidence value indicating the reliability and a second confidence value indicating a reliability value of the detection when the object is detected based on the second region; Selecting an imaging device to be used for detection of the object at the predetermined position from the first imaging device and the second imaging device based on a ratio between the first confidence value and the second confidence value; .

  The selection program according to the third aspect of the present invention virtually arranges a three-dimensional model imitating an object to be detected in the predetermined area at a predetermined position in a three-dimensional space indicating the predetermined area. In this case, a first area indicating an area of the image indicating the three-dimensional model included in the first image captured by the first image capturing apparatus capturing the predetermined area, and a second image capturing apparatus capturing the predetermined area. An area specifying unit that specifies a second area indicating an area of the image indicating the three-dimensional model included in the captured second image, and reliability of the detection when the object is detected based on the first area And a confidence value specifying unit that specifies a second confidence value indicating a reliability value of the detection when the object is detected based on the second region. Based on the ratio between the first confidence value and the second confidence value, an imaging device to be used for detecting the object at the predetermined position is selected from the first imaging device and the second imaging device. It functions as a selection unit.

  According to the present invention, it is possible to select an imaging device that captures an image capable of accurately detecting an object.

It is a figure which shows the structure of the object tracking system which concerns on this embodiment. It is a figure which shows the outline | summary of the selection apparatus which concerns on this embodiment. It is a figure which shows the structure of the selection apparatus which concerns on this embodiment. It is a figure which shows the imaging device selected in each of several position in three-dimensional space. It is a figure which shows the structure of the object tracking apparatus which concerns on this embodiment. It is a flowchart which shows the flow of a process in the selection apparatus concerning this embodiment. It is a flowchart which shows the flow of a process in the object tracking device which concerns on this embodiment.

[Outline of Object Tracking System S]
FIG. 1 is a diagram showing a configuration of an object tracking system S according to the present embodiment. The object tracking system S includes a first imaging device 1A, a second imaging device 1B, a selection device 2, and an object tracking device 3.

  The first imaging device 1A and the second imaging device 1B are imaging devices that image a predetermined area. Here, the predetermined area is, for example, a store floor. In the predetermined area, a person such as a store clerk exists as an object to be tracked in the object tracking system S. Each of the first imaging device 1A and the second imaging device 1B captures an omnidirectional image including at least a part of a predetermined area. Here, it is assumed that the range that can be imaged by the first imaging device 1A and the range that can be imaged by the second imaging device 1B overlap at least partially.

  In the present embodiment, each of the first imaging device 1A and the second imaging device 1B captures an omnidirectional image. However, the present invention is not limited to this, and an image that is not an omnidirectional image may be captured. .

  The omnidirectional images captured by the first imaging device 1A and the second imaging device 1B may include the same object. In this case, although the object can be tracked in any omnidirectional image, depending on the size of the image of the object, the object is tracked by the omnidirectional image captured by the first imaging device 1A. There may be a difference in tracking accuracy when tracking the object with the omnidirectional image captured by the second imaging device 1B.

  For this reason, for each of a plurality of positions included in the predetermined area, the selection device 2 selects an imaging device with high object tracking accuracy among the imaging devices used for specifying the object existing at the position as the first imaging device 1A. The second imaging device 1B is selected.

  FIG. 2 is a diagram illustrating an overview of the selection device 2 according to the present embodiment. As illustrated in FIG. 2, the selection device 2 virtually displays a three-dimensional model M imitating an object to be detected in a predetermined area at a predetermined position in the three-dimensional space A indicating the predetermined area among a plurality of positions. The first region indicating the region of the image indicating the three-dimensional model M included in the first image captured by the first imaging device 1A in the case where the first imaging device 1A is disposed is specified. The selection device 2 also captures the second image captured by the second imaging device 1B when the three-dimensional model M imitating the object is virtually arranged at a predetermined position in the three-dimensional space A indicating the predetermined area. The second region indicating the region of the image showing the three-dimensional model M is included.

  When the selection device 2 detects an object based on the first region, the selection device 2 indicates the first reliability value indicating the reliability of the detection, and the detection reliability when the object is detected based on the second region. The second confidence value indicating the sex is specified. The selection device 2 selects an imaging device to be used for detecting an object at a predetermined position from the first imaging device 1A and the second imaging device 1B based on the ratio between the first reliability value and the second reliability value. By doing in this way, the selection apparatus 2 can select an imaging apparatus that captures an image that can accurately detect an object at a predetermined position.

  The selection device 2 performs the above-described processing at each of a plurality of positions in a predetermined area, and selects an imaging device that captures an image that can accurately detect an object. Based on the selection result, the selection device 2 includes device identification information for identifying the imaging device, position information indicating a position where the imaging device is selected to be used for object detection, and the object is arranged at the position. In this case, responsible area information is generated in association with information indicating the area of the image indicating the object included in the image captured by the imaging device.

  Returning to FIG. The object tracking device 3 acquires an omnidirectional image from the first imaging device 1A and acquires an omnidirectional image from the second imaging device 1B. The object tracking device 3 identifies an area indicating an object included in the omnidirectional image by analyzing the acquired omnidirectional image.

  Here, the omnidirectional image acquired from the first imaging device 1A and the omnidirectional image acquired from the second imaging device 1B may include a region indicating the same object. The object tracking device 3 selects an object region used for tracking the object from the identified object region based on the identified object region and the assigned region information generated by the selection device 2. Then, the object tracking device 3 tracks the object based on the position associated with the area of the selected object in the assigned area information. By doing so, the object tracking device 3 can track the object with high accuracy.

[Configuration of Selection Device 2]
Next, the configuration of the selection device 2 will be described. FIG. 3 is a diagram illustrating a configuration of the selection device 2 according to the present embodiment. As illustrated in FIG. 3, the selection device 2 includes a storage unit 21 and a control unit 22.

  The storage unit 21 is a storage medium including a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 21 stores a program executed by the control unit 22. For example, the storage unit 21 stores a selection program that causes the selection device 2 to function as the region specifying unit 221, the confidence value specifying unit 222, the selecting unit 223, and the storage control unit 224.

  The control unit 22 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The control unit 22 functions as an area specifying unit 221, a confidence value specifying unit 222, a selecting unit 223, and a storage control unit 224 by executing the selection program stored in the storage unit 21.

  The area specifying unit 221 captures a predetermined area when a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in the three-dimensional space indicating the predetermined area. The first region indicating the region of the image indicating the three-dimensional model included in the first image captured by the one imaging device 1A is specified.

  Similarly, the area specifying unit 221 uses the second imaging device 1B that images a predetermined area when a three-dimensional model imitating the object is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area. A second region indicating an image region indicating the three-dimensional model included in the captured second image is specified.

  Specifically, the area specifying unit 221 constructs a virtual three-dimensional space indicating a predetermined area. And the area | region specific part 221 produces | generates an omnidirectional image similarly to the 1st imaging device 1A in the position corresponding to the installation position of the 1st imaging device 1A in a predetermined area in virtual three-dimensional space. Arrange objects. In addition, the area specifying unit 221 generates a omnidirectional image at a position corresponding to the installation position of the second imaging device 1B in a predetermined area in a virtual three-dimensional space, like the second imaging device 1B. Arrange objects. Here, the position in the virtual three-dimensional space is expressed by, for example, the three-dimensional coordinates (X, Y, 0) of the global coordinate system.

  The area specifying unit 221 causes the first camera object corresponding to the first imaging device 1A to generate an omnidirectional image when a three-dimensional model is arranged at a predetermined position in a virtual three-dimensional space indicating a predetermined area. At the same time, the second camera object corresponding to the second imaging device 1B is caused to generate an omnidirectional image. Here, the shape of the three-dimensional model is preferably the shape of the object (person) itself, but may be a shape that simply simulates the object by a combination of a cylinder or the like.

  The area specifying unit 221 analyzes the omnidirectional image generated by the first camera object, and specifies the first area indicating the area of the image indicating the three-dimensional model included in the omnidirectional image. For example, the area specifying unit 221 specifies the outline of the three-dimensional model included in the omnidirectional image generated by the first camera object, and specifies the information indicating the area surrounding the outline, thereby identifying the first area. Identify.

  Here, for the information indicating the region, a polygon that is a three-dimensional model having an arbitrary shape constituted by a group of contour points, or a circumscribed rectangle of the polygon can be used. Here, the circumscribed rectangle is a rectangle that circumscribes the polygon and has two sides parallel to a predetermined axis on the omnidirectional image. The information indicating the rectangular area includes two-dimensional coordinates (x, y) on the image indicating the center point of the rectangular area and two vertices adjacent to the vertex in the local coordinate system defined for the omnidirectional image. The distances w and h and the information (x, y, w, h, θ) including the inclination θ with respect to the predetermined axis are defined. When θ is zero degrees, it is called an upright rectangle, and when it is not zero degrees, it is called a rotating rectangle.

  Similarly, the area specifying unit 221 analyzes the omnidirectional image generated by the second camera object, and specifies the second area indicating the area of the image indicating the three-dimensional model included in the omnidirectional image.

  Here, for each of a plurality of shapes corresponding to the object at a predetermined position, the region specifying unit 221 has a first reliability value and a second reliability value when a three-dimensional model imitating the shape is arranged at the predetermined position. And specify. Since the object is a person, the shape of the object is, for example, the shape of a person when standing upright, the shape of a person when sitting on a chair, the shape of a person showing an adult, the shape of a person showing a child It is.

  In addition, when a chair or the like is not provided at a predetermined position and only a person is assumed to stand, the first confidence value and the second confidence value corresponding to the shape of the person when sitting on the chair The need to specify a value is low. Therefore, when the object can take a plurality of shapes at a predetermined position, the region specifying unit 221 places a three-dimensional model imitating the shape at a predetermined position for each of the plurality of shapes that the object can take. In this case, the first region and the second region are specified.

  For example, the storage unit 21 stores shape-specific position information in which each of a plurality of shapes that can be taken by an object is associated with a position where the object can take the shape. The region specifying unit 221 specifies the shape of an object that can be taken at a predetermined position with reference to the position information by shape. The area specifying unit 221 specifies the first area and the second area when a three-dimensional model imitating the shape is arranged at a predetermined position for each of a plurality of shapes that the object can take at the predetermined position. .

  The area specifying unit 221 performs the processing of the above-described procedure for each of a plurality of positions in a three-dimensional space indicating a predetermined area, and each of the plurality of positions is a cubic that imitates a shape that an object can take at the position. The first area and the second area when the original model is virtually arranged are specified.

  The confidence value specifying unit 222 includes a first confidence value indicating the reliability of the detection when the object is detected based on the first area, and the reliability of the detection when the object is detected based on the second area. The second confidence value indicating the sex is specified.

  For each of one or more shapes that the object can take at the predetermined position, the reliability value specifying unit 222 calculates the first reliability value and the second reliability value when a three-dimensional model imitating the shape is arranged at the predetermined position. Identify.

  The reliability value specifying unit 222 uses the area of the first region for specifying the first reliability value, and uses the area of the second region for specifying the second reliability value. For example, the reliability value specifying unit 222 increases the first reliability value as the area of the first region is larger, and increases the second reliability value as the area of the second region is larger.

  In addition, the confidence value specifying unit 222 uses the distance between the position of the first region and the center of the first image to specify the first confidence value, and uses the distance between the position of the second region and the center of the second image. Used to specify the second confidence value. For example, the reliability value specifying unit 222 decreases the first reliability value as the distance between the position of the first region and the center of the first image is longer, and determines the position of the second region and the center of the second image. The longer the distance is, the smaller the second confidence value is.

  Note that the region specifying unit 221 may virtually arrange the three-dimensional model and the second three-dimensional model imitating the second object not to be detected in the three-dimensional space. Here, the second three-dimensional model is a three-dimensional model corresponding to static objects such as chairs, tables, and shelves arranged in a predetermined area.

  The reliability value specifying unit 222 includes a second region in which the first region is included in the first image when the three-dimensional model and the second three-dimensional model are virtually arranged in the three-dimensional space indicating the predetermined area. The ratio hidden by the image area indicating the three-dimensional model may be used for specifying the first confidence value. Similarly, the confidence value specifying unit 222 may use the ratio of the second area hidden by the image area indicating the second three-dimensional model included in the second image for specifying the second confidence value.

  For example, the reliability value specifying unit 222 increases the first reliability value as the ratio of the first area hidden by the image area indicating the second three-dimensional model included in the first image is smaller. However, the smaller the ratio hidden by the image region indicating the second three-dimensional model included in the second image, the larger the second confidence value.

  In addition, the storage unit 21 includes first reliability value information that associates a position in the image of the first image with a first reliability value corresponding to the position, a position in the image of the second image, and the position. 2nd reliability value information which linked | related the 2nd reliability value corresponding to may be memorize | stored. The confidence value specifying unit 222 may specify the position of the three-dimensional model in the first image and may specify the position of the three-dimensional model in the second image. Then, the confidence value specifying unit 222 refers to the first confidence value information stored in the storage unit 21, and determines the first confidence value associated with the position of the image indicating the three-dimensional model included in the first image. get. Similarly, the confidence value specifying unit 222 refers to the second confidence value information stored in the storage unit 21 and is associated with the position of the image indicating the three-dimensional model included in the second image. Is identified. By doing in this way, the confidence value specifying unit 222 can easily specify the first confidence value and the second confidence value corresponding to the position of the three-dimensional model virtually arranged in the three-dimensional space.

  The confidence value specifying unit 222 performs the processing of the above-described procedure for each of a plurality of positions in a three-dimensional space indicating a predetermined area. The plurality of positions may be defined, for example, by dividing a predetermined area by dividing the predetermined area into lattices with a constant interval. Then, the confidence value specifying unit 222 specifies the first confidence value and the second confidence value of each of a plurality of shapes that can be taken by the object, corresponding to each of the plurality of positions.

  The selection unit 223 detects an object at a predetermined position out of the first imaging device 1A and the second imaging device 1B based on the ratio between the first reliability value and the second reliability value specified by the reliability value specification unit 222. Select the imaging device to use.

  Specifically, when the ratio of the first confidence value to the second confidence value corresponding to the predetermined position is equal to or higher than the first threshold, the selection unit 223 uses the imaging device used for detecting the object at the predetermined position. As a result, the first imaging device 1A is selected.

  In addition, when the ratio of the second confidence value to the first confidence value corresponding to the predetermined position is equal to or greater than the second threshold, the selection unit 223 is the second imaging device that is used to detect the object at the predetermined position. The imaging device 1B is selected. Here, in the present embodiment, the first threshold value and the second threshold value are the same threshold value (predetermined threshold value), but the first threshold value and the second threshold value may be different values. Good. By doing in this way, the selection apparatus 2 can select an imaging apparatus corresponding to an image having a relatively high reliability value.

  In addition, the selection unit 223 has a ratio of the first confidence value to the second confidence value corresponding to the predetermined position that is less than the first threshold value, and a second confidence value for the first confidence value corresponding to the predetermined position. When the ratio is less than the second threshold value, the first imaging device 1A and the second imaging device 1B are selected as imaging devices used for detecting an object at a predetermined position. In this way, when the difference between the reliability values of the two imaging devices is not large, the selection device 2 selects the two imaging devices and detects the object using the two imaging devices. can do.

  For each of a plurality of shapes that can be taken by the object at a predetermined position, the selection unit 223 is used for object detection based on the ratio between the first confidence value and the second confidence value corresponding to each of the plurality of shapes. Select an imaging device. In addition, the selection unit 223 selects an imaging device to be used for object detection based on the ratio between the first confidence value and the second confidence value for each of a plurality of positions in the three-dimensional space.

  Even if the area of the object is the same at a predetermined position, if the object exists in the vicinity of each imaging device, it is better to use the imaging device in the vicinity of the object as the imaging device used for detection. In many cases, the position of can be detected with high accuracy. Therefore, when the predetermined position is within the first distance from the first imaging device 1A, the selection unit 223 detects an object at the predetermined position regardless of the ratio between the first reliability value and the second reliability value. The first imaging device 1A may be selected as the imaging device to be used. In addition, when the predetermined position is within the second distance from the second imaging device 1B, the selection unit 223 detects an object at the predetermined position regardless of the ratio between the first reliability value and the second reliability value. The second imaging device 1B may be selected as the imaging device to be used. In this way, the selection device 2 can select an imaging device that can detect the position of the object with high accuracy.

  The storage control unit 224 includes device identification information for identifying the imaging device, information indicating the shape of the object, position information indicating a position where the imaging device is selected for object detection, When the object is arranged, the storage unit 21 stores the information as the assigned area information in association with the information indicating the area of the image indicating the object included in the image captured by the imaging apparatus.

  Here, the position selected to be used for object detection is represented by position coordinates in the global coordinate system. The information indicating the area of the image indicating the object is information indicating the upright rectangular area, and is the first area or the second area. Note that the information indicating the area of the image indicating the object is represented by the information indicating the upright rectangular area when matching with the upright rectangle in the object tracking process in the subsequent stage. When matching is performed, it is expressed by similar information. The storage control unit 224 causes the storage unit 21 to store assigned area information corresponding to each of a plurality of shapes that can be taken by the object. The assigned area information is transmitted to the object tracking device 3 by the control unit 22, for example. The assigned area information may be provided separately for each of the plurality of shapes, or only one may be provided for each of the plurality of shapes.

  FIG. 4 is a diagram illustrating an imaging device selected at each of a plurality of positions in the three-dimensional space. FIG. 4A is a diagram illustrating an imaging device selected at each of a plurality of positions when the shape of the object is a shape indicating an upright state. FIG. 4B is a diagram illustrating an imaging device selected at each of a plurality of positions when the shape of the object is a shape indicating a seated state.

  An area A1 illustrated in FIG. 4 is an area where the first imaging apparatus 1A is selected as an imaging apparatus used for object detection, and an area A2 is a first imaging apparatus 1A and a second imaging apparatus as imaging apparatuses used for object detection. The region in which the device 1B is selected, the region A3, is a region in which the second imaging device 1B is selected as the imaging device used for object detection.

  As shown in FIGS. 4A and 4B, it can be confirmed that the assigned area is different between the case where the object is in the upright state and the case where the object is in the seated state. In FIG. 4A, in the vicinity of the first imaging device 1A, the first imaging device 1A and the second imaging device 1B are selected as the imaging devices used for object detection. This is because if the three-dimensional model is too close to the first imaging device 1A, only the upper part of the three-dimensional model M is reflected in the first image captured by the first imaging device 1A, and the area of the first region is reduced. is there.

That is, from the viewpoint of the size of the area, the superiority or inferiority of the first imaging device 1A and the second imaging device 1B cannot be attached as an imaging device used for detecting an object that exists immediately below the first imaging device 1A. However, even if the area on the image is the same, if the object exists in the vicinity of each imaging device, the position of the object can be detected with higher accuracy by using the imaging device as the imaging device used for detection. There are many cases. Therefore, after the selection unit 223 determines a responsible region as shown in FIGS. 4A and 4B according to the size of the area, the selection unit 223 determines a certain distance from a neighboring region of each imaging device, for example, a position directly below the imaging device. May be redefined (overwritten) so that the imaging device is selected.
Further, in FIG. 4B, in the action area A, there are areas that are not set as the areas A1 to A3. This area is an area where the object does not take a seated state.

[Configuration of Object Tracking Device 3]
Next, the configuration of the object tracking device 3 will be described. FIG. 5 is a diagram illustrating a configuration of the object tracking device 3 according to the present embodiment. As shown in FIG. 5, the object tracking device 3 includes a storage unit 31 and a control unit 32.

The storage unit 31 is a storage medium including a ROM and a RAM. The storage unit 31 stores a program executed by the control unit 32. For example, the storage unit 31 stores an object tracking program that causes the object tracking device 3 to function as the image acquisition unit 321, the object detection unit 322, and the object tracking unit 323.
Further, it is assumed that responsible area information generated by the selection device 2 is stored in the storage unit 31 in advance.

  The control unit 32 is a CPU, for example. The control unit 32 functions as an image acquisition unit 321, an object detection unit 322, and an object tracking unit 323 by executing the object tracking program stored in the storage unit 31.

  The image acquisition unit 321 acquires an omnidirectional image captured by the first imaging device 1A and also acquires an omnidirectional image captured by the second imaging device 1B. The omnidirectional image includes information indicating the imaging time.

  The object detection unit 322 detects an object included in the omnidirectional image by analyzing the omnidirectional image acquired by the image acquisition unit 321. For example, the object detection unit 322 specifies a contour region indicating the contour of the object included in the omnidirectional image. Then, the object detection unit 322 detects the object by specifying information indicating the upright rectangular area based on the specified outline area.

  The object detection unit 322 performs matching between information indicating the identified upright rectangular area and information indicating the upright rectangular area included in the assigned area information. For example, the object detection unit 322 determines the overlap rate (IOU) between the upright rectangular area indicated by the information indicating the specified upright rectangular area and the upright rectangular area indicated by the information indicating the plurality of upright rectangular areas included in the assigned area information. : Intersection over Union). And the object detection part 322 specifies the thing with the highest duplication rate among the information which shows the upright rectangular area contained in charge area information.

  The object detection unit 322 identifies position information associated with information indicating an upright rectangular area having the highest overlap rate, and sets the position indicated by the position information as the position of the object. Then, the object detection unit 322 associates information indicating the imaging time included in the omnidirectional image including the object with the position information, and causes the storage unit 31 to store the information as object position information. The object detection unit 322 specifies the position of the object based on the above-described procedure for each of the acquired plurality of omnidirectional images, and stores the object position information in the storage unit 31.

  The object tracking unit 323 tracks an object based on the object position information stored in the storage unit 31. For example, the object tracking unit 323 tracks an object by setting the objects whose position indicated by the position information is within a predetermined distance to the same object at two consecutive times.

[Processing Flow in Selection Device 2]
Next, the flow of processing in the selection device 2 will be described. FIG. 6 is a flowchart showing the flow of processing in the selection device 2 according to this embodiment.

First, the area specifying unit 221 selects an unselected position among a plurality of positions included in the action area (S1).
Subsequently, the region specifying unit 221 selects an unselected shape among the shapes that the object can take at the selected position (S2).

Subsequently, the area specifying unit 221 specifies the first area and the second area when the three-dimensional model corresponding to the shape selected in S2 is arranged at the position selected in S1 (S3).
Subsequently, the confidence value identification unit 222 identifies the first confidence value based on the first region and identifies the second confidence value based on the second region (S4). Here, the confidence value specifying unit 222 may specify the first confidence value and the second confidence value based on information other than the first region and the second region.

  Subsequently, the selection unit 223 determines whether or not the ratio of the first confidence value to the second confidence value is greater than or equal to the first threshold value (S5). If the selection unit 223 determines that the value is equal to or greater than the first threshold, the process proceeds to S6. If the selection unit 223 determines that the value is less than the first threshold, the process proceeds to S7.

  In S6, the selection unit 223 selects the first imaging device 1A as the imaging device used for detecting the object corresponding to the shape selected in S2 at the position selected in S1. When the process of S6 ends, the selection unit 223 moves the process to S10.

  Subsequently, the selection unit 223 determines whether or not the ratio of the second confidence value to the first confidence value is greater than or equal to the second threshold value (S7). If the selection unit 223 determines that the value is equal to or greater than the second threshold, the process proceeds to S8. If the selection unit 223 determines that the value is less than the second threshold, the process proceeds to S9.

  In S8, the selection unit 223 selects the second imaging device 1B as the imaging device used for detecting the object corresponding to the shape selected in S2 at the position selected in S1. When the process of S8 ends, the selection unit 223 moves the process to S10.

  In S9, the selection unit 223 selects the first imaging device 1A and the second imaging device 1B as the imaging devices used for detecting the object corresponding to the shape selected in S2 at the position selected in S1.

  Subsequently, the storage control unit 224 includes position information indicating the position selected in S1, shape information indicating the shape selected in S2, device identification information for identifying the imaging device selected by the selection unit 223, When an object is arranged at a position, information indicating an area of the image indicating the object included in an image captured by the imaging device (information indicating an upright rectangle) is associated and stored in the storage unit 21 as responsible area information. (S10).

  Subsequently, the region specifying unit 221 determines whether or not all shapes that can be taken by the object have been selected (S11). If the area specifying unit 221 determines that all the shapes have been selected, the process proceeds to S12. If the area specifying unit 221 determines that all the shapes are not selected, the process proceeds to S2.

  Subsequently, the region specifying unit 221 determines whether all positions included in the action area have been selected (S12). If the area specifying unit 221 determines that all positions have been selected, the process according to this flowchart ends. If the area specifying unit 221 determines that all positions have not been selected, the process proceeds to S1.

[Flow of Processing in Object Tracking Device 3]
Next, the flow of processing in the object tracking device 3 will be described. FIG. 7 is a flowchart showing the flow of processing in the object tracking device 3 according to this embodiment. Note that at the start of this flowchart, the storage unit 31 of the object tracking device 3 stores the assigned area information generated by the selection device 2.

  First, the image acquisition unit 321 acquires a plurality of omnidirectional images from the first imaging device 1A and the second imaging device 1B (S21). Here, it is assumed that the imaging times included in the omnidirectional images captured by the first imaging device 1A and the second imaging device 1B are synchronized.

  Subsequently, the object detection unit 322 selects an omnidirectional image with the oldest imaging time among unselected omnidirectional images (S22). Here, the object detection unit 322 selects two omnidirectional images captured by the first imaging device 1A and the second imaging device 1B as omnidirectional images having the oldest imaging time.

Subsequently, the object detection unit 322 performs image analysis of the selected omnidirectional image, and specifies an upright rectangular area of the object included in the omnidirectional image (S23).
Subsequently, the object detection unit 322 overlaps the upright rectangular area of the identified object with the upright rectangular area indicated by each of the plurality of upright rectangular area information included in the assigned area information stored in the storage unit 31. The rate is calculated (S24).

  Subsequently, the object detection unit 322 specifies the position indicated by the position information associated with the upright rectangular area having the highest overlap rate in the assigned area information as the position of the object (S25).

  Subsequently, the object tracking unit 323 tracks the object based on the position of the object specified in S25 (S26).

  Subsequently, the object detection unit 322 determines whether all omnidirectional images have been selected (S27). If the object detection unit 322 determines that all the omnidirectional images have been selected, the object detection unit 322 ends the process according to this flowchart. If the object detection unit 322 determines that all the omnidirectional images have not been selected, the process proceeds to S22.

[Effect in this embodiment]
As described above, the selection device 2 according to the present embodiment virtually arranges a three-dimensional model imitating an object to be detected in a predetermined area at a predetermined position in a three-dimensional space indicating the predetermined area. In this case, a first area indicating an area of the image indicating the three-dimensional model included in the first image captured by the first image capturing apparatus 1A that captures the predetermined area, and a second image capturing apparatus 1B capturing the predetermined area. The first region indicating the reliability of the detection when the second region indicating the region of the image indicating the three-dimensional model included in the second image is identified and the object is detected based on the first region A value and a second confidence value indicating a reliability value of the detection when the object is detected based on the second region are specified. And the selection apparatus 2 uses the imaging device used for the detection of the object in a predetermined position among 1st imaging device 1A and 2nd imaging device 1B based on the ratio of a 1st reliability value and a 2nd reliability value. select. By doing in this way, the selection apparatus 2 can select the imaging device which images the image which can detect an object accurately.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment.

  In particular, the specific embodiments of the distribution / integration of the devices are not limited to those illustrated above, and all or a part thereof may be arbitrarily selected according to various additions or according to the functional load. It can be configured to be functionally or physically distributed and integrated in units.

  For example, in the above embodiment, the selection device 2 and the object tracking device 3 are different devices, but the present invention is not limited to this. The selection device 2 and the object tracking device 3 may be realized by a single device.

DESCRIPTION OF SYMBOLS 1A ... 1st imaging device, 1B ... 2nd imaging device, 2 ... Selection device, 21 ... Memory | storage part, 22 ... Control part, 221 ... Area | region specific | specification part, 222 ... Trust value identification unit, 223 ... selection unit, 224 ... storage control unit, 3 ... object tracking device, 31 ... storage unit, 32 ... control unit, 321 ... image acquisition unit 322, object detection unit, 323, object tracking unit, S, object tracking system

Claims (15)

When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that captures the predetermined area captures an image A first region indicating an image region indicating the three-dimensional model included in the first image, and an image indicating the three-dimensional model included in the second image captured by the second imaging device that captures the predetermined area. An area specifying unit for specifying the second area indicating the area;
A first reliability value indicating the reliability of the detection when the object is detected based on the first area, and a reliability value of the detection when the object is detected based on the second area A confidence value identifying unit that identifies a second confidence value indicating
A selection unit that selects, from the first imaging device and the second imaging device, an imaging device to be used for detecting the object at the predetermined position based on a ratio between the first reliability value and the second reliability value. When,
A selection device comprising: The reliability value specifying unit uses the area of the first region for specifying the first reliability value, and uses the area of the second region for specifying the second reliability value.
The selection device according to claim 1. The first imaging device or the second imaging device is an imaging device that captures an omnidirectional image,
The confidence value specifying unit uses the distance between the position of the first region and the center of the first image for specifying the first confidence value, and the position of the second region and the center of the second image. Is used to specify the second confidence value,
The selection device according to claim 1 or 2. In the case where the three-dimensional model and the second three-dimensional model imitating the second object that is not the target to be detected are virtually arranged, the first value in the confidence value specifying unit is the first area. A ratio hidden by an image region indicating the second three-dimensional model included in one image is used for specifying the first confidence value, and the second region is included in the second image. The ratio hidden by the image area indicating the model is used to specify the second confidence value.
The selection device according to any one of claims 1 to 3. Information associating a position in the image of the first image with a first confidence value corresponding to the position, a position in the image of the second image, and a second confidence value corresponding to the position A storage unit for storing the received information,
The confidence value specifying unit refers to the storage unit, acquires the first confidence value associated with the position of the image indicating the three-dimensional model included in the first image, and adds the first confidence value to the second image. Identifying the first confidence value and the second confidence value by obtaining the second confidence value associated with a position of an image indicating the included three-dimensional model;
The selection device according to claim 1. The selection unit is an imaging device used for detecting the object at the predetermined position when a ratio of the first reliability value to the second reliability value corresponding to the predetermined position is greater than or equal to a first threshold value. Used to detect the object at the predetermined position when the first imaging device is selected and the ratio of the second reliability value to the first reliability value corresponding to the predetermined position is greater than or equal to a second threshold value. Selecting the second imaging device as an imaging device to perform,
The selection device according to any one of claims 1 to 5. The selection unit is configured such that a ratio of the first reliability value to the second reliability value corresponding to the predetermined position is less than the first threshold, and the ratio to the first reliability value corresponding to the predetermined position. When the ratio of the second confidence value is less than the second threshold value, the first imaging device and the second imaging device are selected as imaging devices used for detecting the object at the predetermined position.
The selection device according to any one of claims 1 to 6. When the predetermined position is within a first distance from the first imaging device, the selection unit is configured to detect the object at the predetermined position regardless of a ratio between the first reliability value and the second reliability value. When the first imaging device is selected as an imaging device to be used for detection of and the predetermined position is within a second distance from the second imaging device, the first confidence value and the second confidence value Regardless of the ratio, the second imaging device is selected as the imaging device used for detecting the object at the predetermined position.
The selection device according to any one of claims 1 to 7. The confidence value specifying unit specifies the first confidence value and the second confidence value when the three-dimensional model is virtually arranged for each of a plurality of positions in the three-dimensional space,
The selection unit selects an imaging device to be used for detection of the object based on a ratio between the first confidence value and the second confidence value for each of a plurality of positions in the three-dimensional space.
The selection device according to any one of claims 1 to 8. A storage control unit that associates and stores device identification information for identifying the imaging device and position information indicating a position selected to use the imaging device for detection of the object in the storage unit;
The selection device according to any one of claims 1 to 9. The reliability value specifying unit, for each of a plurality of shapes corresponding to the object at the predetermined position, the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position and the Identify a second confidence value,
The selection unit selects an imaging device to be used for detection of the object based on a ratio between the first confidence value and the second confidence value for each of a plurality of shapes corresponding to the object at the predetermined position. To
The selection device according to any one of claims 1 to 10. The reliability value specifying unit, for each of one or more shapes that can be taken by the object at the predetermined position, the first reliability value when the three-dimensional model imitating the shape is arranged at the predetermined position Identifying the second confidence value;
The selection device according to claim 11. A storage control unit that associates and stores device identification information for identifying an imaging device, the shape of the object, and position information indicating a position selected to use the imaging device for detection of the object in a storage unit; In addition,
The selection device according to claim 11 or 12. The computer runs,
When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that captures the predetermined area captures an image A first region indicating an image region indicating the three-dimensional model included in the first image, and an image indicating the three-dimensional model included in the second image captured by the second imaging device that captures the predetermined area. Identifying a second region indicative of the region;
A first reliability value indicating the reliability of the detection when the object is detected based on the first area, and a reliability value of the detection when the object is detected based on the second area Identifying a second confidence value indicative of
Selecting an imaging device to be used for detection of the object at the predetermined position from the first imaging device and the second imaging device based on a ratio between the first confidence value and the second confidence value; ,
A selection method comprising: Computer
When a three-dimensional model imitating an object to be detected in the predetermined area is virtually arranged at a predetermined position in a three-dimensional space indicating the predetermined area, the first imaging device that captures the predetermined area captures an image A first region indicating an image region indicating the three-dimensional model included in the first image, and an image indicating the three-dimensional model included in the second image captured by the second imaging device that captures the predetermined area. An area specifying unit for specifying the second area indicating the area;
A first reliability value indicating the reliability of the detection when the object is detected based on the first area, and a reliability value of the detection when the object is detected based on the second area A confidence value identifying unit that identifies a second confidence value indicating
A selection unit that selects, from the first imaging device and the second imaging device, an imaging device to be used for detecting the object at the predetermined position based on a ratio between the first reliability value and the second reliability value. ,
Selection program to function as.

Images