JPWO2020013021A1 - Detection device, processing device, detection method, and processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detection device capable of distinguishing a first object and a second object in contact with each other. A detection device includes a position detection unit that detects position information of each point on an object including a first object and a second object in a target region, and an object surface of the first object based on the detection result of the position detection unit. And an object identification unit that identifies at least a part of the second object in contact with the object surface with respect to the object surface based on the distance to each point on the object detected by the position detection unit.

Description

The present invention relates to a detection device, a processing device, a detection method, and a processing program.

As a technique for detecting an object, for example, there is a technique described in Patent Document 1 below. The object to be detected may include a first object and a second object that are in contact with each other. In such a case, for example, it is desired that the first object and the second object can be identified and detected. For example, when detecting a human body on the floor surface, it is desired that the human body in contact with the floor surface can be identified from the floor surface and can be detected with high accuracy.

Japanese Unexamined Patent Publication No. 2010-134546

According to the aspect of the present invention, a position detection unit that detects the position information of each point on the object including the first object and the second object in the target region, and an object surface of the first object based on the detection result of the position detection unit. A detection device including the object identification unit that identifies at least a part of the second object in contact with the object surface with respect to the object surface based on the distance to each point on the object detected by the position detection unit. Provided.

According to the aspect of the present invention, the object surface of the first object based on the detection result of detecting the position information of each point on the object including the first object and the second object in the target region, and each point on the object. Provided is a processing device including an object identification unit that identifies at least a part of a second object in contact with the object surface with respect to the object surface based on the distance.

According to the mode of the present invention, the position information of each point on the object including the first object and the second object in the target region is detected, and the object surface of the first object based on the detection result and the object on the object. A detection method is provided that includes identifying at least a portion of a second object in contact with an object surface with respect to the object surface based on the distance to each point.

According to the aspect of the present invention, the object surface of the first object based on the detection result of detecting the position information of each point on the object including the first object and the second object in the target region by the computer, and each on the object. A processing program is provided that executes identification of at least a part of the second object in contact with the object surface with respect to the object surface based on the distance from the point.

It is a figure which shows the detection apparatus which concerns on 1st Embodiment. It is a figure which shows the position detection part which concerns on 1st Embodiment. It is a figure which shows the point cloud processing which concerns on 1st Embodiment. It is a figure which shows the plane estimation process and object identification process which concerns on 1st Embodiment. It is a flowchart which shows the detection method which concerns on 1st Embodiment. It is a flowchart which shows the plane estimation process which concerns on 1st Embodiment. It is a flowchart which shows the object identification process which concerns on 1st Embodiment. It is a figure which shows the detection apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the detection method which concerns on 2nd Embodiment. It is a figure which shows the object identification process which concerns on 3rd Embodiment. It is a figure which shows the object identification process which concerns on 4th Embodiment. It is a figure which shows the detection apparatus which concerns on 5th Embodiment. It is a figure which shows the plane estimation process which concerns on 5th Embodiment. It is a figure which shows the detection apparatus which concerns on 6th Embodiment. It is a figure which shows the detection apparatus which concerns on 7th Embodiment. It is a figure which shows the detection apparatus which concerns on 8th Embodiment.

[First Embodiment]
The first embodiment will be described. FIG. 1 is a diagram showing a detection device according to an embodiment. The detection device 1 detects the object M in the target area AR. The target area AR is an area (eg, a detection area of the detection device 1 and a visual field) to be detected by the detection device 1. In FIG. 1, the object M includes a first object M1 and a second object M2. The first object M1 includes, for example, an object fixed in the target area AR (eg, a stationary object such as a floor, a wall, a road, a staircase, a ground, a building, or a structure). The first object M1 has a surface SF (eg, a floor surface, a wall surface, a ceiling surface, a ground surface, a road surface, a slope, a curved surface having a curvature such as a concave surface or a convex surface) which is an example of an object surface.

The second object M2 includes an object (eg, a moving object) that is separable (eg, movable) with respect to the first object M1. The second object M2 includes, for example, a human body, an animal, or a robot. The second object M2 can come into contact with the surface SF of the first object M1. For example, the second object M2 moves from outside the target area AR into the target area AR and comes into contact with the surface SF of the first object M1. Further, the second object M2 is in a state of not contacting the surface SF (eg, a state of being placed outside the target area AR) and a state of being in contact with the surface SF (eg, a state of being placed in the target area AR). Any of the states can be taken.

The detection device 1 (detection system) includes a position detection unit 2 and a processing device 3. The position detection unit 2 detects the position information (eg, depth) of each point on the object M including the first object M1 and the second object M2 in the target area AR. The position detection unit 2 inputs a signal (input signal) to the object M, and detects a signal (output signal) output from the object M. The above signals (input signal, output signal) include, for example, an energy wave selected from light (visible light, invisible light, infrared light, ultraviolet light) and sound waves (eg, ultrasonic waves).

The position detection unit 2 includes a detection unit 4 (eg, a depth sensor, a depth camera, a distance measuring unit). The position detection unit 2 (detection unit 4) detects the depth (distance, depth, depth) from a predetermined point to each point on the object M. The predetermined point is, for example, a point at a position that serves as a reference for detection by the position detection unit 2 (eg, a viewpoint, a point at the detection source, a point at the position of the position detection unit 2).

FIG. 2 is a diagram showing a position detection unit 2 according to the first embodiment. The position detection unit 2 (detection unit 4) includes an irradiation unit 5, an optical system 6, and an image pickup device 7. The irradiation unit 5 irradiates (eg, projects) light La (eg, pattern light, irradiating light, pulsed light) on the detection target area AR (space, detection area). Light La includes, for example, infrared light. The optical system 6 includes, for example, an imaging optical system (imaging optical system). The image sensor 7 includes, for example, a CMOS image sensor or a CCD image sensor. The image pickup device 7 has a plurality of pixels arranged two-dimensionally. The image sensor 7 images the target region AR including the object M via the optical system 6. The image sensor 7 detects light Lb (infrared light, return light) emitted by irradiation of light La from an object (object M) in the target region AR. The light Lb includes, for example, infrared light.

The detection unit 4 is based on, for example, a pattern of light La emitted from the irradiation unit 5 (eg, intensity distribution) and a pattern of light Lb detected by the image sensor 7 (intensity distribution, captured image). The depth from the point on the target region AR corresponding to each pixel of 7 to each pixel of the image sensor 7 is detected. As the detection result (area position information), the position detection unit 2 transmits a depth map (eg, depth image, depth information, distance information) showing the depth distribution in the target area AR to the processing device 3 (see FIG. 1). Output.

The position detection unit 2 may detect a moving object (eg, a moving object, a second object M2) in the target area AR. In this case, the detection unit 4 may repeat the detection of the object at a predetermined sampling frequency. The detection unit 4 may generate position information (eg, depth map) at each detection each time the detection is executed. The position detection unit 2 may detect the movement of the object based on the time change of the position information of the object generated by the detection unit 4. Further, the position detection unit 2 may generate position information by calculating a plurality of detection results (eg, calculation of an average value) having different detection times by the detection unit 4.

The position information detection method by the position detection unit 2 may be passive measurement or active measurement, and is arbitrarily set. The position detection unit 2 may be a device that detects the depth by the TOF (time of flight) method. Further, the position detection unit 2 may be a device that detects the depth by a method other than the TOF method. The position detection unit 2 may include, for example, a laser scanner (eg, a laser rangefinder) and may be a device that detects the depth by laser scanning. The position detection unit 2 may be, for example, a device including a phase difference sensor and detecting the depth by the phase difference method. The position detection unit 2 may be, for example, a device that detects the depth by the DFD (depth from defocus) method.

The position detection unit 2 may irradiate the object M with light other than infrared light (eg, visible light) and detect the light emitted from the object M (eg, visible light). The position detection unit 2 may include, for example, a stereo camera or the like, and may detect (eg, image) an object M from a plurality of viewpoints. The position detection unit 2 may be a device that detects the depth by triangulation using captured images obtained by capturing the object M from a plurality of viewpoints. The position detection unit 2 may detect the depth by a method other than the optical method (eg, scanning by ultrasonic waves). The position detection unit 2 includes a first detection unit (eg, detection unit 4) that detects the distance between each point on the object M and a predetermined point, and a second detection unit (eg, visible) that detects the texture of the object M. An imaging unit) that detects light) may be provided.

Returning to the description of FIG. 1, the processing device 3 is an information processing device that processes information (eg, detection result) output by the position detection unit 2. The processing device 3 includes a model generation unit 11, a surface estimation unit 12, an object identification unit 13, and a storage unit 14. The storage unit 14 is, for example, a non-volatile memory, a hard disk (HDD), a solid state drive (SSD), or the like. The storage unit 14 stores the information output from the position detection unit 2. Further, the storage unit 14 stores the information processed by each unit of the processing device 3.

The model generation unit 11 generates three-dimensional model information using the detection result of the position detection unit 2. For example, the model generation unit 11 performs computer graphic processing (CG processing) on at least a part of the object in the target area AR to calculate model information (eg, three-dimensional CG model data). The model information includes the shape information of the object in the target area AR. The model information may include the texture information of the object in the target area AR.

The model generation unit 11 includes a point cloud data generation unit 15 and a surface information generation unit 16. The point cloud data generation unit 15 executes a point cloud process that generates point cloud data as position information of each point on the object M. The point cloud data includes the three-dimensional coordinates of a plurality of points on the object M (eg, the first object M1 and the second object M2) in the target area AR. The point cloud data generation unit 15 calculates the point cloud data of the object M based on the position information (eg, depth) detected by the position detection unit 2. The point cloud data generation unit 15 reads out the detection result of the position detection unit 2 stored in the storage unit 14 and calculates the point cloud data. The point cloud data generation unit 15 may generate point cloud data as shape information of the object M.

FIG. 3 is a diagram showing point cloud processing by the point cloud data generation unit according to the first embodiment. Reference numeral DM1 is a depth map (depth image) corresponding to the detection result of the position detection unit 2. The depth map DM1 is information (eg, an image) representing the spatial distribution of the depth measurement values by the position detection unit 2. The depth map DM1 is a grayscale image in which the depth at each point of the target area AR is represented by a gradation value. In the depth map DM1, a portion having a relatively high gradation value (eg, a white portion, a bright portion) is a portion having a relatively small depth (a portion relatively close to the position detection unit 2). In the depth map DM1, a portion having a relatively low gradation value (eg, a black portion, a dark portion) is a portion having a relatively large depth (a portion relatively far from the position detection unit 2).

The point cloud data generation unit 15 (see FIG. 1) reads the data of the depth map DM1 from the storage unit 14, and corresponds to each pixel based on the gradation value (measured value of depth) of each pixel of the depth map DM1. The three-dimensional coordinates of the points in the real space are calculated, and the point cloud data PD is generated. In the following description, the three-dimensional coordinates of one point are appropriately referred to as point data. The point cloud data PD is data in which a plurality of point data are set as a set.

Reference numeral PD in FIG. 3 represents point group data corresponding to a plane (eg, floor surface) as the surface SF of the first object M1 and the second object M2 (eg, feet and hands of the human body). Here, the point cloud data PD is conceptually represented by arranging circles at the positions of each point instead of the point data. Further, for convenience of explanation, the point P1 in the first object M1 is represented by a black circle, and the point P2 in the second object M2 is represented by a white circle.

In the following description, the processing result obtained by subjecting the detection result (eg, the measured value of depth) of the position detection unit 2 to the above point cloud processing is referred to as the first point cloud data. The first point cloud data includes point cloud data corresponding to the first object M1 and point cloud data corresponding to the second object M2. In the first point cloud data, the point cloud data corresponding to the second object M2 is not distinguished from the point cloud data corresponding to the first object M1. For example, in the first point cloud data, the first object M1 and the second object M2 are not distinguished, and whether or not the point P2 on the second object M2 belongs to the first object M1 is not identified.

In the portion where the first object M1 and the second object M2 are in contact with each other, the point P1 on the first object M1 and the point P2 on the second object M2 are continuously distributed. In the portion where the first object M1 and the second object M2 are in contact with each other (including, for example, a boundary portion, an overlapping portion, etc.), the coordinates of the point P1 have a small difference from the coordinates of the point P2. Therefore, it may be difficult for the second object M2 to identify the contact area with respect to the first object M1 in the portion (contact area) in contact with the first object M1.

The detection device 1 according to the embodiment has an object surface estimation process (hereinafter referred to as a plane estimation process) that estimates (approximates) the surface SF of the first object M1 as a plane, and a surface SF estimated by the plane estimation process. The object identification process for identifying (distinguishing, classifying, segmenting, segmenting) the second object M2 is executed. Hereinafter, the plane estimation process and the object identification process will be described with reference to FIGS. 1 and 4. FIG. 4 is a diagram showing a plane estimation process by the surface estimation unit and an object identification process by the object identification unit according to the first embodiment. The reference numeral PD1 in FIG. 4 corresponds to the first point cloud data, and the reference numeral P3 corresponds to the point data included in the first point cloud data.

The surface estimation unit 12 estimates the plane of the first object M1 based on the detection result of the position detection unit 2. The surface estimation unit 12 estimates the surface SF (see FIG. 1) of the first object M1 as a plane (approximate by a plane). In the following description, the plane estimated by the surface estimation unit 12 is appropriately referred to as an estimation plane. The reference numeral FP in FIG. 4 corresponds to an estimation plane. The surface estimation unit 12 generates information representing the estimated plane FP (hereinafter, referred to as estimated plane information). The surface estimation unit 12 stores the estimated plane information in the storage unit 14.

The surface estimation unit 12 estimates the above-mentioned plane so as to satisfy a preset calculation condition, for example. The above calculation conditions are stored in the storage unit 14, and the surface estimation unit 12 reads out the calculation conditions stored in the storage unit 14 and executes the plane estimation process. The above calculation condition includes, for example, a condition in which the angle with respect to the vertical direction is within a predetermined range. In the position information (eg, point cloud data) of the object obtained from the detection result of the position detection unit 2, the direction corresponding to the vertical direction in the real space is preset based on the detection direction of the position detection unit 2. The surface estimation unit 12 derives, for example, a plane (eg, a horizontal plane) satisfying the condition that the angle with respect to the vertical direction is within a predetermined range (eg, 80 ° or more and 90 ° or less) as an estimation plane.

The calculation condition of the estimated plane FP may include a condition other than the angle with respect to the vertical direction. For example, the calculation condition of the estimated plane FP may include a condition that the area is equal to or larger than a predetermined value. In this case, the surface estimation unit 12 derives a candidate for the estimation plane FP based on the point cloud data, and when the area of this candidate is equal to or larger than a predetermined value, adopts this candidate as the estimation plane (eg, determination, identification). ) May. The surface estimation unit 12 does not have to adopt this candidate as the estimation plane when, for example, the scale of the candidate of the estimation plane is smaller than the scale of the assumed second object M2 (eg, human body). The above predetermined value may be given, for example, as a ratio to the size of the target region AR (eg, the field of view of the position detection unit 2).

Further, the surface estimation unit 12 estimates the above plane based on the point cloud data generated by the point cloud data generation unit 15. The surface estimation unit 12 estimates a plane using the three-dimensional coordinates of a plurality of points selected from the point cloud data. The surface estimation unit 12 estimates a plane using, for example, the least squares method. In this case, the plane estimation unit 12 uses the plane that minimizes the sum of squares of the distances from the selected plurality of points as the estimation plane FP. The surface estimation unit 12 derives a mathematical formula representing the estimated plane FP. The surface estimation unit 12 calculates the coefficients (a, b, c, f) in the mathematical formula (eg, ax + by + cz + d = 0) representing the estimated plane FP. The surface estimation unit 12 stores the coefficient of the mathematical formula representing the estimated plane FP in the storage unit 14 as the estimated plane information.

The plane estimation unit 12 may estimate the plane by using an estimation method other than the least squares method. For example, the surface estimation unit 12 may estimate the plane using at least one estimation method of the M estimation method, RAMSAC, minimum median method, and Thompson method. Further, the surface estimation unit 12 may determine the estimated plane FP by comparing the plane estimated by the first estimation method with the plane estimated by the second estimation method.

Further, the surface estimation unit 12 may select all of the plurality of points included in the first point cloud data and estimate the plane using the selected points. Further, the surface estimation unit 12 may select a part (eg, subset) of a plurality of points included in the first point cloud data and estimate a plane using the selected points. The condition for the surface estimation unit 12 to select a plurality of points may be fixed or variable.

The surface estimation unit 12 may adjust the conditions for selecting a plurality of points. For example, the surface estimation unit 12 may update the selection of a plurality of points based on the residuals between the selected plurality of points and the estimated plane, and estimate the above plane based on the updated plurality of points. good. The above residuals may be, for example, the root mean square (RMS) of the distance between each selected point and the estimated plane. For example, the surface estimation unit 12 estimates the first plane as a candidate for the estimation plane based on the first group including a plurality of points. Further, the surface estimation unit 12 may select a second group in which points having a relatively large distance from the first plane in the first group are excluded from the first group. The surface estimation unit 12 may estimate the second plane as a candidate for the estimation plane based on the second group. The surface estimation unit 12 may update the selection of a plurality of points until the residual is equal to or less than the threshold value, and adopt the obtained plane as the estimation plane.

The object identification unit 13 makes at least a part of the second object M2 (eg, a portion near the above-mentioned plane) in contact with the plane of the object M (eg, a surface SF having no or less unevenness) a plane (eg, a surface SF). ) To identify (identify, extract). The object identification unit 13 identifies (segments, segments) at least a part of the second object M2 in the target area AR. The object identification unit 13 executes the object identification process based on the distance between each point on the object M detected by the position detection unit 2 and the estimated plane FP estimated by the surface estimation unit 12. The object identification unit 13 excludes each point on the object M whose distance from the estimated plane FP estimated by the surface estimation unit 12 is less than the threshold value (eg, less than 1 mm and less than 1 cm), and the position of the second object M2. Identify (extract, identify) information (eg, point data, point group data).

In FIG. 4, reference numeral FP1 and reference numeral FP2 are planes in which the distance D with respect to the estimated plane FP is the threshold value. The plane FP1 is arranged on the first side (eg, vertically upward side) with respect to the estimated plane FP. The plane FP2 is arranged on the second side (eg, vertically downward) opposite to the first side with respect to the estimated plane. The object identification unit 13 excludes (eg, deletes) the point data P4 (indicated by the dotted line) of the points arranged between the plane FP1 and the plane FP2 from the point cloud data of the second object M2 by, for example, masking processing. .. In this way, the object identification unit 13 generates the second point cloud data PD2 in which the point data P4 whose distance from the estimated plane FP is less than the threshold value is excluded from the first point cloud data PD1. The second point cloud data PD2 is a plurality of point data obtained by removing the set of point data P4 from the set of point data P3. Further, the second point cloud data PD2 is a plurality of point data including the point data P5 among the point data P4 and the point data P5 described later.

The object identification unit 13 identifies whether or not each point included in the first point cloud data belongs to the second object M2. The object identification unit 13 selects a point to be identified from the first point cloud data. The point to be identified is a point arbitrarily selected from the first point cloud data. For example, a number (eg, 1, 2, 3, ...) Is assigned to each point included in the first point cloud data, and the object identification unit 13 assigns points from the first point cloud data in numerical order. select. Then, the object identification unit 13 calculates the distance between the estimated plane FP and the selected point to be identified. For example, the object identification unit 13 reads the estimated plane information (eg, the coefficient of the mathematical formula representing the estimated plane FP) from the storage unit 14 and calculates the distance between the estimated plane represented by the estimated plane information and the selected point to be identified. ..

The object identification unit 13 compares the calculated distance with the threshold value and executes the object identification process. The object identification unit 13 determines whether or not the point to be identified belongs to the first object M1 (whether or not it is a point on the first object M1). For example, when the distance between the point to be identified and the estimated plane FP is less than the threshold value, the object identification unit 13 determines that the point to be identified belongs to the first object M1. In FIG. 4, the point data P4 corresponds to the point data determined to belong to the first object M1 by the object identification process.

Further, when the distance between the point to be identified and the estimated plane is equal to or greater than the threshold value, the object identification unit 13 determines that the point to be identified does not belong to the first object M1. In FIG. 4, the reference numeral P5 corresponds to the point data determined not to belong to the first object M1 by the object identification process. The point data P5 corresponds to the point data of a point presumed to belong to the second object M2. As described above, the point data P5 presumed to belong to the second object M2 is identified and specified with respect to the point data P4 presumed to belong to the first object M1.

The object identification unit 13 generates identification information representing the identification result of the object M. The object identification unit 13 generates information indicating whether or not the point to be identified belongs to the first object M1 as the identification information. For example, the object identification unit 13 generates the second point cloud data PD2 representing the second object M2 as the identification information. The second point cloud data PD2 is point cloud data in which points to be identified that are determined to belong to the first object M1 are excluded from the first point cloud data PD1. As the identification information, the object identification unit 13 may generate information in which a flag (eg, attribute information) indicating whether or not the object belongs to the first object M1 is added to the point data. The object identification unit 13 stores the generated identification information in the storage unit 14.

The object identification unit 13 may determine whether or not the point to be identified belongs to the second object M2 (whether or not it is a point on the second object M2). For example, the object identification unit 13 may determine that the point to be identified does not belong to the second object M2 when the distance between the point to be identified and the estimated plane is less than the threshold value. Further, the object identification unit 13 may determine that the point to be identified belongs to the second object M2 when the distance between the point to be identified and the estimated plane is equal to or greater than the threshold value.

Further, the object identification unit 13 may generate information indicating whether or not the point to be identified belongs to the second object M2 as the identification information. The object identification unit 13 may generate the second point cloud data PD2 by extracting the points to be identified that are determined to belong to the second object M2 from the first point cloud data PD1 as the identification information. As the identification information, the object identification unit 13 may generate information in which a flag (eg, attribute information) indicating whether or not the object belongs to the second object M2 is added to the point data.

The surface information generation unit 16 in FIG. 1 calculates surface information as shape information. In the following description, the process of calculating surface information as appropriate is referred to as surface process. The surface information includes, for example, at least one of polygon data, vector data, and draw data. The surface information includes the coordinates of a plurality of points on the surface of the object and the connection information between the plurality of points. The connection information (eg, attribute information) includes, for example, information that associates the points at both ends of the line corresponding to the ridge line (eg, edge) on the surface of the object with each other. Further, the connection information includes, for example, information for associating a plurality of lines corresponding to the contour of the surface of the object with each other.

In the surface processing, the surface information generation unit 16 estimates a surface between a point selected from a plurality of points included in the point cloud data and a point in the vicinity thereof. Further, the surface information generation unit 16 converts the point cloud data into polygon data having plane information between points in the surface processing. The surface information generation unit 16 converts the point cloud data into polygon data by, for example, an algorithm using the least squares method. This algorithm may be, for example, an application of an algorithm published in a point cloud processing library.

The surface information generation unit 16 executes surface processing based on the identification result of the object identification unit 13. The surface information generation unit 16 reads the identification information generated by the object identification unit 13 from the storage unit 14, and executes surface processing based on the identification information. For example, the surface information generation unit 16 estimates a surface on the second object M2 by using the second point cloud data PD2 representing the second object M2 identified by the object identification unit 13. The surface information generation unit 16 generates information representing a surface on the estimated second object M2 as at least a part of the surface information. The surface information generation unit 16 stores the calculated surface information in the storage unit 14.

The model generation unit 11 described above may generate the texture information of the surface defined by the three-dimensional point coordinates and the related information as the model information. The texture information includes, for example, at least one piece of information such as characters and figures on the surface of an object, a pattern, a texture, a pattern, information defining irregularities, a specific image, and a color (eg, chromatic color, achromatic color). The model generation unit 11 may store the generated texture information in the storage unit 14.

In addition, the model generation unit 11 may generate spatial information (eg, lighting conditions, light source information) of the image as model information. The light source information includes, for example, the position of the light source that irradiates the illumination light with reference to the object, the direction in which the light is irradiated from this light source to the object (irradiation direction), the wavelength of the light emitted from this light source, and the wavelength thereof. Contains information on at least one item of the type of light source. The model generation unit 11 may calculate the light source information by using, for example, a model assuming Lambertian reflection, a model including Albedo estimation, and the like. The model generation unit 11 stores at least a part of the generated (eg, calculated) model information in the storage unit 14. The model generation unit 11 does not have to generate a part of the model information.

Next, the detection method according to the embodiment will be described based on the configuration of the detection device 1. FIG. 5 is a flowchart showing a detection method according to the first embodiment. FIG. 6 is a flowchart showing a plane estimation process according to the first embodiment. FIG. 7 is a flowchart showing the object identification process according to the first embodiment. Regarding the configuration of the detection device 1, FIG. 1 is referred to as appropriate.

In step S1 of FIG. 5, the position detection unit 2 detects the position information (eg, depth) on the object M in the target area AR. In step S2, the point cloud data generation unit 15 of the processing device 3 generates the point cloud data of the object M. For example, the point cloud data generation unit 15 executes the point cloud process using the depth obtained in step S1 to generate the first point cloud data. The point cloud data generation unit 15 stores the first point cloud data in the storage unit 14.

In step S3, the surface estimation unit 12 estimates the plane of the first object M1. For example, the surface estimation unit 12 executes the plane estimation process using the first point cloud data generated by the point cloud data generation unit 15. In step S4, the object identification unit 13 identifies the second object with respect to the estimated plane based on the distance between each point on the object and the estimated plane.

Here, an example of the plane estimation process in step S3 will be described with reference to FIG. In step S11 of FIG. 6, the surface estimation unit 12 sets a point set. The surface estimation unit 12 selects all or a part of a plurality of points included in the first point cloud data, and sets a point set consisting of the selected points.

In step S12, the surface estimation unit 12 derives a mathematical formula representing the approximate plane of the point set set in step S11. The approximate plane is a candidate for the estimated plane FP. The surface estimation unit 12 uses the least squares method to determine the coefficients (a, b, c, d) of the mathematical formula (ax + by + cz + d = 0) representing the plane. The surface estimation unit 12 calculates the square of the distance between the plane and each point included in the point set, and a coefficient is set so that the sum of the distance squared point sets (hereinafter referred to as the sum of squares of the distances) is minimized. (A, b, c, d) is calculated.

In step S13, the surface estimation unit 12 calculates the residual of the approximate plane with respect to the point set. This residual is, for example, the RMS of the distance between the approximate plane defined by the coefficients (a, b, c, d) calculated in step S12 and each point of the point set. The residual may be the sum of squares of the distances between the approximate plane and each point in the point set.

In step S14, the surface estimation unit 12 determines whether or not the residual calculated in step S13 is equal to or less than the threshold value. When the surface estimation unit 12 determines that the residual is not equal to or less than the threshold value (step S14; No), the surface estimation unit 12 returns to step S11 and sets the point set. For example, the surface estimation unit 12 resets (updates) the point set by excluding the points (or large points) that are relatively long (or large) from the approximate plane derived in step S12 from the previously selected point set. do. The surface estimation unit 12 repeats the processes from step S12 to step S14 using the reset point set.

The surface estimation unit 12 may randomly select some points in step S11. Further, the plane estimation unit 12 may estimate a plane (eg, a candidate for the estimated plane FP) based on the points selected in step S11 in step S12. In step S13, the surface estimation unit 12 may calculate an evaluation value (eg, residual) using the distance from the plane estimated in step S12 to its adjacent point as an evaluation function. The surface estimation unit 12 repeats the random selection of points as described above, the estimation of the plane using the selected points, and the evaluation of the estimated plane, and even if the candidate with the best evaluation value is adopted as the estimation plane FP. good.

In step S14, when the surface estimation unit 12 determines that the residual is equal to or less than the threshold value (step S14; Yes), the surface estimation unit 12 adopts the approximate plane as the estimation plane FP in step S15. In step S16, the surface estimation unit 12 stores the estimated plane information representing the estimated plane FP in the storage unit 14. The surface estimation unit 12 stores the coefficients (a, b, c, d) of the mathematical formula (ax + by + cz + d = 0) representing the estimated plane FP in the storage unit 14 as the estimated plane information.

Next, an example of the object identification process in step S4 of FIG. 5 will be described with reference to FIG. 7. In step S21, the object identification unit 13 selects a point to be identified. The object identification unit 13 selects a point to be identified from the first point cloud data. In step S22, the object identification unit 13 calculates the distance between the point selected in step S21 and the estimated plane FP. The object identification unit 13 reads information (eg, coefficient) of a mathematical formula representing the estimated plane FP from the storage unit 14, and calculates the distance from the point data (three-dimensional coordinates) of the point to be identified and the mathematical formula representing the estimated plane. do.

In step S23, the object identification unit 13 determines whether or not the distance calculated in step S22 is less than the threshold value. When the object identification unit 13 determines that the calculated distance is less than the threshold value (step S23; Yes), the object identification unit 13 identifies that the point to be identified is not a point on the second object M2 in step S24. In step S24, the object identification unit 13 excludes the point to be identified from the point cloud data of the second object M2 by masking processing or the like. The object identification unit 13 may classify (eg, register) the points to be identified into point cloud data of an object (eg, first object M1) different from the second object M2. In addition, the object identification unit 13 may generate identification information (eg, a flag) indicating that the point to be identified is not a point on the second object M2 in step S24. For example, when the point to be identified is identified as a non-target point that is not a point on the second object M2, the object identification unit 13 bases the identification information on the non-target point among the identification target points (in this case, the non-target point). , A point that is not the second object M2) is deleted from the point group data of the second object M2.

In addition, the object identification unit 13 may identify the point to be identified as a point on the first object M1 in step S24. In this case, the object identification unit 13 may classify (eg, register) the points to be identified into the point cloud data of the first object M1. Further, the object identification unit 13 may generate identification information (eg, a flag) indicating that the point to be identified is a point on the first object M1.

In step S23, when the object identification unit 13 determines that the calculated distance is not less than the threshold value (greater than or equal to the threshold value) (step S23; No), in step S25, the point to be identified is on the first object M1. Identify as not a point. In step S25, the object identification unit 13 excludes the point to be identified from the point cloud data of the first object M1. The object identification unit 13 may classify (eg, register) the points to be identified into point cloud data of an object different from the first object M1. In step S25, the object identification unit 13 may generate identification information (eg, a flag) indicating that the point to be identified is not a point on the first object M1.

In addition, the object identification unit 13 may identify the point to be identified as a point on the second object M2 in step S25. In this case, the object identification unit 13 may classify (eg, register) the points to be identified into the point cloud data of the second object M2. Further, the object identification unit 13 may generate identification information (eg, a flag) indicating that the point to be identified is a point on the second object M2.

After the processing of step S24 or after the processing of step S25, in step S26, the object identification unit 13 determines whether or not there is a point to be identified next. When the object identification unit 13 has a plurality of points for which the object identification process is scheduled for which the object identification process has not been executed, the object identification unit 13 determines that there is a point to be identified next. When the object identification unit 13 determines that there is a point to be identified next (step S26; Yes), the object identification unit 13 returns to step S21 and selects the next point to be identified. When the object identification unit 13 determines that there is no next point to be identified (step S26; No), the object identification unit 13 ends the object identification process.

In step S5 of FIG. 5, the model generation unit 11 generates model information based on the position information of the second object M2 identified in step S4. For example, the model generation unit 11 selectively uses the point cloud data of the second object M2 identified by the object identification unit 13 among the first point cloud data to generate model information of the second object M2. The model generation unit 11 generates shape information (eg, surface information) of the second object M2 as model information. The model generation unit 11 stores the generated model information in the storage unit 14.

The model generation unit 11 may generate shape information representing the first object M1 by using the point cloud data obtained by excluding the second point cloud data from the first point cloud data. Further, the model generation unit 11 may generate the first shape information representing the first object M1 and the second shape information representing the second object M2. The model generation unit 11 may combine the first shape information and the second shape information to generate shape information representing the first object M1 and the second object M2.

As described above, the detection device 1 according to the embodiment estimates the plane in the first object M1 by the surface estimation unit 12, and identifies the second object M2 with respect to the estimated plane FP by the object identification unit 13. Therefore, the detection device 1 according to the embodiment can identify the second object M2 in contact with the first object M1, and contributes to segmenting the second object M2 with high accuracy, for example.

The second object M2 does not have to be a human body. The second object M2 may be a non-human organism (eg, animal, plant, dog, cat) or an article (eg, device, robot, automobile). The second object M2 may be a stationary object that is stationary in the target area AR, or may be a moving body that moves (eg, self-propelled) in the target area AR.

Further, the first object M1 may be an object that is stationary in the target area AR or an object that is moving in the target area AR. For example, the detection device 1 may identify a second object M2 (eg, a human body) that comes into contact with a moving first object M1 (eg, an automobile). In the first object M1, the surface that is supposed to come into contact with the second object M2 may not be a flat surface but may be a curved surface, or may be a composite surface having a flat surface and a curved surface. For example, the surface of the first object M1 as an object surface may be a surface that is regularly arranged, such as the upper surface of a staircase. Further, the surface of the first object M1 may include irregularities, and the surface estimation unit 12 may approximate the surface of the first object M1 with a plane.

In FIG. 1 and the like, the second object M2 comes into contact with a surface arranged downward in the vertical direction with respect to the second object M2. The second object M2 may come into contact with a surface (eg, ceiling) arranged vertically above the second object M2. Further, the second object M2 may come into contact with a surface (eg, a wall surface) arranged horizontally with respect to the second object M2. The storage unit 14 stores in advance the conditions of the surface on which the first object M1 is expected to come into contact (eg, the calculation condition of the estimated plane FP), and the surface estimation unit 12 stores the surface of the surface stored in the storage unit 14. The estimated plane FP may be derived based on the conditions. The portion of the second object M2 that comes into contact with the first object M1 is arbitrary and may be a portion (eg, a hand) other than the foot of the human body.

The surface estimation unit 12 may derive an estimation plane without using the point cloud data. For example, the surface estimation unit 12 may determine that this region belongs to the estimation plane FP when the depth distribution in the region satisfies a predetermined condition in the depth map generated by the position detection unit 2. For example, the surface estimation unit 12 may determine that this region belongs to the estimation plane FP when the depth changes regularly in a region including a plurality of pixels in the depth map.

Further, the surface estimation unit 12 may derive an estimated plane FP based on the surface information. For example, the surface estimation unit 12 may extract surface information approximated by a plane from the surface information generated by the surface information generation unit 16, and use the extracted surface information as a candidate for the estimation plane FP. In this case, the surface estimation unit 12 does not have to derive a candidate for the estimation plane FP. Further, the surface estimation unit 12 may derive the estimated plane FP based on the normal direction of the plane estimated by the surface information generation unit 16 by the surface processing. The surface estimation unit 12 may derive a region having the same or similar normal directions in the surface information as an estimation plane. The surface estimation unit 12 may derive this region as an estimation plane FP when the variance in the normal direction in a region of a predetermined size is less than a predetermined value.

At least a part of the detection device 1 may be a stationary device or a portable device. At least a part of the position detection unit 2 may be provided on the same body as the processing device 3. Further, at least a part of the processing device 3 may be provided in the position detecting unit 2. For example, the position detection unit 2 may include a point cloud data generation unit 15 and output the point cloud data to the processing device 3 as the position information of the object obtained by the detection.

[Second Embodiment]
The second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 8 is a diagram showing a detection device according to the present embodiment. In the present embodiment, the surface estimation unit 12 sets the surface SF of the object M as an object surface (in this case, based on the detection result of the position detection unit 2 in a state where the second object M2 is not arranged in the target area AR, for example. Estimated as a plane). The object identification unit 13 has a plane estimated by the surface estimation unit 12 based on the detection result of the position detection unit 2 in a state where the second object M2 is not arranged in the target area AR, and the second object M2 is in the target area AR. The second object M2 is identified based on the detection result of the position detection unit 2 in the arranged state.

Reference numeral Q1 in FIG. 8 represents a first state in which the second object M2 is arranged at the first position (eg, a position outside the target area AR). The first state Q1 is, for example, a state in which the second object M2 is not arranged in the target area AR. Further, the reference numeral Q2 in FIG. 8 represents a second state in which the second object M2 is arranged at the second position. The second position is a position inside the target area AR and is a position different from the first position.

In the first state Q1, the position detection unit 2 detects the position information of the first object M1. Then, the surface estimation unit 12 estimates (approximates) the surface SF of the first object M1 as a plane based on the position information of the first object M1 detected in the first state Q1.

Further, in the second state Q2, the position detection unit 2 detects the position information of the object M including the first object M1 and the second object M2. Then, the object identification unit 13 includes a plane estimated by the surface estimation unit 12 based on the position information of the first object M1 detected in the first state Q1 and the position information of the object M detected in the second state Q2. Is used to identify the second object M2.

The order of the detection in the first state Q1 and the detection in the second state Q2 is arbitrarily set. The position detection unit 2 may execute the detection in the second state Q2 after executing the detection in the first state Q1. The position detection unit 2 may execute the detection in the first state Q1 after executing the detection in the second state Q2. In this case, the surface estimation unit 12 estimates the plane after the detection in the first state Q1 is completed, and the object identification unit 13 uses the detection result in the second state after the surface estimation unit 12 estimates the plane. The object M2 may be identified.

Although the second object M2 is arranged outside the target area AR in the first state Q1 of FIG. 8, it may be arranged at a position different from the second state Q2 in the target area AR. For example, the detection device 1 detects the position information of the object M in the third state in which the second object M2 is located at a position (eg, the end of the target area AR) farther from the center of the target area AR than in the second state. Then, the surface estimation unit 12 may estimate the plane based on the detection result. Further, the surface estimation unit 12 may estimate the plane based on the change in the detection result of the position detection unit 2 due to the movement of the second object M2. For example, the position detection unit 2 detects the position information of the object M under a plurality of conditions in which the position of the second object M2 is different, and the surface estimation unit 12 changes the position information of the object M detected under a plurality of conditions ( The plane may be estimated by excluding at least a part of the position information of the second object M2 from the position information of the object M based on the difference).

Next, the detection method according to the embodiment will be described based on the configuration of the detection device 1. FIG. 9 is a flowchart showing a detection method according to the second embodiment. For the configuration of the detection device 1, reference to FIGS. 1 and 8 as appropriate. In step S31, the position detection unit 2 has position information (eg, depth) in the target area AR in the first state Q1 (see FIG. 8) in which the second object M2 is arranged in the first position (eg, outside the target area AR). ) Is detected.

In step S32, the processing device 3 removes noise from the detection result of the position detection unit 2 obtained in step S31. The noise removing process for removing noise includes, for example, a process for removing spatial noise from the depth map generated by the position detection unit 2. For example, the processing device 3 determines the depth in the first region when the difference in depth between the adjacent first region (eg, first pixel) and the second region (eg, second pixel) exceeds the threshold value in the depth map. Alternatively, it is determined that the depth in the second region is noise. When the processing device 3 determines that the depth in the first region is noise, the processing device 3 performs interpolation using the depth of the region (eg, third pixel, fourth pixel) around the first region to obtain the first region. Spatial noise is removed by estimating the depth and updating (replacing, correcting) the depth of the first region with the estimated depth.

Further, the noise removing process for removing noise includes, for example, a process for removing temporal noise (eg, noise that changes with time) from the depth map generated by the position detection unit 2. For example, the position detection unit 2 repeats detection at a predetermined sampling frequency, and the processing device 3 compares the detection results (eg, depth map) of the position detection units 2 having different detection timings. For example, the processing device 3 calculates the amount of change in depth (time change amount) in each pixel from the depth map in the first frame and the depth map in the second frame following the first frame. When the amount of change in depth in each pixel satisfies a predetermined condition, the processing device 3 determines that the depth in the first frame or the depth in the second frame of this pixel is noise. When the processing device 3 determines that the depth in the second frame is noise, the processing device 3 determines the depth of the second frame by interpolation using the depth in the first frame and the depth in the third frame following the second frame. presume. The processing device 3 removes temporal noise by updating (replacing, correcting) the depth of the pixel corresponding to the noise with the estimated depth.

The noise removal process may not include a process for removing spatial noise or a process for removing temporal noise. Further, the noise removal process may be executed in a portion other than the processing device 3 (eg, the position detection unit 2). Further, the detection device 1 does not have to execute the noise removal process.

In step S33 of FIG. 9, the point cloud data generation unit 15 generates the third point cloud data. For example, the point cloud data generation unit 15 executes point cloud processing on the depth map from which noise has been removed in step S32 to generate third point cloud data. The point cloud data generation unit 15 stores the generated third point cloud data in the storage unit 14. In step S34, the surface estimation unit 12 estimates the plane. For example, the surface estimation unit 12 reads the third point cloud data from the storage unit 14 and derives the estimated plane FP based on the third point cloud data. The surface estimation unit 12 stores information representing the derived estimated plane FP in the storage unit 14 (eg, registration of the initial state).

In step S35, the position detection unit 2 is the second state Q2 (see FIG. 8) in which the second object M2 is arranged at the second position in the target area AR, and the position information of the object M in the target area AR (eg, Depth) is detected. In step S37, the processing device 3 removes noise from the detection result of the position detection unit 2 obtained in step S36. The process of step S37 is the same as the process of step S32. In step S38, the point cloud data generation unit 15 generates the fourth point cloud data. For example, the point cloud data generation unit 15 executes point cloud processing on the depth map from which noise has been removed in step S37 to generate fourth point cloud data. The point cloud data generation unit 15 stores the generated fourth point cloud data in the storage unit 14.

The processing of steps S39 to S44 corresponds to the object identification processing. In step S39, the object identification unit 13 sets (eg, selects) a point to be identified from the fourth point cloud data. In step S40, the object identification unit 13 calculates the distance between the point to be identified and the estimated plane FP estimated in step S35. The object identification unit 13 reads information (eg, coefficient) of a mathematical formula representing the estimated plane FP from the storage unit 14, and calculates the distance from the point data (three-dimensional coordinates) of the point to be identified and the mathematical formula representing the estimated plane. do.

In step S41, the object identification unit 13 determines whether or not the distance calculated in step S30 is less than the threshold value. When the object identification unit 13 determines that the calculated distance is equal to or greater than the threshold value (step S41; Yes), the object identification unit 13 registers the point to be identified in the fifth point cloud data in step S42. When the object identification unit 13 determines that the calculated distance is not equal to or greater than the threshold value (less than the threshold value) (step S41; No), or after the processing of step S42, there is a next identification target point in step S42. Judge whether or not. When the object identification unit 13 has a plurality of points for which the object identification process is scheduled for which the object identification process has not been executed, the object identification unit 13 determines that there is a point to be identified next. When the object identification unit 13 determines that there is a point to be identified next (step S43; Yes), the object identification unit 13 returns to step S39 and selects the next point to be identified.

When the object identification unit 13 determines that there is no next point to be identified (step S43; No), the object identification unit 13 stores the fifth point cloud data in the storage unit 14 and ends the object identification process. The fifth point cloud data is point cloud data in which points in the vicinity of the estimated plane FP are excluded from the fourth point cloud data. The fifth point cloud data can be used as point cloud data representing the second object M2 identified with respect to the estimated plane FP.

The surface estimation unit 12 may estimate the plane based on the change in the detection result of the position detection unit 2 due to the movement of the second object M2. For example, when the second object M2 moves from the first position to the second position, the point cloud data based on the change in the detection result of the position detection unit 2 is changed from the third point cloud data to the fourth point cloud data. Change. It is presumed that the portion of the shape indicated by the third point cloud data and the shape indicated by the fourth point cloud data whose amount of change is less than the threshold value belongs to, for example, a stationary object in the target region AR. The surface estimation unit 12 extracts a portion where the amount of change is less than the threshold value between the shape indicated by the third point cloud data and the shape indicated by the fourth point cloud data, and derives the plane in the extracted portion as an estimation plane. May be good.

In FIG. 8, the first position where the second object M2 is arranged is outside the target area AR, but the first position may be inside the target area AR. Further, the detection process of the position detection unit 2 corresponding to the first position (example, step S31 in FIG. 9) ends the detection process of the position detection unit 2 corresponding to the second position (example, step S36 in FIG. 9). It may be executed later. The processes of steps S32 to S34 may be executed at any timing from the end of the process of step S31 to the start of the object identification process (eg, steps S39 to S44). Further, the processes of steps S37 to S38 may be executed at any timing from the end of the process of step S36 to the start of the object identification process (eg, steps S39 to S44).

[Third Embodiment]
The third embodiment will be described. In the present embodiment, FIG. 1 is appropriately referred to for the configuration of the detection device 1. The object identification unit 13 (see FIG. 1) according to the present embodiment emphasizes the distance between the point on the object M detected by the position detection unit 2 and the object surface (in this case, a plane) estimated by the surface estimation unit 12. The evaluation value is calculated, and the position information of the second object M2 is identified based on the evaluation value.

FIG. 10 is a diagram showing an object identification process by the object identification unit 13 according to the present embodiment. In the XYZ Cartesian coordinate system of FIG. 10, the Y direction is the direction perpendicular to the estimated plane FP. Further, the X direction and the Z direction are parallel to the estimated plane FP and perpendicular to each other. For example, the Y direction corresponds to the vertical direction of the real space, and the X direction and the Z direction correspond to the horizontal direction of the real space, respectively.

The object identification unit 13 divides the area in the Z direction and identifies the position information of the second object M2 for each divided area. _{Reference numeral PD m} in FIG. 10 is point cloud data of the object M in one of the divided regions (region of Z _m ≤ Z <Z _{m + 1} , where m is a positive integer). The object identification unit 13 _{executes the object identification process by using the point cloud data PD m} as point data in the two-dimensional region (for example, ignoring the value of the Z coordinate). The object identification unit 13 sequentially performs object identification processing on the divided regions for m = 1, 2, ..., From the point group data of the object M relating to a predetermined region (eg, target region AR). 2 Identify the point group data of the object M2.

Sign _{P i} in FIG. 10 represents one point included in the point cloud data PD _m. The subscript i is a number assigned to each point (eg, i = 1, 2, ...). Further, _{X i} is the X-coordinate of the point _{P i, Y i} is the Y coordinate of the point _{P i.} Y ₀ is the Y coordinate of the estimated plane FP, _{h i} is the distance between the point _{P i} and the estimated plan _{FP (h i = Y i -Y} 0, the height from the estimated plane FP). Reference numeral Q3 is a region representing a part of the object M2 (eg, the heel of the human body).

Also, _{d i} is the evaluation value for the point _{P i.} For example, in the region Q3 (see the enlarged view shown by the circle in FIG. 10), d _j is an evaluation value with respect to the point P _j. Object identification unit 13 as the evaluation value d _i, a value is calculated by increasing the distance h _i based on the distance h _i between the point P _i and the estimated plane FP. evaluation function for calculating the d _i is represented by the formula (1) in FIG. 10, for example. In the formula (1), V1 is a threshold value (second threshold value). The threshold value V1 is a value that is arbitrarily set. The threshold value V1 is set to, for example, a value (example, 0) equal to or less than the first threshold value (example, distance D in FIG. 4).

The object identification unit 13 evaluates the first point based on the first distance between the first point and the estimated plane FP in the object M and the second distance between the second point around the first point and the estimated plane FP. Calculate the value. The first point is, for example, a point _Pi , and the second point is, for example, a point _Pi-1 and a point _{Pi + 1} . The first distance is a distance _{h i,} for example, the point _{P i} and the estimated plane FP, the second distance is the distance _{h i-1,} and the point _{P i + 1} and the estimated plane between the point _{P i-1} and the estimated plane FP The distance to the FP is hi _{+ 1} . In the formula (1), _{f i} is the increment for the distance _{(h i).}

As shown in the equation (1) of FIG. 10, the object identification unit 13 has a case where the second distance (hi _-1 , hi _{+ 1} ) is equal to or less than the threshold value V1 (second threshold value) (hi _-1 ≤ V1 and). the h _{i + 1} ≦ V1), the evaluation value _{d i} for point _{P i} and the first distance _{(h i).} Object identification unit 13, the second distance _{(h i-1, h i} + 1) is larger than the threshold value V1 (second threshold _{value) (h i-1> V1} , or h i + 1> V1), for point _{P i} as the evaluation value _{d i,} the first distance calculating a value obtained by increasing _{(h i} + _f i) on the basis of the second distance.

Object identification unit 13, when the second distance _{(h i-1, h i} + 1) is larger than the second threshold value (V1), as the evaluation value, a value which decreases with respect to the second distance _{(f i)} first calculating a value obtained by adding the first distance _{(h i).} Increment f _i is a function of the second distance as a parameter. Increment f _i, to the reduction of the second distance is a function of increasing values (function with slope negative function, a negative correlation). The amount of increase _fi is represented by, for example, the following equation (2).
f _i = (1 / h _i-1 ) + (1 / h _{i + 1} ) ... Equation (2)

Code Q4 in FIG. 10 is a region showing the evaluation value d _i of each point corresponding to the region Q3. In the region Q4, the evaluation value d _i of each point is represented by a black circle, the distance h _i of each point is represented by a white circle. The points P _j -2 and the points P _j -1 are points sufficiently separated from the estimated plane FP. For example, _{the distance h j-1 between the point P j-1} and the estimated plane FP (see the white circle corresponding to the point P _j-1 in the region Q4) is used to determine whether or not it belongs to the second object M2. Greater than one threshold (eg, D in region Q4). Further, the point P _j , the point P _{j + 1} , and the point P _{j + 2} are points in the vicinity of the estimated plane FP. For example, _{the distance h j between the point P j} and the estimated plane FP (see the white circle corresponding to the point P _j in the region Q4) is the first threshold value (eg, region) used to determine whether or not it belongs to the second object M2. It is smaller than D) of Q4.

Here, the point P _j-1 is set as the first point, and the second points around the _{point P j-1 are set as the point P j-2} and the point P _j . The second point (point P _j-2 and point P _j ) is, for example, the point where the distance from the first point (point P _{j-1 ) is the shortest in the point cloud data PD m} , and the point where the distance is the next shortest. Is. Since the second distance (h _j-2 ) with respect to the second point (point P _j-2 ) is larger than V1 (h _j-2 > V1) in the _{object identification unit 13, the evaluation value d j-1} is h _{j. -1} + f _j-1 is calculated. Evaluation value d _i-1, since in that point P _i-1 is sufficiently distant from the estimated plane FP, but the increase f _i is smaller distance h _i is large, is whether a second object M2 It is larger than the first threshold value (eg, D in region Q4) used for the determination. The object identification unit 13 _{determines whether or not the evaluation value di-1} is equal to or higher than the first threshold value (D). The object identification unit 13 determines that the evaluation value di _-1 is equal to or higher than the first threshold value (D) _{, and identifies that the point Pi-1} is a point on the second object M2 (on the first object M1). Identify not the point of).

Next, _{let the point P j} be the first point, and the second points around it be the point P _j-1 and the point P _{j + 1} . Since the second distance (h _j-1 ) with respect to the second point (point P _j-1 ) of the object identification unit 13 is larger than V1 (h _j-1 > V1), the evaluation value d _j is h _j + f _j. Is calculated. Evaluation value d _i, since the point P _i is a point in the vicinity of the estimated plane FP, the distance h _i is small large increase f _i, first used in the determination of whether the second object M2 It becomes larger than the threshold value (eg, D of region Q4). Object identification unit 13, the evaluation value d _i is equal to or first threshold value (D) above. Object identification unit 13, the evaluation value d _i is determined to be the first threshold value (D) above, the identification point P _i is not a point on the (first object M1 identifies that a point on the second object M2 do).

Next, the point P _{j + 1} is set as the first point, and the second points around it are set as the point P _j and the point P _{j + 2} . Object identification unit 13, the second point (point _{P j,} the point _{P j + 2)} the second distance (distance _{h j,} the distance _{h j + 2)} of less than or equal to V1 _{(h j ≦ V1 and h j} + 2 ≦ V1) , so the evaluation value d _{Let j + 1} be h _{j + 1} . The object identification unit 13 _{determines whether or not the evaluation value di + 1} is equal to or higher than the first threshold value (D). The object identification unit 13 determines that the evaluation value di _{+ 1} is less than the first threshold value (D) _{, and identifies that the point Pi + 1} is not a point on the second object M2 (identification as a point on the first object M1). do).

Object identification unit 13 according to the present embodiment, the first point (e.g., _{P j)} second point _{(eg, P i-1, P i} + 1) around the plane of the surface estimation unit 12 has estimated (estimated plane FP second distance (example of _a), the evaluation value based on the _{h i-1, h i +} 1) ( example calculates the _{d j),} compares the evaluation value (e.g., _{d j)} and the first threshold value (D) Then, it is determined whether or not the first point belongs to the second object. Therefore, the detection device 1 according to the present embodiment can _{distinguish a point near the estimated plane FP (eg, Pj} ) from a point on the estimated plane FP and identify it as a point belonging to the second object M2. In this way, the object identification unit 13 can identify and separate the points in the vicinity of the estimated plane FP and the points on the estimated plane FP with high accuracy, and the detection device 1 can, for example, determine the shape of the second object M2. It can be detected with high accuracy.

Incidentally, the object identification unit 13, for the points in the range distance (h _i) is given from the estimated plane FP, the distance is calculated emphasized evaluation value is compared with the calculated evaluation value and the first threshold value The process of identifying the position information of the second object M2 may be executed. For example, object identification unit 13, the distance from the estimated plane FP (h _i) a first threshold value (e.g., D) may not derive the evaluation value for a point greater than. Further, the object identifying unit 13, a second threshold value (e.g., V1) with a smaller fourth threshold than, derive the evaluation value for a point distance from the estimated plane FP (h _i) is smaller than the fourth threshold value You don't have to. Object identification unit 13, for example, the distance from the estimated plane FP (h _i) is a point less than the fourth threshold value, identified as not a point on the second object M2 (a point on the first object M1 is) You may.

The evaluation function (eg, equation (1)) for calculating the evaluation value is not limited to the form described with reference to FIG. 10, and can be changed as appropriate. For example, the evaluation value d _i is a value obtained by adding the increment f _i to the distance h _i in the formula (1), the coefficient may be a value obtained by multiplying the a distance h _i. Also, increment _{f i may} be given by a function other than _{(1 / h i-1)} + (1 / h i + 1).

Incidentally, the object identification unit 13 calculates as the primary evaluation value evaluation value d _i based on the distance h _i as in equation (1), the primary evaluation value instead of the distance h _i in equation (1) The secondary evaluation value may be calculated by using it (for example, the object identification unit 13 may recalculate the evaluation value). The object identification unit 13 derives a k-th evaluation value (k is an integer of 2 or more) by such recalculation of the evaluation value, and sets the k-th evaluation value and the threshold value (eg, the first threshold value, D in FIG. 4). The position information of the second object M2 may be identified by the comparison of. For example, larger than _P 1 Assessment value of _{j (h j} + _{f j)} is _{h j} the point indicated in the region Q4 of FIG. 10, secondary evaluation of the point _{P j + 1} with the primary evaluation values of the point _{P j} After calculating the value, in that the second evaluation value of the point P _{j + 1} is the first threshold value (e.g., D region Q4) above, the point P _{j + 1} in some cases can be identified as points on the second object M2. In this case, a point near the estimated plane FP (eg, point _{Pj + 1} ) is recognized as a point on the second object M2, and for example, the shape of the second object M2 can be detected with high accuracy.

[Fourth Embodiment]
A fourth embodiment will be described. In the present embodiment, FIG. 1 is appropriately referred to for the configuration of the detection device 1. In the present embodiment, the threshold value (eg, the distance D in FIG. 4) used for determining whether or not the point is on the second object M2 is a variable value. The object identification unit 13 (see FIG. 1) sets a threshold value based on the first point on the object M detected by the position detection unit 2 and the second point around the object M, and uses the set threshold value to make the first threshold value. It identifies (determines) whether or not the point is a point on the second object M2.

FIG. 11 is a diagram showing an object identification process by the object identification unit 13 according to the present embodiment. In FIG. 11, the same elements as those in FIG. 10 are designated by the same reference numerals to simplify or omit the description thereof. Similar to FIG. 10, the object identification unit 13 divides the area in the Z direction and identifies the position information of the second object M2 for each divided area. The object identification unit 13 may execute the object identification process without dividing the area.

In the region Q5, the code _{V i} is the threshold set by the object identification unit 13 (third threshold value). The object identification unit 13 includes a second point (eg, point P _j-1 , point P _{j + 1} ) and a surface estimation unit 12 _{around the first point (eg, point P j} ) in the object M detected by the position detection unit 2. There (in this case, the plane) estimated object plane setting a threshold value V _i based on the distance between the (estimated plane FP). Threshold _{V i} is the first distance (eg, _{h j)} and the second distance is a function with _(e.g., one or both of the _{h j-1} and _{h j + 1)} variables. Threshold V _i is, for example, the first distance and the weighted value and a second distance (weighted average), represented by the following formula (3). _{C 1} , C ₂ , and C ₃ in the formula (3) are coefficients (weighting coefficients). C ₁ , C ₂ , and C ₃ may have different values, or at least two may have the same value.
V _i = C ₁ h _i-1 + C ₂ h _i + C ₃ h _{i + 1} ... Equation (3)

Object identification unit 13, a first point (e.g., point _{P i)} the distance between the surface estimation unit 12 has estimated plane (estimated plane FP) (eg, the distance _{h i)} and the third threshold value (e.g., the threshold _{V i} ) To determine whether or not the first point belongs to the second object M2. Object identification unit 13, if the distance _{h i} is equal to or greater than the threshold value _{V i,} the point _{P i} is a point on the second object M2 (not a point on the first object M1) and identify. For example, a point near the estimated plane FP (eg, point P _j ) has a smaller threshold (eg, threshold V _j ) with respect to the distance from the estimated plane FP (eg, distance h _j). Therefore, the object identification unit 13 identifies a point (eg, point _Pj ) in the vicinity of the estimated plane FP as a point on the second object M2. The first point (e.g., point _{P i)} when is a point on the estimated plane FP, but the threshold _{V i} is smaller becomes h (i) is 0. Thus, the object identifying unit 13, when the distance h _i is less than the threshold value V _i, the point P _i is not a point on the second object M2 (a point on the first object M1) and identify. In this way, the object identification unit 13 can discriminate between the points in the vicinity of the estimated plane FP and the points on the estimated plane FP with high accuracy, and the detection device 1 can, for example, accurately determine the shape of the second object M2. It is detectable.

As described in FIGS. 10 and 11, the object identification unit 13, a first point (e.g., point _{P i)} the distance between the estimated plane FP (eg, _{h i)} values (examples based on the distance _h i, evaluation value d _i), the relative value of the threshold (e.g., by adjusting the contrast), the first point (e.g., also identify whether the point P _i) is a point on the second object M2 good. The processing device 3 may have a first mode that emphasizes the relative value (eg, contrast) and a second mode that does not emphasize the relative value (eg, contrast). The processing device 3 may be configured to be switchable between the first mode and the second mode, for example, according to the user's designation. Further, the processing device 3 does not have to have the first mode or the second mode.

Further, as described with reference to FIGS. 10 and 11, when the object identification unit 13 _{executes the object identification process for each divided region (eg, region of Z m} ≤ Z <Z _{m + 1} ), for example, the first point. The processing load for searching for the second point around the object is reduced. Further, when the object identification unit 13 _{executes the object identification process using the point data included in the point cloud data PD m} of the divided area as two-dimensional data (two-dimensional coordinates), for example, the load of the process of calculating the distance. Is reduced.

The object identification unit 13 may collectively execute the object identification process (for example, without dividing the area) for the entire area. Further, the object identification unit 13 may execute the object identification process using the point data included in the point cloud data PD as three-dimensional data (three-dimensional coordinates). In this case, the object identification unit 13 can identify, for example, a second point (eg, a recent point) around the first point with high accuracy. Further, the object identification unit 13 can, for example, calculate the distance between each point and the estimated plane FP with high accuracy to identify the points belonging to the second object M2. The number of the second points is two in FIGS. 10 and 11, but may be one or three or more.

Incidentally, the object identifying unit 13, the estimated distance h _i from the plane FP sets the third threshold value relative to a point within a predetermined range, the distance h _i between the second object as compared to the third threshold set M2 You may execute the process of identifying the position information of. Furthermore, for example, the object identification unit 13, the distance (h _i) a first threshold value (e.g., D) from the estimated plane FP for points greater than performs the object identification process using the first threshold value, the 3 It is not necessary to calculate the threshold value. Further, the object identifying unit 13, a first threshold value (e.g., D) with a small fifth threshold than for points the distance from the estimated plane FP (h _i) is smaller than the fifth threshold value is a third threshold value It does not have to be derived. Object identification unit 13, for example, the distance from the estimated plane FP (h _i) is a point less than the fifth threshold value, identified as not a point on the second object M2 (a point on the first object M1 is) You may. Further, the function for calculating the third threshold value is not limited to the function of the equation (3), and can be changed as appropriate.

The object identification unit 13 identifies the position information of the second object M2 by a process that combines at least two of the object identification process of FIG. 4, the object identification process of FIG. 10, and the object identification process of FIG. You may. For example, the object identification unit 13 may determine which object identification process is to be executed by using a plurality of threshold values as described above. Further, the object identifying unit 13, the evaluation value d _i calculated as in Figure 10, by comparison with the threshold value V _i that is set as shown in FIG. 11, may identify the position information of the second object M2.

[Fifth Embodiment]
A fifth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 12 is a diagram showing a detection device according to a fifth embodiment. The detection device 1 according to the present embodiment estimates the surface SF of the first object M1 as an object surface (in this case, a plane) based on, for example, a condition specified by the user.

The detection device 1 includes an input unit 21 and a display unit 22. The input unit 21 receives input of various information (eg, data, command) to the processing device 3. The input unit 21 receives input of information (eg, information on the initial state regarding the object surface) that specifies the surface of the first object M1. The user can input various information to the processing device 3 by operating the input unit 21. The input unit 21 includes, for example, at least one of a keyboard, a mouse, a trackball, a touch panel, and a voice input device (eg, a microphone). The display unit 22 displays this image based on the image data output from the processing device 3. The display unit 22 includes, for example, a liquid crystal display. The display unit 22 and the input unit 21 may be a touch panel or the like.

Next, the plane estimation process according to the present embodiment will be described. In the present embodiment, in the position detection unit 2, the first state Q1 in which the second object M2 is arranged at the first position (eg, outside the target area AR) and the second object M2 are second as described with reference to FIG. The position information (eg, depth) of the object M in the target area AR is detected in each of the second state arranged in the second position (eg, in the target area) different from the first position. The surface estimation unit 12 executes the plane estimation process based on the detection result (eg, the measured value of the depth) of the position detection unit 2 corresponding to the first state Q1.

FIG. 13 is a diagram showing a plane estimation process according to the fifth embodiment. The depth map DM1 of FIG. 3 includes the detection result for the second object M2, whereas the depth map DM3 of FIG. 13 does not include the detection result for the second object M2. The point cloud data generation unit 15 executes point cloud processing on the depth map DM3 and generates third point cloud data. The surface estimation unit 12 estimates the plane based on the third point cloud data. For example, the plane estimation unit 12 estimates the plane FP3, the plane FP4, and the plane FP5 as candidates for the estimation plane FP. For example, the plane FP3 and the plane FP4 are wall surfaces around a space in which the second object M2 is assumed to be arranged, respectively. The plane FP4 is, for example, a floor surface, a ground surface, or a road surface of the above space.

The processing device 3 displays an image Im representing the plane FP3, the plane FP4, and the plane FP5 on the display unit 22. The input unit 21 receives input of information for designating (eg, selecting) a plane among the candidates for the estimated plane (plane FP3, plane FP4, plane FP5) as the information for designating the plane of the first object M1. For example, the user operates the mouse as the input unit 21 to move the pointer PT on a desired plane (eg, plane FP4) on the display unit 22, and designates the plane by a click operation or the like.

The input unit 21 outputs the information input by the user (eg, mouse movement information, click presence / absence information) to the processing device 3. The processing device 3 (plane estimation unit 12) identifies a designated plane (eg, plane FP4) based on the information output from the input unit 21. The surface estimation unit 12 adopts a plane specified as a plane designated by the user as the estimation plane FP, and stores information representing the estimation plane FP in the storage unit 14.

Further, the point cloud data generation unit 15 executes point cloud processing on the detection result of the position detection unit 2 corresponding to the second state Q2 (eg, the measured value of the depth, the depth map DM1 in FIG. 3), and the point cloud processing is performed. Generate 4-point cloud data. The object identification unit 13 executes the object identification process based on the estimated plane FP obtained from the third point cloud data and the fourth point cloud data.

The information for designating the surface of the first object M1 may include information for designating the position (eg, relative position) of the first object M1 with respect to the second object M2 in the target area AR. For example, the position of the first object M1 with respect to the second object M2 may be the position where the first object M1 exists, which is set based on the position where the second object M2 exists. Further, the position of the first object M1 with respect to the second object M2 may be the position where the first object M1 is set based on the position where the second object M2 is scheduled to be arranged. Further, the information for designating the surface of the first object M1 may include information for designating the direction of the first object M1 with respect to the second object M2 in the target area AR. For example, the direction of the first object M1 with respect to the second object M2 may be the direction in which the first object M1 exists with respect to the position where the second object M2 exists. Further, the direction of the first object M1 with respect to the second object M2 may be the direction in which the first object M1 exists with respect to the position where the second object M2 is scheduled to be arranged.

The plane estimation unit 12 may estimate the above plane based on preset calculation conditions. The input unit 21 may be used for inputting information when setting calculation conditions. The detection device 1 does not have to include one or both of the input unit 21 and the display unit 22. For example, the input unit 21 is a device that can be externally attached to the detection device 1, and the detection device 1 may be provided without the input unit 21 attached. Further, the display unit 22 is a device that can be externally attached to the detection device 1, and the detection device 1 may be provided in a state where the display unit 22 is not attached.

[Sixth Embodiment]
The sixth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 14 is a diagram showing a detection device according to a sixth embodiment. The detection device 1 according to the present embodiment includes a direction detection unit 23 that detects a vertical direction in the target region AR. The surface estimation unit 12 estimates an object surface (in this case, a plane) below the vertical direction detected by the direction detection unit 23 as an estimation plane.

The direction detection unit 23 includes, for example, at least one of an acceleration sensor, a geomagnetic sensor, and an inertial measurement unit (IMU). The direction detection unit 23 detects the direction of gravitational acceleration. The direction detection unit 23 may detect the posture of the detection device 1 (eg, the position detection unit 2). The surface estimation unit 12 calculates the direction (eg, vector) corresponding to the lower part in the vertical direction in the point cloud data based on the detection result of the direction detection unit 23. The surface estimation unit 12 derives a plane whose normal direction is the calculated direction as the estimated plane FP.

[7th Embodiment]
A seventh embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 15 is a diagram showing a detection device according to a seventh embodiment. In the present embodiment, the processing device 3 includes a rendering processing unit 24.

The rendering processing unit 24 includes, for example, a graphics processing unit (GPU). The rendering processing unit 24 may have a mode in which the CPU and the memory execute each processing according to the image processing program. The rendering processing unit 24 performs at least one of drawing processing, texture mapping processing, and shading processing, for example.

In the drawing process, the rendering processing unit 24 can calculate, for example, an estimated image (eg, a reconstructed image) of the shape defined in the shape information of the model information viewed from an arbitrary viewpoint. In the following description, the shape indicated by the shape information is referred to as a model shape. The rendering processing unit 24 can reconstruct the estimated image from the model information (eg, shape information) by, for example, drawing processing. The rendering processing unit 24 stores, for example, the calculated estimated image data in the storage unit 56.

Further, in the texture mapping process, the rendering processing unit 24 can calculate, for example, an estimated image in which the image indicated by the texture information of the model information is pasted on the surface of the object on the estimated image. The rendering processing unit 24 can calculate an estimated image in which a texture different from the object is attached to the surface of the object on the estimated image. In the shading process, the rendering processing unit 24 can calculate, for example, an estimated image in which a shadow formed by the light source indicated by the light source information of the model information is added to the object on the estimated image. Further, in the shading process, the rendering processing unit 24 can calculate, for example, an estimated image in which a shadow formed by an arbitrary light source is added to an object on the estimated image.

The processing device 3 causes the display unit 22 to display the estimated image generated by the rendering processing unit 24. For example, the rendering processing unit 24 executes drawing processing based on the conditions (eg, the position of the viewpoint, the direction of the viewpoint) specified by the user using the input unit 21 to generate an estimated image, and displays the estimated image. Display on 22. The user can see the estimated image generated under the specified conditions on the display unit 22.

In the embodiment, since the object identification unit 13 can identify the second object M2 with respect to the first object M1, the model generation unit 11 generates shape information by segmenting the second object M2 with high accuracy, for example. can do. Since the rendering processing unit 24 generates an estimated image based on the shape information generated by segmenting the second object M2 with high accuracy, for example, the second object M2 can be represented with high accuracy in the estimated image. ..

The processing device 3 does not have to include the rendering processing unit 24. The rendering processing unit 24 may be provided outside the processing device 3. The processing device 3 may output the model information generated by the model generation unit 11 based on the identification result of the object identification unit 13 to the rendering processing unit 24 provided outside the processing device 3.

[8th Embodiment]
An eighth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 16 is a diagram showing a detection device according to an eighth embodiment. In the present embodiment, the processing device 3 includes a posture estimation unit 25. The posture estimation unit 25 estimates the posture of the second object M2 using the model information generated by the model generation unit 11 based on the identification result of the object identification unit 13 (posture analysis processing). Here, it is assumed that the second object M2 is a human body and the surface SF as an object surface is a floor surface.

The posture estimation unit 25 generates position information of a characteristic portion (eg, feature portion, feature point) of the human body M2. The characteristic portion of the human body M2 is, for example, a portion of the human body M2 that can be distinguished from other parts. Characteristic parts of the human body M2 include, for example, at least one of the ends of the human body (eg, toes), joints (eg, knees), or between the ends and the joints or between the two joints. .. The position information of the feature portion includes, for example, the coordinates of a point representing the feature portion (eg, three-dimensional coordinates).

The posture estimation unit 25 uses, for example, recognition processing (eg, pattern recognition, shape recognition, skeleton recognition) using shape information (eg, point cloud data) generated by the model generation unit 11 (eg, point cloud data generation unit 15). ) Etc. to generate the position information of the above-mentioned feature portion. The posture estimation unit 25 calculates the coordinates of the points representing the feature parts by the above recognition process, and at least a part of the human body (eg, the toes, knees, hip joints) depending on the relative positions of the plurality of feature parts (eg, toes, knees, hip joints). Estimate the posture of the legs).

When the human body M2 walks on the floor surface SF, for example, there is a state in which the toes (the portion from the ankle to the tip) of the foot in contact with the floor surface SF are arranged near the floor surface SF. First, the surface estimation unit 12 according to the embodiment estimates the floor surface SF on a plane using the above position information detected by the position detection unit 2. After that, the object identification unit 13 includes a portion where the distance from the floor surface SF estimated by the surface estimation unit 12 is equal to or greater than the threshold value and a portion where the distance between the floor surface SF estimated by the surface estimation unit 12 is less than the threshold value (near the floor surface). The contrast with the part) is emphasized. By using the identification result of the object identification unit 13, the model generation unit 11 can reduce the extraction error of the foot and improve the separation accuracy. The model generation unit 11 can generate, for example, highly accurate shape information of a portion (eg, hand, toe) of the human body M2 that comes into contact with the floor surface SF. The posture estimation unit 25 can extract the characteristic portion of the human body M2 with high accuracy based on the highly accurate shape information, and can estimate the posture of the human body M2 with high accuracy.

The posture estimation unit 25 may detect (eg, estimate) the movement (eg, movement) of the human body M2. The posture estimation unit 25 includes sports such as fencing, baseball, soccer, golf, kendo, American football, ice hockey, and gymnastics, dance, performing arts, running, exercise, yoga, body building, walking or posing such as fashion shows, games, etc. The movement of the human body M2 may be detected in person authentication or work. For example, the posture estimation unit 25 may detect the walking motion of the human body M2 based on the posture of the human body M2.

The processing device 3 does not have to include the posture estimation unit 25. The posture estimation unit 25 may be provided outside the processing device 3. The processing device 3 may output the model information generated by the model generation unit 11 based on the identification result of the object identification unit 13 to the posture estimation unit 25 provided outside the processing device 3.

In the above-described embodiment, the processing device 3 includes, for example, a computer system. The processing device 3 reads a processing program (detection program) stored in the storage unit 14 and executes various processes according to the processing program. This processing program detects the position information of each point on the object M including the first object M1 and the second object M2 in the target area AR on the computer, and the object surface (estimated plane FP) of the first object M1 based on the detection result. ) And at least a part of the second object M2 in contact with the object surface (estimated plane FP) based on the distance to each point on the object M with respect to the object surface (estimated plane FP). Let it run. The processing program may be recorded and provided on a computer-readable storage medium (eg, non-transitory tangible media).

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. For example, when the detection device 1 includes the position detection unit 2, the surface estimation unit 12, and the object identification unit 13, the second object M2 can be identified and detected as the first object M1. The detection device 1 does not have to include at least a part of a portion excluding the position detection unit 2, the surface estimation unit 12, and the object identification unit 13. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. The detection device 1 may include a portion that executes a predetermined process in addition to the position detection unit 2, the surface estimation unit 12, and the object identification unit 13. In addition, to the extent permitted by law, the disclosure of all documents cited in the above-mentioned embodiments and the like shall be incorporated as part of the description in the main text.

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, to the extent permitted by law, the disclosure of Japanese Patent Application No. 2018-133517 and all the documents cited in the above-described embodiments will be incorporated as part of the description of the main text.

1 ... Detection device, 2 ... Position detection unit, 3 ... Processing device, 4 ... Detection unit, M ... Object, M1 ... First object, M2 ... Second object , 12 ... plane estimation unit, 13 ... object identification unit, 14 ... storage unit, 15 ... point cloud data generation unit, 21 ... input unit, 22 ... display unit, AR ...・ ・ Target area, FP ・ ・ ・ Estimated plane

Claims (19)

A position detection unit that detects the position information of each point on the object including the first object and the second object in the target area, and
At least of the second object in contact with the object surface based on the distance between the object surface of the first object based on the detection result of the position detection unit and each point on the object detected by the position detection unit. A detection device including an object identification unit that identifies a part of the object surface. A point cloud data generation unit that generates point cloud data as position information of each point on the object is provided.
The object surface is obtained based on the point cloud data generated by the point cloud data generation unit.
The detection device according to claim 1. The position detection unit includes a detection unit that detects the distance between each point on the object and a predetermined point as position information of each point on the object.
The point cloud data generation unit generates the point cloud data based on the detection result of the detection unit.
The detection device according to claim 2. The object identification unit identifies a point at which the distance from the object surface is equal to or greater than a predetermined value from the second object.
The detection device according to any one of claims 1 to 3. The object identification unit determines the first distance between the first point and the object surface of the object based on the second distance between the second point around the first point and the object surface. Is calculated, and the evaluation value is compared with the first threshold value to determine whether or not the first point belongs to the second object.
The detection device according to any one of claims 1 to 4. When the second distance is larger than the second threshold value, the object identification unit calculates, as the evaluation value, a value obtained by increasing the first distance based on the second distance.
The detection device according to claim 5. When the second distance is larger than the second threshold value, the object identification unit calculates, as the evaluation value, a value obtained by adding a value that decreases with respect to the second distance to the first distance.
The detection device according to claim 6. The object identification unit sets a third threshold value based on the distance between the second point around the first point in the object and the object surface, and the distance between the first point and the object surface and the third object surface. It is determined whether or not the first point belongs to the second object by comparing with the threshold value.
The detection device according to any one of claims 1 to 7. A direction detection unit for detecting the vertical direction in the target region is provided.
The lower surface in the vertical direction detected by the direction detection unit is defined as the object surface.
The detection device according to any one of claims 1 to 8. Derivation of the mathematical formula representing the object surface,
The detection device according to any one of claims 1 to 9. The object identification unit calculates the distance between the object surface and each point on the object by using the mathematical formula.
The detection device according to claim 10. It is provided with an input unit that accepts input of information that specifies the surface of the first object.
The object surface is determined based on the information input to the input unit.
The detection device according to any one of claims 1 to 11. The object surface is determined based on a change in the detection result of the position detection unit due to the movement of the second object.
The detection device according to any one of claims 1 to 12. A model generation unit that generates model information including shape information of the second object based on the position information of the second object identified by the object identification unit is provided.
The detection device according to any one of claims 1 to 13. The object surface includes a floor surface or the ground in the target area.
The second object includes the human body
The object identification unit identifies the position information of a portion of the human body that comes into contact with the object surface.
The detection device according to any one of claims 1 to 14. Based on the distance between the object surface of the first object and each point on the object based on the detection result of detecting the position information of each point on the object including the first object and the second object in the target area, the said A processing device including an object identification unit that identifies at least a part of the second object in contact with the object surface with respect to the object surface. To detect the position information of each point on the object including the first object and the second object in the target area, and
At least a part of the second object in contact with the object surface is identified with respect to the object surface based on the distance between the object surface of the first object and each point on the object based on the detection result. And how to detect, including. On the computer
Based on the distance between the object surface of the first object and each point on the object based on the detection result of detecting the position information of each point on the object including the first object and the second object in the target area, the said A processing program that executes identification of at least a part of the second object in contact with the object surface with respect to the object surface. A detection unit that detects the position information of each point on the object including the first object and the second object in the target area, and
A detection device including an identification unit that identifies the first object and the second object based on the position information.

Images