WO2022123786A1 - Reliability estimation device, position estimation device, and reliability estimation method - Google Patents

Abstract

A reliability estimation device (20) estimates the reliability of an estimated position of an object estimated on the basis of a camera image, and is provided with: a feature quantity calculation unit (21) that calculates a feature quantity of the object on the basis of feature point information pertaining to a plurality of feature points of the object in the camera image; and a reliability calculation unit (22) that calculates a reliability on the basis of the feature quantity calculated by the feature quantity calculation unit (21).

Description

Reliability estimation device, position estimation device, and reliability estimation method

This disclosure relates to a reliability estimation device.

A three-dimensional position estimation algorithm for estimating the three-dimensional position of an object based on an image has been developed (in the present specification, the three-dimensional position of an object is synonymous with a three-dimensional posture). For example, in Non-Patent Document 1, by applying a neural network model to a camera image obtained by a monocular camera taking an object, the object in the camera image is recognized and the position of a feature point of the object is recognized. A three-dimensional position estimation method for estimating the three-dimensional position of an object based on the estimated position of the feature point is described. In the following, the position of the object estimated by such a three-dimensional position estimation method (for example, the position of the feature point of the object or the three-dimensional position of the object) is referred to as an estimated position.

S. Peng, Y. Liu, Q. Huang, H. Bao, X. Zhou, "PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation," In Computer Vision and Pattern Recognition (CVPR), 2019.

In the conventional 3D position estimation method that applies a neural network model to an image, the reliability of the estimated position is determined by using the same neural network model as the machine learning model used to estimate the estimated position. It may be calculated and the calculated reliability may be used to evaluate the estimated position.

In that case, it is considered that there is a correlation between the reliability of the estimated position and the reliability of the reliability itself. Therefore, when the reliability of the estimated position is poor, the reliability of the reliability itself is also poor, and there is a problem that the validity of the evaluation of the estimated position using the reliability becomes uncertain.
The present disclosure has been made to solve the above-mentioned problems, and an object of the present disclosure is to provide a technique for improving the reliability of the estimated position itself.

The reliability estimation device according to the present disclosure is a reliability estimation device that estimates the reliability of the estimated position of the object estimated based on the camera image, and is used for feature point information regarding a plurality of feature points of the object in the camera image. Based on this, it includes a feature amount calculation unit that calculates the feature amount of the object, and a reliability calculation unit that calculates the reliability based on the feature amount calculated by the feature amount calculation unit.

According to the present disclosure, the reliability of the estimated position itself can be improved.

It is a block diagram which shows the structure of the position estimation system which concerns on Embodiment 1. FIG. It is a flowchart which shows the reliability estimation method by the position estimation apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows the detail of the feature amount calculation method in the reliability estimation method by the position estimation apparatus which concerns on Embodiment 1. FIG. FIG. 4A is a block diagram showing a hardware configuration that realizes the function of the position estimation device according to the first embodiment. FIG. 4B is a block diagram showing a hardware configuration for executing software that realizes the function of the position estimation device according to the first embodiment.

Hereinafter, in order to explain the present disclosure in more detail, a mode for carrying out the present disclosure will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1 is a block diagram showing a configuration of a position estimation system 100 according to a first embodiment. As shown in FIG. 1, the position estimation system 100 includes a position estimation device 1, N cameras 2 (N is a positive integer), and a storage device 3. The position estimation device 1 includes a camera selection unit 40, a feature point estimation unit 10, a reliability estimation device 20, and a position calculation unit 30. The reliability estimation device 20 includes a feature amount calculation unit 21 and a reliability calculation unit 22.

The N cameras 2 are composed of each camera 2 from the first camera 2 to the Nth camera 2. Each of the N cameras 2 acquires a camera image (camera images D11 to D1N) by photographing an object. Each of the N cameras 2 outputs the acquired camera image to the camera selection unit 40 of the position estimation device 1.

The camera selection unit 40 selects at least one camera image D2 from a plurality of camera images. More specifically, in the first embodiment, the camera selection unit 40 selects at least one camera image D2 from the plurality of camera images D11 to D1N acquired by the N cameras 2. The detailed configuration of the camera selection unit 40 will be described later. The camera selection unit 40 outputs the selected camera image D2 to the feature point estimation unit 10 and the feature amount calculation unit 21, respectively.

The feature point estimation unit 10 estimates feature point information regarding a plurality of feature points of an object based on a camera image. More specifically, in the first embodiment, the feature point estimation unit 10 estimates the feature point information based on the camera image D2 selected by the camera selection unit 40.

More specifically, in the first embodiment, the storage device 3 stores PVNet, which is a convolutional neural network model for object position detection. The feature point estimation unit 10 applies PVNet stored in the storage device 3 to the camera image D2 selected by the camera selection unit 40, so that each estimated position D3 of a plurality of feature points can be used as feature point information. , The dispersion D4 of the candidate point group of the feature points, and the object area D5 which is the area occupied by the object in the camera image are estimated.

The feature point estimation unit 10 outputs each estimated position D3 of the estimated plurality of feature points to the position calculation unit 30. Further, the feature point estimation unit 10 outputs each estimated position D3 of the estimated plurality of feature points, the variance D4 of the candidate point group of the feature points, and the object region D5 to the feature amount calculation unit 21.

The feature amount calculation unit 21 calculates the feature amount of the object based on the feature point information regarding a plurality of feature points of the object in the camera image. More specifically, in the first embodiment, the feature amount calculation unit 21 calculates the feature amount of the object based on the feature point information estimated by the feature point estimation unit 10.

More specifically, in the first embodiment, the feature amount calculation unit 21 includes m feature amount calculation units from the first feature amount calculation unit 201 to the mth feature amount calculation unit 20 m.
The first feature amount calculation unit 201 calculates the feature amount D61 of the object based on the variance D4 of the candidate point cloud of the feature points estimated by the feature point estimation unit 10. In the first embodiment, the feature amount D61 calculated by the first feature amount calculation unit 201 is a feature amount proportional to the reciprocal of the variance D4 of the candidate point group of the feature points. In other words, the feature amount D61 calculated by the first feature amount calculation unit 201 is a feature amount that is inversely proportional to the variance D4 of the candidate point cloud of the feature points. The first feature amount calculation unit 201 outputs the calculated feature amount D61 to the reliability calculation unit 22.

The second feature amount calculation unit 202 inputs a region composed of a plurality of feature points and an object region based on the estimated positions D3 and the object region D5 of the plurality of feature points estimated by the feature point estimation unit 10. The feature amount D62 which is a function to be taken is calculated. More specifically, in the first embodiment, the second feature amount calculation unit 202 has a plurality of feature points based on the estimated positions D3 and the object area D5 of the plurality of feature points estimated by the feature point estimation unit 10. The area ratio between the convex plane composed of and the object area is calculated, and the feature amount proportional to the inverse number of the calculated area ratio or the feature amount D62 proportional to the inverse number of the difference between the calculated area ratio and 1 is calculated. .. Here, the "convex plane composed of a plurality of feature points" is, for example, the largest convex plane in which a plurality of feature points are stretched, or the smallest convex plane containing all the plurality of feature points. The second feature amount calculation unit 202 outputs the calculated feature amount D62 to the reliability calculation unit 22.

Each feature amount calculation unit from the third feature amount calculation unit to the mth feature amount calculation unit 20m (not shown) includes a camera image D2 selected by the camera selection unit 40 and a plurality of feature points estimated by the feature point estimation unit 10. Based on the information of at least one of each estimated position D3, the dispersion D4 of the candidate point group of the feature points, and the object area D5, the feature amount (feature amount D63 to D6m) that can be calculated from the at least one information is calculated. do. Each feature amount calculation unit from the third feature amount calculation unit (not shown) to the mth feature amount calculation unit 20 m outputs the calculated feature amount to the reliability calculation unit 22.

The reliability calculation unit 22 calculates the reliability of the estimated position of the object based on the feature amount calculated by the feature amount calculation unit 21. More specifically, in the first embodiment, the reliability calculation unit 22 correlates with the estimation error of the estimated position of the object based on the feature amounts (feature amounts D61 to D6m) calculated by the feature amount calculation unit 21. The reliability D7 (for example, the reliability D7 having a positive correlation with the estimation error) is calculated.

More specifically, in the first embodiment, the storage device 3 stores a neural network model that calculates the reliability from the feature amount. The reliability calculation unit 22 applies the neural network model stored in the storage device 3 to the feature amounts (feature amounts D61 to D6m) calculated by the feature amount calculation unit 21, and thereby, the estimated position of the object. The reliability D7, which is inversely proportional to the estimation error of, is calculated. The reliability calculation unit 22 outputs the calculated reliability D7 to the outside of the camera selection unit 40 and the position estimation device 1, respectively.

The camera selection unit 40 described above selects at least one camera image D2 from the N camera images D11 to D1N based on the reliability D7 calculated by the reliability calculation unit 22. The camera selection unit 40 outputs the camera image D2 selected based on the reliability D7 to the feature point estimation unit 10 and the feature amount calculation unit 21, respectively.

The position calculation unit 30 calculates the estimated position D8 of the object based on the feature point information estimated by the feature point estimation unit 10. More specifically, in the first embodiment, the storage device 3 stores the position of the feature point of the object. The position calculation unit 30 solves the PnP problem by solving the PnP problem based on each estimated position D3 of the plurality of feature points estimated by the feature point estimation unit 10 and the feature point position of the object stored in the storage device 3. The estimated position D8 of is calculated. An example of the estimated position D8 calculated by the position calculation unit 30 is a three-dimensional position (three-dimensional posture) of an object. The position calculation unit 30 outputs the calculated estimated position D8 to the outside of the position estimation device 1.
The reliability D7 output by the reliability calculation unit 22 to the outside of the position estimation device 1 is used, for example, for the evaluation of the estimated position D8 output to the outside of the position estimation device 1 by the position calculation unit 30.

Hereinafter, the operation of the position estimation device 1 according to the first embodiment will be described with reference to the drawings. FIG. 2 is a flowchart showing a reliability estimation method by the position estimation device 1 according to the first embodiment. Before each step described below, it is assumed that each of the N cameras 2 acquires a camera image by photographing an object and outputs the acquired camera image to the camera selection unit 40.

As shown in FIG. 2, the camera selection unit 40 selects at least one camera image D2 from a plurality of camera images (step ST1). The camera selection unit 40 outputs the selected camera image D2 to the feature point estimation unit 10 and the feature amount calculation unit 21, respectively.

In step ST1, the camera selection unit 40 selects an arbitrary camera image D2 from a plurality of camera images when the reliability D7 has not yet been calculated by the step of step ST4 described later. Alternatively, in step ST1, when the reliability D7 has already been calculated by the step of step ST4 described later, the camera selection unit 40 has at least one of N camera images D11 to D1N based on the reliability D7. Select one camera image D2.

Next, the feature point estimation unit 10 uses the camera image D2 selected by the camera selection unit 40 as feature point information such as each estimated position D3 of a plurality of feature points, a dispersion D4 of a candidate point group of the feature points, and a feature point cloud. The object region D5 is estimated (step ST2). The feature point estimation unit 10 outputs each estimated position D3 of the estimated plurality of feature points, the variance D4 of the candidate point group of the feature points, and the object region D5 to the feature amount calculation unit 21.

Next, the feature amount calculation unit 21 includes a camera image D2 selected by the camera selection unit 40, each estimated position D3 of a plurality of feature points estimated by the feature point estimation unit 10, a dispersion D4 of a group of candidate points of the feature points, and an object. The feature amounts D61 to D6m of the object are calculated in the area D5 based on the information of at least one of the feature amount calculation units 21 (step ST3). The feature amount calculation unit 21 outputs the calculated feature amounts D61 to D6m to the reliability calculation unit 22.

Next, the reliability calculation unit 22 calculates the reliability D7 that correlates with the estimation error of the estimated position of the object based on the feature amounts D61 to D6m calculated by the feature amount calculation unit 21 (step ST4). Although not shown, the position estimation device 1 repeatedly executes a series of steps from step ST1 to step ST4.

The details of step ST3 will be described below. FIG. 3 is a flowchart showing details of the feature amount calculation method in step ST3.
As shown in FIG. 3, the first feature amount calculation unit 201 is based on the variance D4 of the candidate point group of the feature points estimated by the feature point estimation unit 10, and the feature amount of the object proportional to the reciprocal of the variance D4. Calculate D61 (step ST31). The first feature amount calculation unit 201 outputs the calculated feature amount D61 to the reliability calculation unit 22.

The second feature amount calculation unit 202 includes a convex plane composed of a plurality of feature points and an object area based on the estimated positions D3 and the object area D5 of the plurality of feature points estimated by the feature point estimation unit 10. The area ratio is calculated, and the feature amount proportional to the inverse number of the calculated area ratio or the feature amount D62 proportional to the inverse number of the difference between the calculated area ratio and 1 is calculated (step ST32). The second feature amount calculation unit 202 outputs the calculated feature amount D62 to the reliability calculation unit 22.

Each feature amount calculation unit from the third feature amount calculation unit to the mth feature amount calculation unit 20m is a camera image D2 selected by the camera selection unit 40 and each of a plurality of feature points estimated by the feature point estimation unit 10. The other feature quantities D63 to D6m are calculated based on the information of at least one of the estimated position D3, the dispersion D4 of the candidate point group of the feature points, and the object region D5 (step ST33). Each feature amount calculation unit from the third feature amount calculation unit to the mth feature amount calculation unit 20m outputs the calculated feature amount to the reliability calculation unit 22.

Hereinafter, a specific example of the camera selection method (step ST1) in the reliability estimation method by the position estimation device 1 according to the first embodiment will be described in detail. It is assumed that the step of step ST1 described below is performed again after performing a series of steps from step ST1 to step ST4 once or more.

In the specific example, in step ST1, the camera selection unit 40 has at least one camera out of N camera images D11 to D1N based on the reliability D7 calculated by the reliability calculation unit 22 in step ST4 described above. Select image D2. More specifically, in the specific example, the camera selection unit 40 uses one camera image based on the N camera images D11 to D1N and the reliability D7 calculated by the reliability calculation unit 22. Select either one of the modes for estimating the position or the mode for estimating the position by sequentially using all the camera images.

More specifically, in the specific example, in step ST1, when the reliability D7 calculated by the reliability calculation unit 22 is equal to or greater than the threshold value, the camera selection unit 40 is a camera from the camera 2 corresponding to the previously selected camera image. Select image D2. In other words, when the reliability D7 calculated by the reliability calculation unit 22 is equal to or greater than the threshold value, the camera selection unit 40 selects the camera image D2 from the camera 2 that captured the previously selected camera image. In that case, in step ST2 described above, as described above, the feature point estimation unit 10 estimates the feature point information based on the camera image D2 selected by the camera selection unit 40.

On the other hand, in step ST1, the camera selection unit 40 selects all N camera images D11 to D1N when the reliability calculated by the reliability calculation unit 22 is less than the threshold value. Next, in step ST2 described above, when the camera selection unit 40 selects all N camera images D11 to D1N in step ST1, the feature point estimation unit 10 estimates the feature point information for each camera image.

Next, in step ST3 described above, the feature amount calculation unit 21 calculates the feature amount of the object for each camera image based on the feature point information estimated by the feature point estimation unit 10 for each camera image in step ST2. ..
Next, in step ST4 described above, the reliability calculation unit 22 calculates the reliability of the estimated position of the object for each camera image based on the feature amount calculated by the feature amount calculation unit 21 for each camera image.

Next, the position estimation device 1 returns to the step of step ST1 described above, and the camera selection unit 40 is a camera calculated by the reliability calculation unit 22 when the reliability calculation unit 22 calculates the reliability for each camera image. At least one camera image D2 is selected from the N camera images D11 to D1N based on the maximum reliability of the reliability of each image. More specifically, for example, when the reliability calculation unit 22 calculates the reliability for each camera image, the camera selection unit 40 has the maximum reliability of the reliability for each camera image calculated by the reliability calculation unit 22. The camera image D2 taken by the camera 2 corresponding to the degree is selected.

Each function of the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1 is realized by a processing circuit. To. That is, the position estimation device 1 includes a processing circuit for executing the processing of each step shown in FIGS. 2 and 3. This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in the memory.

FIG. 4A is a block diagram showing a hardware configuration that realizes the function of the position estimation device 1. FIG. 4B is a block diagram showing a hardware configuration for executing software that realizes the functions of the position estimation device 1.

When the processing circuit is the processing circuit 50 of the dedicated hardware shown in FIG. 4A, the processing circuit 50 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific Integrated Circuitd). Circuit), FPGA (Field-Programmable Gate Array) or a combination thereof is applicable.

The functions of the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1 are realized by separate processing circuits. Alternatively, these functions may be integrated into one processing circuit.

When the processing circuit is the processor 51 shown in FIG. 4B, the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20 in the position estimation device 1, and the reliability calculation unit 22. Each function of the position calculation unit 30 is realized by software, firmware, or a combination of software and firmware.
The software or firmware is described as a program and stored in the memory 52.

By reading and executing the program stored in the memory 52, the processor 51 reads and executes the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 of the reliability estimation device 20, and the reliability in the position estimation device 1. Each function of the calculation unit 22 and the position calculation unit 30 is realized. That is, the position estimation device 1 stores a memory 52 for storing a program in which the processing of each step shown in FIGS. 2 and 3 is executed as a result when each of these functions is executed by the processor 51. Be prepared.

These programs are the procedures or methods of the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1. Let the computer run. The memory 52 causes the computer to function as a camera selection unit 40, a feature point estimation unit 10, a feature amount calculation unit 21 and a reliability calculation unit 22 of the reliability estimation device 20, and a position calculation unit 30 in the position estimation device 1. The program may be stored in a computer-readable storage medium.

The processor 51 corresponds to, for example, a CPU (Central Processing Unit), a processing device, a computing device, a processor, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

The memory 52 may include, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EPROM (Electrically-volatile) semiconductor, or an EPROM (Electrically-EPROM). This includes hard disks, magnetic disks such as flexible disks, flexible disks, optical discs, compact disks, mini disks, CDs (Compact Disc), DVDs (Digital Versailles Disc), and the like.

Dedicated hardware for each function of the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1. It may be realized by hardware and partly by software or firmware.

For example, the functions of the camera selection unit 40, the feature point estimation unit 10, and the position calculation unit 30 are realized by a processing circuit as dedicated hardware. The features of the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20 may be realized by the processor 51 reading and executing the program stored in the memory 52.
As described above, the processing circuit can realize each of the above functions by hardware, software, firmware, or a combination thereof.

As described above, the reliability estimation device 20 according to the first embodiment is a reliability estimation device 20 that estimates the reliability of the estimated position of the object estimated based on the camera image, and is the reliability estimation device 20 of the object in the camera image. The reliability of the estimated position of the object is calculated based on the feature amount calculation unit 21 that calculates the feature amount of the object based on the feature point information about a plurality of feature points and the feature amount calculated by the feature amount calculation unit 21. A reliability calculation unit 22 is provided.

According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature amount of the object calculated from the feature point information. As a result, the reliability of the estimated position itself does not depend on the reliability of the estimated position, as compared with the case of calculating the reliability directly from the estimated position, so that the reliability of the estimated position itself can be improved. can. Therefore, for example, even when an image for which it is difficult to estimate the estimated position is input, the reliability of the estimated position itself can be improved.

The feature point information in the reliability estimation device 20 according to the first embodiment includes the variance of the candidate point cloud of the feature point, and the feature amount calculation unit 21 is based on the variance of the candidate point cloud of the feature point, and the feature of the object. Calculate the amount.
According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature amount of the object calculated from the variance of the candidate point group of the feature points. As a result, the reliability of the reliability of the estimated position itself can be suitably improved.

The feature amount calculation unit 21 in the reliability estimation device 20 according to the first embodiment calculates a feature amount proportional to the reciprocal of the variance.
According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature amount proportional to the reciprocal of the variance. As a result, the reliability of the reliability of the estimated position itself can be suitably improved.

The feature point information in the reliability estimation device 20 according to the first embodiment includes each estimated position of a plurality of feature points and an object area which is an area occupied by an object in a camera image, and the feature amount calculation unit 21 has a plurality of features. Based on each estimated position of the feature point and the object area, a feature amount that is a function of taking a region composed of a plurality of feature points and the object region as an input is calculated.

According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature amount that is a function that takes the region composed of a plurality of feature points and the object region as inputs. As a result, the reliability of the reliability of the estimated position itself can be suitably improved.

The feature point information in the reliability estimation device 20 according to the first embodiment includes each estimated position of a plurality of feature points and an object area which is an area occupied by an object in a camera image, and the feature amount calculation unit 21 has a plurality of features. Based on each estimated position of the feature point and the object area, the area ratio between the convex plane composed of a plurality of feature points and the object area is calculated, and the feature amount or calculation is proportional to the inverse of the calculated area ratio. The feature amount proportional to the inverse number of the difference between the area ratio and 1 is calculated.

According to the above configuration, based on each estimated position of a plurality of feature points, a feature amount proportional to the reciprocal of the area ratio calculated from the object area, or a feature amount proportional to the reciprocal of the difference between the area ratio and 1. Calculate the reliability of the estimated position of the object. As a result, the reliability of the reliability of the estimated position itself can be suitably improved.

The feature amount calculation unit 21 in the reliability estimation device 20 according to the first embodiment calculates a plurality of feature amounts of the object, and the reliability calculation unit 22 is based on the feature amounts calculated by the feature amount calculation unit 21. Calculate the reliability that correlates with the estimation error of the estimated position of the object.
According to the above configuration, the estimated position of the object can be evaluated using the reliability that correlates with the estimation error of the estimated position of the object.

The position estimation device 1 according to the first embodiment includes a reliability estimation device 20 according to the first embodiment, a feature point estimation unit 10 that estimates feature point information based on a camera image, and a feature point estimation unit 10. A position calculation unit 30 for calculating an estimated position of an object based on the estimated feature point information is provided, and the feature amount calculation unit 21 is characterized by an object based on the feature point information estimated by the feature point estimation unit 10. Calculate the amount.

According to the above configuration, each of the above-mentioned effects of the reliability estimation device 20 according to the first embodiment can be realized in the position estimation device 1. For example, the position estimation device 1 according to the first embodiment is applicable to the position estimation of an object by a camera image. For example, it can be used for an automatic burning vehicle or an FA device such as a robot arm.

For example, when gripping an object with a robot arm, it is necessary to detect the position of the object. Therefore, when an image taken by a monocular camera or the like attached to the arm is applied as the above-mentioned camera image to the reliability estimation device 20 according to the first embodiment, an image in which position estimation is difficult due to poor shooting conditions is input. However, the above-mentioned reliability with high reliability can be calculated, and the estimated position can be evaluated based on the reliability. Therefore, stable robot control can be realized.

The position estimation device 1 according to the first embodiment further includes a camera selection unit 40 that selects at least one camera image from a plurality of camera images based on the reliability calculated by the reliability calculation unit 22. The point estimation unit 10 estimates feature point information based on the camera image selected by the camera selection unit 40.

According to the above configuration, feature point information is estimated from the camera image selected based on the reliability of the estimated position. This makes it possible to improve the reliability of the estimated feature point information. Therefore, it is possible to improve the reliability of the estimated position and the reliability based on the feature point information.

When the reliability calculated by the reliability calculation unit 22 is equal to or greater than the threshold value, the camera selection unit 40 in the position estimation device 1 according to the first embodiment selects a camera image from the camera 2 corresponding to the previously selected camera image. If the reliability calculated by the reliability calculation unit 22 is less than the threshold value, all the plurality of camera images are selected.

According to the above configuration, a camera image suitable for estimating the estimated position can be suitably selected. Therefore, it is possible to improve the reliability of the feature point information, the estimated position, and the reliability based on the camera image.

When the camera selection unit 40 selects all a plurality of camera images, the feature point estimation unit 10 in the position estimation device 1 according to the first embodiment estimates feature point information for each camera image, and the feature amount calculation unit 21. Calculates the feature amount of the object for each camera image based on the feature point information estimated by the feature point estimation unit 10 for each camera image, and the reliability calculation unit 22 has the feature amount calculation unit 21 for each camera image. The reliability of the estimated position of the object is calculated for each camera image based on the feature amount calculated in the above, and the camera selection unit 40 calculates the reliability for each camera image when the reliability calculation unit 22 calculates the reliability for each camera image. At least one camera image is selected from a plurality of camera images based on the maximum reliability of the reliability of each camera image calculated by the calculation unit 22.

According to the above configuration, even if the previously selected camera image is not suitable for estimating the estimated position, the camera image suitable for estimating the estimated position can be selected next time. Therefore, it is possible to improve the reliability of the feature point information, the estimated position, and the reliability based on the camera image.

The reliability estimation method according to the first embodiment is a reliability estimation method for estimating the reliability of an estimated position of an object estimated based on a camera image, and is a feature point relating to a plurality of feature points of the object in the camera image. It includes a feature amount calculation step for calculating the feature amount of the object based on the information, and a reliability calculation step for calculating the reliability based on the feature amount calculated in the feature amount calculation step.
According to the above configuration, the same effect as the above-mentioned effect played by the reliability estimation device 20 according to the first embodiment is obtained.
It is possible to modify any component of the embodiment or omit any component of the embodiment.

The reliability estimation device according to the present disclosure can be used as a position estimation device because the reliability of the reliability of the estimated position itself can be improved.

1 position estimation device, 2 camera, 3 storage device, 10 feature point estimation unit, 20 reliability estimation device, 21 feature quantity calculation unit, 22 reliability calculation unit, 30 position calculation unit, 40 camera selection unit, 50 processing circuit, 51 processor, 52 memory, 100 position estimation system, 201 first feature amount calculation unit, 202 second feature amount calculation unit, 20m second feature amount calculation unit.

Claims (10)

It is a reliability estimation device that estimates the reliability of the estimated position of an object estimated based on a camera image.
A feature amount calculation unit that calculates a feature amount of the object based on feature point information about a plurality of feature points of the object in the camera image.
A reliability estimation device comprising a reliability calculation unit for calculating the reliability based on the feature amount calculated by the feature amount calculation unit. The feature point information includes the variance of the candidate point group of the feature points.
The reliability estimation device according to claim 1, wherein the feature amount calculation unit calculates the feature amount based on the variance of the candidate point group of the feature points. The reliability estimation device according to claim 2, wherein the feature amount calculation unit calculates a feature amount proportional to the reciprocal of the variance. The feature point information includes each estimated position of a plurality of feature points and an object region which is a region occupied by the object in the camera image.
The feature amount calculation unit obtains a feature amount that is a function of taking a region composed of the plurality of feature points and the object region as inputs based on each estimated position of the plurality of feature points and the object region. The reliability estimation device according to claim 1, wherein the calculation is performed. The feature amount calculation unit calculates a plurality of the feature amounts, and
The first aspect of the present invention, wherein the reliability calculation unit calculates a reliability that correlates with an estimation error of an estimated position of the object based on a plurality of feature amounts calculated by the feature amount calculation unit. Reliability estimation device. The reliability estimation device according to claim 1 and
A feature point estimation unit that estimates the feature point information based on the camera image,
A position calculation unit for calculating an estimated position of the object based on the feature point information estimated by the feature point estimation unit is provided.
The feature amount calculation unit is a position estimation device, characterized in that the feature amount is calculated based on the feature point information estimated by the feature point estimation unit. A camera selection unit that selects at least one camera image from a plurality of camera images based on the reliability calculated by the reliability calculation unit is further provided.
The position estimation device according to claim 6, wherein the feature point estimation unit estimates the feature point information based on a camera image selected by the camera selection unit. The camera selection unit
When the reliability calculated by the reliability calculation unit is equal to or higher than the threshold value, the camera image from the camera corresponding to the previously selected camera image is selected.
The position estimation device according to claim 7, wherein when the reliability calculated by the reliability calculation unit is less than the threshold value, all the plurality of camera images are selected. When the camera selection unit selects all of the plurality of camera images, the feature point estimation unit estimates the feature point information for each camera image.
The feature amount calculation unit calculates the feature amount for each camera image based on the feature point information estimated by the feature point estimation unit for each camera image.
The reliability calculation unit calculates the reliability for each camera image based on the feature amount calculated by the feature amount calculation unit for each camera image.
When the reliability calculation unit calculates the reliability for each camera image, the camera selection unit is based on the maximum reliability of the reliability for each camera image calculated by the reliability calculation unit. The position estimation device according to claim 8, wherein at least one camera image is selected from a plurality of camera images. It is a reliability estimation method that estimates the reliability of the estimated position of an object estimated based on a camera image.
A feature amount calculation step for calculating a feature amount of the object based on feature point information about a plurality of feature points of the object in the camera image, and a feature amount calculation step.
A reliability estimation method comprising a reliability calculation step for calculating the reliability based on the feature amount calculated in the feature amount calculation step.

Images