JP2019139575A - Vehicle position estimating apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately estimate the position of a vehicle even when the direction of a light source is different from that at the time of acquiring a feature amount of an environmental map. SOLUTION: For each of a plurality of feature points, a correction coefficient determination unit 370 obtains a feature amount of the feature point, an estimated direction of a light source, and a feature amount of an environmental map obtained in advance. Based on the direction of, the feature amount correction coefficient for correcting the feature amount of the feature point so that it becomes the feature amount of the feature point when it is the direction of the light source when the feature amount of the environmental map is acquired. After determining, the estimation unit 400 determines the feature amount of the feature point and corrects the feature amount of the feature point based on the feature amount of the feature point and the feature amount correction coefficient of the feature point, and a plurality of feature points of the environmental map. Based on each feature amount, the feature amount correction unit 390 estimates the position of the vehicle based on the feature amount obtained by correcting the feature amount of the feature point and the feature amount of each of the plurality of feature points on the environmental map. do. [Selection diagram] Fig. 1

Description

  The present invention relates to a vehicle position estimation device and a program.

  Conventionally, SLAM (Simultaneous Localization and Mapping) is known as a technique for estimating the position of a vehicle. SLAM has a Mapping mode for creating an environment map and a Localization mode for estimating a self-position on the environment map.

  Also, ORB-SLAM is known in which feature points such as corners of structures are detected by FAST (circles in FIG. 7) for images obtained by cameras, etc., and ORB feature values are used as feature value descriptions. (Non-Patent Document 1).

  Here, for the ORB feature quantity, for example, two 5 × 5 pixel regions are selected from 31 × 31 pixels centered on the obtained feature point (a pair of 1a and 1b in FIG. 7), and the like. Obtaining the magnitude relationship of the average luminance is performed 256 times and described as a 32-byte feature amount (FIG. 8).

Raul Mur-Artal, J. M. M. Montiel and Juan D. Tardos, "ORB-SLAM: A Versatile and Accurate Monocular SLAM System.", IEEE Transactions on Robotics, vol. 31, no. 5, 2015, pp. 1147-1163.

  By comparing the feature quantities obtained as described above, it is determined whether or not the feature points in another image are the same. However, as shown in FIG. 9, the light ray direction is different from that in FIG. When the shadowable surface changes, even if the feature point is the same, the feature amount is different (FIG. 10), so it cannot be determined that the points are the same.

  For this reason, the feature points obtained from the image taken when the environment map is first created in the mapping mode and the information on the feature amount are included in the environment map. However, if the lighting condition changes, the localization mode is set. There was a problem that it did not work well and could not recognize its own position.

  The present invention has been made to solve the above-described problems, and can estimate the position of the vehicle with high accuracy even when the direction of the light source is different from the time when the feature amount of the environmental map is acquired. An object is to provide an apparatus and a program.

  In order to achieve the above object, a vehicle position estimation apparatus according to the present invention estimates a vehicle position using an environmental map in which feature quantities at a plurality of feature points whose positions are known are stored. A feature point extraction unit that extracts a plurality of feature points from an image of the periphery of the vehicle, and a feature amount calculation that calculates a feature amount based on luminance of the feature points for each of the plurality of feature points. A light source direction estimating unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image based on sensor information, a feature amount of the feature point for each of the plurality of feature points, and the estimation The direction of the light source when the feature amount of the environmental map is acquired based on the direction of the light source and the direction of the light source obtained in advance when the feature amount of the environment map is acquired Before A correction coefficient determining unit that determines a feature quantity correction coefficient for correcting the feature quantity of the feature point so as to be a feature quantity of the feature point; and for each of the plurality of feature points, the feature quantity of the feature point; Based on the feature amount correction coefficient of the feature point, a feature amount correction unit that corrects the feature amount of the feature point, the corrected feature amount for each of the plurality of feature points, and a plurality of the environment map And an estimation unit that estimates the position of the vehicle based on the feature amount of each feature point.

  The program of the present invention is a program for estimating the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points whose positions are known are stored. Based on sensor information, a feature point extraction unit that extracts a plurality of feature points from the captured image, a feature amount calculation unit that calculates a feature amount based on luminance of the feature points for each of the plurality of feature points, A light source direction estimating unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting an image; for each of the plurality of feature points, the feature amount of the feature point, the estimated direction of the light source, and the environment map The feature amount of the feature point when the feature amount of the environmental map is obtained based on the direction of the light source obtained in advance when the feature amount of the feature map is acquired. A correction coefficient determination unit for determining a feature quantity correction coefficient for correcting the feature quantity of the feature point, and for each of the plurality of feature points, the feature quantity of the feature point and the feature quantity correction of the feature point A feature amount correcting unit that corrects the feature amount of the feature point based on a coefficient, the corrected feature amount for each of the plurality of feature points, and a feature of each of the plurality of feature points of the environment map It is a program for functioning as an estimation part which estimates the position of the said vehicle based on quantity.

  According to the state prediction device and the program of the present invention, the feature point extraction unit extracts a plurality of feature points from an image obtained by photographing the periphery of the vehicle, and the feature amount calculation unit applies to each of the plurality of feature points. A feature amount based on the luminance of the feature point is calculated, and the light source direction estimation unit estimates the direction of the light source with respect to the shooting direction at the time of shooting the image based on the sensor information.

  Then, the correction coefficient determination unit is the direction of the light source when the feature amount of the environment map is acquired based on the feature amount of the feature point and the estimated direction of the light source for each of the plurality of feature points. A feature amount correction coefficient for correcting the feature amount of the feature point is determined so as to be the feature amount of the feature point at the time, and the feature amount correction unit performs a feature of the feature point for each of the plurality of feature points The feature amount of the feature point is corrected based on the feature amount and the feature amount correction coefficient of the feature point, and the estimation unit corrects the feature amount corrected for each of the feature points and the features of the environment map. The position of the vehicle is estimated based on the feature amount of each point.

  Thus, for each of the plurality of feature points, based on the feature amount of the feature point, the estimated light source direction, and the direction of the light source obtained in advance when the feature amount of the environment map is acquired, A feature amount correction coefficient for correcting the feature amount of the feature point is determined so as to be the feature amount of the feature point in the direction of the light source when the feature amount of the environment map is acquired, and the feature point Based on the feature amount and the feature amount correction coefficient of the feature point, the feature amount obtained by correcting the feature amount of the feature point and the feature amount of each of the plurality of feature points of the environment map By estimating the position of the vehicle, it is possible to accurately estimate the position of the vehicle even when the direction of the light source is different from that at the time of acquiring the feature amount of the environmental map.

  A vehicle position estimation apparatus according to the present invention is a vehicle position estimation apparatus that estimates the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points whose positions are known are stored. Based on the sensor information, a feature point extraction unit that extracts a plurality of feature points from the image, a feature amount calculation unit that calculates a feature amount based on the luminance of the feature points for each of the plurality of feature points, A light source direction estimating unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image; a feature amount of the feature point for each of a plurality of feature points of the environment map; and the estimated direction of the light source And the feature of the feature point when it is the direction of the light source when the feature amount of the environment map is acquired based on the direction of the light source obtained in advance when the feature amount of the environment map is acquired With quantity A correction coefficient determination unit for determining a feature quantity correction coefficient for correcting a feature quantity of the feature point of the environment map, and for each of a plurality of feature points of the environment map, the feature point of the environment map And a feature amount correction unit that corrects the feature amount of the feature point based on the feature amount and a feature amount correction coefficient corresponding to the feature point, and the correction for each of the plurality of feature points of the environment map An estimation unit configured to estimate a position of the vehicle based on a feature amount and a feature amount for each of the plurality of feature points.

  The program of the present invention is a program for estimating the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points whose positions are known are stored. Based on sensor information, a feature point extraction unit that extracts a plurality of feature points from the captured image, a feature amount calculation unit that calculates a feature amount based on luminance of the feature points for each of the plurality of feature points, A light source direction estimating unit for estimating a direction of a light source with respect to a shooting direction at the time of shooting an image; for each of a plurality of feature points of the environment map; and a feature amount of the feature point; The feature point when it is the direction of the light source when the feature amount of the environment map is acquired based on the direction of the light source obtained in advance when the feature amount of the environment map is acquired A correction coefficient determining unit for determining a feature quantity correction coefficient for correcting the feature quantity of the feature point of the environment map so as to be a feature quantity; for each of a plurality of feature points of the environment map, the environment map A feature amount correcting unit that corrects a feature amount of the feature point based on a feature amount of the feature point and a feature amount correction coefficient corresponding to the feature point, and each of the plurality of feature points of the environment map A program for causing the vehicle to function as an estimation unit that estimates the position of the vehicle based on the corrected feature value and the feature value for each of the plurality of feature points.

  According to the state prediction device and the program of the present invention, the feature point extraction unit extracts a plurality of feature points from an image obtained by photographing the periphery of the vehicle, and the feature amount calculation unit applies to each of the plurality of feature points. A feature amount based on the luminance of the feature point is calculated, and the light source direction estimation unit estimates the direction of the light source with respect to the shooting direction at the time of shooting the image based on the sensor information.

  Then, for each of a plurality of feature points of the environment map, the correction coefficient determination unit obtains the feature amount of the feature point, the estimated direction of the light source, and the light source obtained in advance when the feature amount of the environment map is acquired. The feature amount for correcting the feature amount of the feature point of the environment map to be the feature amount of the feature point in the direction of the light source when the feature amount of the environment map is acquired based on the direction of the environment map A correction coefficient is determined, and the feature amount correction unit, for each of a plurality of feature points of the environment map, based on the feature amount of the feature point of the environment map and the feature amount correction coefficient corresponding to the feature point, The feature amount of the feature point is corrected, and the estimation unit determines the position of the vehicle based on the corrected feature amount for each of the plurality of feature points of the environment map and the feature amount for each of the plurality of feature points. presume.

  Thus, for each of the plurality of feature points of the environment map, the feature amount of the feature point, the estimated direction of the light source, and the direction of the light source determined in advance when the feature amount of the environment map is acquired Based on the feature map, a feature value correction coefficient for correcting the feature value of the feature point of the environment map to be the feature value of the feature point in the direction of the light source when the feature value of the environment map is acquired. A plurality of features determined by correcting the feature amount of the feature point of the environment map based on the feature amount of the feature map of the environment map and the feature amount correction coefficient corresponding to the feature point By estimating the position of the vehicle based on the feature amount for each of the points, the position of the vehicle can be accurately estimated even when the direction of the light source is different from that at the time of acquiring the feature amount of the environmental map. it can.

  The vehicle position estimation apparatus of the present invention further includes a sunshine condition estimation unit that estimates a sunshine condition of the vehicle based on brightness obtained from sensor information, and the correction coefficient determination unit includes the sunshine condition estimation. When the sunshine condition obtained by the unit is a sunshine condition that affects a feature amount due to a difference in the direction of the light source, the estimated feature point and the direction of the light source for each of the plurality of feature points The feature amount correction coefficient for correcting the feature amount of the feature point can be determined according to the direction of the light source obtained in advance when the feature amount of the environment map is acquired.

  As described above, according to the vehicle position estimation apparatus and program of the present invention, it is possible to accurately estimate the position of the vehicle even when the direction of the light source is different from that at the time of acquiring the feature amount of the environmental map.

It is a block diagram which shows schematic structure in the 1st Embodiment of this invention. It is a figure which shows an example of a feature-value correction coefficient. It is a flowchart which shows the content of the position estimation process routine in the 1st Embodiment of this invention. It is a flowchart which shows the content of the correction coefficient determination processing routine in the 1st Embodiment of this invention. It is a block diagram which shows schematic structure in the 2nd Embodiment of this invention. It is a flowchart which shows the content of the correction coefficient determination processing routine in the 2nd Embodiment of this invention. It is an image figure which shows the example of the method of calculating | requiring the feature point detection and feature amount in a prior art. It is an image figure which shows the example of the feature-value in a prior art. In a prior art, it is an image figure which shows the example when a light beam direction changes. It is an image figure which shows the example of the feature-value when a light ray direction changes in a prior art.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Configuration of vehicle position estimation apparatus according to first embodiment of the present invention>
As shown in FIG. 1, the vehicle position estimation system 10 according to the embodiment of the present invention is configured to include an imaging device 100, a sensor 200, and a vehicle position estimation device 300.

  The photographing apparatus 100 is a camera that is fixed to a vehicle and photographs a predetermined direction with respect to the vehicle.

  Specifically, the imaging device 100 is, for example, an in-vehicle monocular camera that captures a road ahead of the vehicle, and captures a road image in front of the vehicle as an example of an image of a runway on which the vehicle travels.

  Here, the photographed image is not limited to the front of the vehicle, and may be the rear or the left and right of the vehicle.

  The sensor 200 is a sensor that acquires brightness such as an illuminometer, a sensor that acquires time, and a sensor that acquires one or more three-dimensional positioning information and directions such as GPS and magnetic sensors. Composed. These sensors may be configured in other devices. In the present embodiment, the case where the sensor 200 includes an illuminometer and a GPS will be described.

  The vehicle position estimation apparatus 300 is a vehicle position estimation apparatus that estimates the position of a vehicle using an environmental map in which feature amounts at a plurality of feature points whose positions are known is stored, and includes a CPU, a RAM, and a later-described CPU. And a ROM that stores a program for executing the state estimation processing routine to be performed, and is functionally configured as follows.

  The vehicle position estimation apparatus 300 includes an image input unit 310, a feature point extraction unit 320, a feature amount calculation unit 330, a sensor information input unit 340, a light source direction estimation unit 350, a sunshine condition estimation unit 360, and a correction coefficient. The determination unit 370, the correction coefficient storage unit 380, the feature amount correction unit 390, the estimation unit 400, the environment map feature amount storage unit 410, and the driving support system 500 are configured.

  The image input unit 310 receives an image input from the imaging apparatus 100.

  Specifically, the image input unit 310 receives an input of a photographed image from the photographing apparatus 100 and passes the image to the feature point extraction unit 320.

  The feature point extraction unit 320 extracts a plurality of feature points from an image obtained by photographing the front of the vehicle.

  Specifically, the feature point extraction unit 320 extracts a plurality of feature points from the image using a feature point extraction algorithm (for example, FAST).

  Then, the feature point extraction unit 320 passes the extracted feature points to the feature amount calculation unit 330 and the correction coefficient determination unit 370.

  The feature amount calculation unit 330 calculates, for each of a plurality of feature points, a feature amount based on the luminance of the feature points.

  Specifically, the feature amount calculation unit 330 calculates a feature amount for each of the plurality of feature points obtained by the feature point extraction unit 320. For example, the feature amount calculation unit 330 calculates a 32-byte feature amount using the ORB feature amount.

  Then, the feature amount calculation unit 330 passes the calculated feature amount to the feature amount correction unit 390.

  The sensor information input unit 340 receives input of the brightness obtained by the illuminometer, the three-dimensional positioning information obtained by the GPS, and the time from the sensor 200.

  Then, the sensor information input unit 340 passes the three-dimensional positioning information and the time thereof to the light source direction estimation unit 350 and the brightness to the sunshine condition estimation unit 360.

  The light source direction estimation unit 350 estimates the direction of the light source with respect to the shooting direction at the time of shooting an image based on the sensor information.

  Specifically, the light source direction estimation unit 350 estimates the direction of the light source (for example, the direction of the sun with reference to the north) from the GPS positioning time. This embodiment demonstrates the case where a light source is the sun. For example, the light source direction estimation unit 350 estimates the direction and altitude of the sun for the positioning time from the average sunrise, sunset, south-central time, etc. on the date of that time.

  Further, the light source direction estimation unit 350 estimates the direction of the shooting direction from sensor information for calculating the direction of the vehicle. For example, using the latitude and longitude by GPS and the latitude and longitude by GPS acquired at the previous time, the direction of the vehicle with respect to the north is estimated, and the direction of the vehicle and the shooting direction are the same. The direction of the vehicle is estimated as the shooting direction. In addition, when the direction of the vehicle and the photographing direction are deviated, the photographing direction may be estimated in consideration of the deviation.

  The light source direction estimation unit 350 estimates the direction of the light source with respect to the shooting direction at the time of shooting an image based on the estimated direction of the light source and the direction of the shooting direction. Then, the light source direction estimation unit 350 passes the light source direction with respect to the estimated shooting direction to the correction coefficient determination unit 370.

  The sunshine condition estimating unit 360 estimates the sunshine condition of the vehicle based on the brightness obtained from the sensor information.

  Specifically, the sunshine condition estimation unit 360 estimates the cloud amount from the illuminance information obtained by the illuminometer, and estimates whether or not a shadow is formed. For example, if the brightness measured by the illuminometer is less than or equal to a predetermined value, it is estimated to be cloudy and a shadow is not estimated.

  Then, the sunshine condition estimation unit 360 passes to the correction coefficient determination unit 370 whether or not a shadow is formed.

  For each of the plurality of feature points, the correction coefficient determination unit 370 obtains the feature amount of the feature point, the direction of the light source with respect to the estimated shooting direction, and the shooting direction obtained in advance when the feature amount of the environment map is acquired. Based on the direction of the light source with respect to, the feature amount correction for correcting the feature amount of the feature point to be the feature amount of the feature point when the feature amount of the environment map is obtained Determine the coefficient.

  Specifically, the correction coefficient determination unit 370 first acquires the sunshine conditions at the time of creating the environment map from the environment map feature quantity storage unit 410, and the sunshine conditions estimated by the sunshine condition estimation unit 360 are the environment map creation. It is determined whether or not it influences the case of the sunshine condition of the hour.

  The presence or absence of the influence of the sunshine condition is estimated by the difference between the presence or absence of the shadow estimated by the sunshine condition estimation unit 360, the direction of the sun estimated by the light source direction estimation unit 350 and the direction of the sun at the time of acquiring the feature quantity of the environmental map. Judgment is made based on whether or not the direction of the sun is forward (that is, whether or not the camera is backlit with respect to the camera).

  For example, it can be estimated that the amount of clouds is large when there is no shadow in the sunshine conditions when creating the environmental map. For this reason, the correction coefficient determination unit 370 determines that there is no influence on the case of the sunshine condition at the time of creating the environmental map when the estimated sunshine condition cannot be shaded.

  In addition, when the correction coefficient determination unit 370 determines that the sunshine condition at the time of creating the environmental map has an influence, the correction coefficient determination unit 370 has the direction of the sun with respect to the shooting direction at the time of creating the environmental map and the shooting direction at the time of shooting the image. It is determined whether or not the directions of the sun match.

  The correction coefficient determination unit 370 determines that there is no effect when the sun direction with respect to the shooting direction estimated by the light source direction estimation unit 350 matches the sun direction with respect to the shooting direction at the time of acquiring the feature amount of the environmental map. .

  Further, even when the sun direction with respect to the shooting direction estimated by the light direction estimation unit 350 and the sun direction with respect to the shooting direction at the time of acquiring the feature amount of the environmental map do not match, That is, if light is irradiated from the front, the light is backlit, and the luminance is uniformly reduced. Therefore, it is determined that there is no influence of the sunshine condition.

  On the other hand, if the irradiation is from the rear, the correction coefficient determination unit 370 determines whether the sun direction with respect to the imaging direction estimated by the light direction estimation unit 350 is on the right side or the left side with respect to the sun direction at the time of creating the environmental map. .

  Here, the correction coefficient determination unit 370 estimates whether the feature point is convex or concave. For example, there is a method of obtaining the average luminance on the left and right with respect to the feature point. Specifically, within the range centered on the feature point (for example, ± 15 pixels), the average luminance is obtained separately for the left half and the right half, and the sunlight hits the feature point on the left side, and When the left average luminance is high, it is estimated as convex, and when the right average luminance is high, it is estimated as concave. Further, when the sunlight hits from the right side, it is estimated that the projection is convex when the right average luminance is high, and concave when the left average luminance is high.

  In addition, as a method for estimating the unevenness of the feature points, a certain range (eg, ± 16 pixels) centered on the feature points is divided into grids or radial regions centered on the feature points, and the average luminance of each region A means for calculating and comparing the values may be used.

  As described above, the correction coefficient determination unit 370 calculates unevenness for all feature points.

  Then, the correction coefficient determination unit 370 determines whether the sun direction relative to the shooting direction estimated by the light source direction estimation unit is on the right side or the left side of the sun direction at the time of creating the environmental map, and the uneven shape of the feature point. Determine the quantity correction factor.

  First, the correction coefficient determination unit 370 does not need to correct the feature amount when it is determined that the estimated sunshine condition has no influence on the sunshine condition at the time of creating the environment map. A feature amount correction coefficient A (FIG. 2) whose coefficients are all 0 is determined as a feature amount correction coefficient.

  Further, the correction coefficient determination unit 370 is characterized when the direction of the sun relative to the photographing direction estimated by the light direction estimation unit 350 is on the right side of the sun direction at the time of creating the environmental map and the shape of the feature point is convex. The amount correction coefficient B (FIG. 2) is determined as the feature amount correction coefficient. Further, when the sun direction with respect to the photographing direction estimated by the light direction estimation unit 350 is on the right side of the sun direction at the time of creating the environmental map and the shape of the feature point is concave, the feature amount correction coefficient B ′ (FIG. 2) is determined as a feature amount correction coefficient.

  The correction coefficient determination unit 370 is characterized in that the sun direction with respect to the photographing direction estimated by the light direction estimation unit 350 is on the left side of the sun direction at the time of creating the environmental map and the feature point has a convex shape. The amount correction coefficient C (FIG. 2) is determined as the feature amount correction coefficient. Further, when the sun direction with respect to the photographing direction estimated by the light direction estimation unit 350 is on the left side of the sun direction at the time of creating the environmental map and the shape of the feature point is concave, the feature amount correction coefficient C ′ (FIG. 2) is determined as a feature amount correction coefficient.

  Then, when the correction coefficient determination unit 370 determines the feature amount correction coefficients for all feature points, the correction factor determination unit 370 passes the determined feature amount correction coefficients to the feature amount correction unit 390.

  The correction coefficient storage unit 380 stores the feature amount correction coefficients of various patterns, which are determined in advance, with respect to the feature amounts of the feature points for each of the feature points of the environment map.

  The feature amount correction coefficient is determined according to the presence or absence of the influence of sunlight conditions, the difference from the sun direction with respect to the feature point at the time of acquiring the feature amount of the environmental map, the unevenness of the feature point, the sun direction with respect to the shooting direction. It is done.

  An example of the feature amount correction coefficient is shown in FIG. In FIG. 2, the feature amount correction coefficient is a coefficient having a length corresponding to a feature amount having any value of {+1, 0, −1} (256 columns of coefficients in the example of FIG. 2). Since each bit string of the feature amount is represented by {1, 0}, the feature amount correction coefficient {+1, 0, −1} changes each bit string of the corresponding feature amount to “1”, “change” "None" and "Change to 0".

  As a method of determining the feature amount correction coefficient, each bit of the feature amount is assumed assuming various effects depending on the presence / absence of sunshine conditions at the time of environmental map creation, unevenness of feature points, and the direction of the sun relative to the shooting direction. The feature amount correction coefficients of various patterns corresponding to the above are determined.

  When there is no influence of the sunshine condition, it is not necessary to correct the feature amount, and therefore a feature amount correction coefficient in which each bit string is all 0 is determined (feature amount correction coefficient A in FIG. 2).

  In addition, when there is an influence of sunlight conditions, that is, when shadows are formed, the direction of the sun with respect to the shooting direction at the time of image shooting is the sun at the time of environment map creation for each bit of the feature value of the feature point of the environment map. The feature amount correction coefficient (feature amount correction coefficient B in FIG. 2) is determined on the right side of the direction and the shape of the feature point is assumed to be convex.

  In this case, the two points (for example, two points 1a and 1b in FIG. 9) for performing the luminance calculation corresponding to the bit compared with each bit of the feature amount of the feature point of the environment map are the environment. A feature amount correction coefficient is determined for each bit of the feature amount of the feature point of the environment map, depending on whether the feature point is on the same left or right side with respect to the feature point of the map.

  When the two points are on the same side with respect to the feature point, even if a shadow is formed on the feature point, it is considered that the corresponding feature amount bit is not affected. Set the coefficient bit to 0.

  For example, the determination can be made by using two points (xa, xb) of 5 × 5 pixels in which the luminance calculation compared in obtaining the x-th bit of the feature amount is performed. When both of the two points in the region are on the left side or the right side of the feature point, the x-th bit of the feature amount correction coefficient at the feature point is set to 0.

  In addition, when the region xa where the luminance calculation is performed is on the right side and xb is on the left side (for example, 1a and 1b in FIG. 9), the sun direction relative to the shooting direction at the time of image shooting is more than the sun direction at the time of environmental map creation. If the shape of the feature point is convex on the right side, a shadow is formed on the left side of the feature point, so the left side should be dark (low brightness). That is, since xa> xb, the x-th bit of the feature quantity affected by the sunshine condition is 1.

  At this time, if the x-th bit of the feature quantity of the feature point of the environment map is 1, there is no need to modify, so the x-th bit of the feature quantity correction coefficient is set to 0. On the other hand, if the xth bit of the feature value of the feature point of the environment map is 0, the xth bit of the extracted feature value should be corrected to 0. Let bit be -1.

  Conversely, if xa is on the left side and xb is on the right side, if the direction of the sun relative to the shooting direction at the time of image shooting is to the right of the sun direction at the time of creating the environmental map and the shape of the feature points is convex, xa Since xb, the x-th bit of the feature quantity affected by the sunshine condition is 0.

  At this time, if the x-th bit of the feature value of the feature point of the environment map is 0, there is no need for correction, so the x-th bit of the feature value correction coefficient is set to 0. On the other hand, if the xth bit of the feature value of the feature point of the environment map is 1, the xth bit of the extracted feature value should be corrected to 1, so the xth bit of the feature value correction coefficient Set the bit to +1.

  The feature amount correction coefficient B is determined by performing this process for all bits.

  Similarly, a feature amount correction coefficient B ′ is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the right of the sun direction at the time of environmental map creation and the shape of the feature point is concave. A feature amount correction coefficient C is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the left of the sun direction at the time of creating the environmental map and the shape of the feature points is convex. A feature amount correction coefficient C ′ is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the left of the sun direction at the time of creating the environmental map and the shape of the feature point is concave.

  In this way, by setting the feature amount correction coefficients of various patterns, the two points to be compared are the direction of the sunlight and the shape of the feature points (unevenness) even when the sunshine conditions are affected by shadows, for example. Accordingly, it is possible to correct the feature amount of the extracted feature point in consideration of the influence.

  The feature amount correcting unit 390 corrects the feature amount of each feature point based on the feature amount of the feature point and the feature amount correction coefficient for the pattern corresponding to the feature point.

  Specifically, the feature amount correcting unit 390 corrects the feature amount by adding each value of each bit sequence of the feature amount and each bit sequence of the feature amount correction coefficient for the pattern corresponding to the feature point. At this time, the feature amount correction unit 390 sets the bit to 1 when there is a bit exceeding +1 among the feature amounts obtained by addition, and sets the bit 0 when there is a bit less than 0. To correct.

  For example, in the case of FIG. 9, the feature amount is “1, 0, 0,..., 1” as shown in FIG. 10, but the feature amount correction coefficient is “−1, 1, 1,. Then, the corrected value is “0, 1, 1,..., 1”, which matches the feature amount (FIG. 8) in the case of FIG.

  Then, the feature amount correction unit 390 passes the feature amounts corrected for the feature amounts of all feature points to the estimation unit 400.

  The estimation unit 400 estimates the position of the vehicle based on the corrected feature quantity for each of the plurality of feature points and the feature quantities of each of the plurality of feature points on the environment map.

  Specifically, the estimation unit 400 matches the corrected feature quantity with the feature quantity of the environment map, and estimates where the vehicle is on the environment map. For example, the position corresponding to the feature quantity having the highest similarity among the corrected feature quantity and the feature quantities of the plurality of feature points on the environment map is estimated as the position of the vehicle on the environment map.

  Then, the estimation unit 400 outputs the estimated position of the vehicle to the driving support system 500.

  The environment map feature quantity storage unit 410 stores an environment map created in advance, feature quantities of the environment map, and sunshine conditions at the time of creating the environment map. The environment map is created, for example, in the SLAM Mapping mode.

  The driving support system 500 performs driving support based on the estimated position of the vehicle.

<Operation of Vehicle Position Estimation Device According to First Embodiment of the Present Invention>
Next, a position estimation processing routine of the vehicle position estimation apparatus 300 according to the present embodiment will be described with reference to FIG.

  First, in step S <b> 100, the correction coefficient determination unit 370 and the estimation unit 400 acquire the feature amount of the environment map from the environment map feature amount storage unit 410. Further, the correction coefficient determination unit 370 acquires the sunshine conditions at the time of creating the environment map from the environment map feature amount storage unit 410.

  In step S110, the sensor information input unit 340 receives input of the brightness obtained by the illuminometer, the three-dimensional positioning information obtained by the GPS, and the time from the sensor 200.

  In step S <b> 120, the image input unit 310 receives an image input from the imaging apparatus 100.

  In step S130, the feature point extraction unit 320 extracts a plurality of feature points from an image obtained by photographing the front of the vehicle.

  In step S140, the feature amount calculation unit 330 calculates, for each of the plurality of feature points, a feature amount based on the luminance of the feature point.

  In step S150, the correction coefficient determination unit 370 obtains in advance, for each of the plurality of feature points, the feature amount of the feature point, the direction of the light source with respect to the estimated shooting direction, and the feature amount of the environment map. To correct the feature amount of the feature point to be the feature amount of the feature point when the feature amount of the environment map is obtained based on the direction of the light source with respect to the taken shooting direction The feature amount correction coefficient is determined.

  In step S160, the feature amount correcting unit 390, for each of the plurality of feature points, based on the feature amount of the feature point and the feature amount correction coefficient for the pattern corresponding to the feature point. To correct.

  In step S170, the estimation unit 400 estimates the position of the vehicle based on the corrected feature quantity for each of the plurality of feature points and the feature quantities of each of the plurality of feature points on the environment map.

  In step S180, the estimation unit 400 outputs the estimated position of the vehicle to the driving support system 500, returns to step S110, and repeats the processes of steps S110 to S180.

  Further, the above step S150 is realized by a correction coefficient determination processing routine shown in FIG.

  In step S200, the sunshine condition estimation unit 360 estimates the sunshine condition of the vehicle based on the brightness obtained from the sensor information.

  In step S210, the light source direction estimation unit 350 estimates the position of the light source and the direction of the vehicle at the time of capturing an image based on the sensor information.

  In step S220, the light source direction estimation unit 350 estimates the direction of the light source with respect to the shooting direction at the time of shooting an image, based on the direction of the light source estimated in step S210 and the direction of the shooting direction.

  In step S230, the correction coefficient determination unit 370 selects the first feature point.

  In step S240, the correction coefficient determination unit 370 determines whether or not the sunshine condition estimated in step S200 has an influence on the sunshine condition at the time of creating the environmental map.

  When there is no influence by the sunshine condition (NO in step S240), in step S250, the correction coefficient determination unit 370 determines the feature quantity correction coefficient A as the feature quantity correction coefficient, and the process proceeds to step S330.

  On the other hand, when there is an influence due to sunshine conditions (YES in step S240), in step S260, the correction coefficient determination unit 370 determines that the sun direction relative to the shooting direction estimated in step S220 is greater than the sun direction at the time of creating the environmental map. Determine whether it is right or left.

  When the sun direction relative to the shooting direction is on the right side of the sun direction at the time of creating the environmental map (NO in step S260), in step S270, the correction coefficient determination unit 370 determines whether or not the shape of the feature point is concave. To do.

  When the shape of the feature point is convex (NO in step S270), in step S280, the correction coefficient determination unit 370 determines the feature quantity correction coefficient B as the feature quantity correction coefficient, and the process proceeds to step S330.

  On the other hand, when the shape of the feature point is concave (YES in step S270), in step S290, the correction coefficient determination unit 370 determines the feature quantity correction coefficient B 'as the feature quantity correction coefficient, and the process proceeds to step S330.

  If the sun direction relative to the shooting direction is to the left of the sun direction at the time of creating the environmental map (YES in step S260), in step S300, the correction coefficient determination unit 370 determines whether the feature point shape is concave. Determine.

  When the shape of the feature point is convex (NO in step S300), in step S310, the correction coefficient determination unit 370 determines the feature quantity correction coefficient C as the feature quantity correction coefficient, and the process proceeds to step S330.

  On the other hand, when the shape of the feature point is concave (YES in step S300), in step S320, the correction coefficient determination unit 370 determines the feature quantity correction coefficient C ′ as the feature quantity correction coefficient, and the process proceeds to step S330.

  In step S330, the correction coefficient determination unit 370 determines whether or not the feature amount correction coefficient has been determined for all feature points.

  If there is a feature point for which the feature amount correction coefficient is not yet determined (NO in step S330), the next feature point is selected, and the process returns to step S240.

  On the other hand, if the feature amount correction coefficient has been determined for all feature points (YES in step S330), the process returns.

  As described above, according to the vehicle position estimation apparatus according to the first embodiment of the present invention, for each of a plurality of feature points, the feature amount of the feature point, the estimated direction of the light source, and the environment Based on the direction of the light source obtained in advance when the feature amount of the map is acquired, the feature to be the feature amount of the feature point when it is the direction of the light source when the feature amount of the environmental map is acquired A feature amount correction coefficient for correcting the feature amount of the point is determined, and a feature amount obtained by correcting the feature amount of the feature point based on the feature amount of the feature point and the feature amount correction coefficient of the feature point; By estimating the position of the vehicle on the basis of the feature amounts of the plurality of feature points of the environment map, even if the direction of the light source is different from that at the time of acquiring the feature amount of the environment map, the vehicle The position can be estimated.

<Outline of vehicle position estimation apparatus according to second embodiment of the present invention>
In the first embodiment, the feature amount of the extracted image is corrected. In the second embodiment, the feature amount of the feature point of the environment map is corrected.

  In the present embodiment, the feature amount correction coefficient is determined in the same manner as in the first embodiment when there is no influence of the sunshine condition, but is determined as follows in other cases.

  In other words, when there is an influence of the sunshine condition, for each bit of the feature amount of the feature point of the environment map, the feature amount assuming the right or left side of the sun direction at the time of creating the environment map and the feature point uneven shape A correction factor is determined.

  For example, assume that the direction of the sun with respect to the shooting direction at the time of image shooting is on the right side of the direction of the sun at the time of creating the environment map and the shape of the feature points is convex for each bit of the feature amount of the feature point of the environment map The determined feature amount correction coefficient (feature amount correction coefficient B in FIG. 2) is determined in advance.

  If any of the 2 points (xa, xb) of the 5 × 5 pixel area for which the luminance calculation is compared when obtaining the x-th bit of the feature amount is on the left or right side of the feature point, the feature at the feature point The xth bit of the quantity correction coefficient is set to 0.

  In addition, when the region xa for luminance calculation is on the right side and xb is on the left side, the sun direction with respect to the shooting direction at the time of image shooting is on the right side of the sun direction at the time of environmental map creation, and the shape of the feature point is If it is convex, a shadow is formed on the left side of the feature point, so the left side should be dark (low brightness). That is, since xa> xb, the x-th bit of the feature quantity affected by the sunshine condition is 1.

  At this time, if the x-th bit of the feature quantity of the feature point of the environment map is 1, there is no need to modify, so the x-th bit of the feature quantity correction coefficient is set to 0. On the other hand, if the xth bit of the feature value of the feature point of the environment map is 0, the xth bit of the feature value of the feature point of the environment map should be corrected to 1. Let the xth bit of +1 be +1.

  Conversely, if xa is on the left side and xb is on the right side, if the direction of the sun relative to the shooting direction at the time of image shooting is to the right of the sun direction at the time of creating the environmental map and the shape of the feature points is convex, xa Since xb, the x-th bit of the feature quantity affected by the sunshine condition is 0.

  At this time, if the x-th bit of the feature value of the feature point of the environment map is 0, there is no need for correction, so the x-th bit of the feature value correction coefficient is set to 0. On the other hand, if the x-th bit of the feature value of the feature point of the environment map is 1, the x-th bit of the feature value of the feature point of the environment map should be corrected to 0. Let the x th bit of −1 be −1.

  By performing this process for all bits, the feature amount correction coefficient is determined.

  Similarly, a feature amount correction coefficient B ′ is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the right of the sun direction at the time of environmental map creation and the shape of the feature point is concave. A feature amount correction coefficient C is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the left of the sun direction at the time of creating the environmental map and the shape of the feature points is convex. A feature amount correction coefficient C ′ is determined for a case where the sun direction with respect to the shooting direction at the time of image shooting is to the left of the sun direction at the time of creating the environmental map and the shape of the feature point is concave.

  In this way, by setting the feature amount correction coefficients of various patterns, the two points to be compared are the direction of the sunlight and the shape of the feature points (unevenness) even when the sunshine conditions are affected by shadows, for example. Accordingly, the feature amount of the feature point of the environmental map can be corrected in consideration of the influence.

<Configuration of vehicle position estimation apparatus according to second embodiment of the present invention>
A configuration of the vehicle position estimation apparatus 20 according to the second embodiment of the present invention will be described. In addition, about the structure similar to the vehicle position estimation apparatus 10 which concerns on 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  FIG. 5 is a block diagram illustrating a configuration of the vehicle position estimation apparatus 300 according to the present embodiment.

  The feature amount calculation unit 330 calculates, for each of a plurality of feature points, a feature amount based on the luminance of the feature points.

  Specifically, the feature amount calculation unit 330 calculates a feature amount for each of the plurality of feature points obtained by the feature point extraction unit 320. For example, the feature amount calculation unit 330 calculates a 32-byte feature amount using the ORB feature amount.

  Then, the feature amount calculation unit 330 passes the calculated feature amount to the estimation unit 400.

  For each of the plurality of feature points of the environment map, the feature amount correction unit 390 calculates the feature points based on the feature amount of the feature point of the environment map and the corresponding feature amount correction coefficient corresponding to the feature point. Correct the feature value.

  Then, the feature amount correction unit 390 passes the corrected feature amount of the environment map to the estimation unit 400.

  The estimation unit 400 estimates the position of the vehicle based on the corrected feature amount for each of the plurality of feature points of the environment map and the feature amount for each of the plurality of feature points.

  Specifically, the estimation unit 400 matches the corrected feature quantity for each of the plurality of feature points of the environment map with the feature quantity for each of the extracted feature points, and determines which vehicle is on the environment map. Estimate if it is in position. For example, the position corresponding to the feature quantity having the highest similarity among the feature quantities of the modified environment map and the feature quantities of the plurality of feature points is estimated as the position of the vehicle on the environment map.

  Then, the estimation unit 400 outputs the estimated position of the vehicle to the driving support system 500.

<Operation of the vehicle position estimation apparatus according to the second embodiment of the present invention>
FIG. 6 is a flowchart showing a vehicle position estimation processing routine according to the second embodiment of the present invention. In addition, about the process similar to the vehicle position estimation process routine which concerns on 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  In step S460, the feature amount correcting unit 390, for each of the plurality of feature points of the environment map, based on the feature amount of the feature point of the environment map and the corresponding feature amount correction coefficient corresponding to the feature point. The feature amount of the feature point is corrected.

  As described above, according to the vehicle position estimation device according to the second embodiment of the present invention, for each of the plurality of feature points of the environment map, the feature amount of the feature point and the estimated light source Based on the direction and the direction of the light source obtained in advance when the feature amount of the environment map is acquired, the feature amount of the feature point when the feature amount of the environment map is obtained is obtained. Determining a feature amount correction coefficient for correcting the feature amount of the feature point, and determining the feature amount of the feature point based on the feature amount of the feature point and the feature amount correction coefficient corresponding to the feature point. By estimating the position of the vehicle based on the corrected feature quantity and the feature quantity for each of the plurality of feature points, even if the direction of the light source is different from that at the time of acquiring the feature quantity of the environment map, the accuracy is improved. The position of the vehicle can be estimated well.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  In the above-described embodiment, the sunshine condition estimation unit 360 estimates whether or not a shadow is formed by using an illuminometer, but may estimate based on an input captured image. In this case, a method of estimating that there is no shadow when there is no difference equal to or greater than a predetermined threshold between the average brightness in the shadow area and the average brightness in other areas in a certain area may be considered.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Vehicle position estimation apparatus 100 Image pick-up apparatus 200 Sensor 300 Vehicle position estimation apparatus 310 Image input part 320 Feature point extraction part 330 Feature quantity calculation part 340 Sensor information input part 350 Light source direction estimation part 360 Sunlight condition estimation part 370 Correction coefficient determination part 380 Correction coefficient storage unit 390 Feature amount correction unit 400 Estimation unit 410 Environmental map feature amount storage unit 500 Driving support system

Claims (5)

A vehicle position estimation device that estimates the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points each having a known position are stored,
A feature point extraction unit that extracts a plurality of feature points from an image of the periphery of the vehicle;
For each of the plurality of feature points, a feature amount calculation unit that calculates a feature amount based on the luminance of the feature points;
Based on sensor information, a light source direction estimation unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image;
For each of the plurality of feature points, based on the feature amount of the feature point, the estimated direction of the light source, and the direction of the light source obtained in advance when the feature amount of the environmental map is acquired. A correction coefficient for determining a feature quantity correction coefficient for correcting the feature quantity of the feature point so as to be the feature quantity of the feature point in the direction of the light source when the feature quantity of the environment map is acquired A decision unit;
For each of the plurality of feature points, a feature amount correction unit that corrects the feature amount of the feature point based on the feature amount of the feature point and the feature amount correction coefficient of the feature point;
An estimation unit that estimates the position of the vehicle based on the corrected feature amount for each of the plurality of feature points and the feature amount of each of the plurality of feature points of the environment map;
A vehicle position estimation device including: A vehicle position estimation device that estimates the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points each having a known position are stored,
A feature point extraction unit that extracts a plurality of feature points from an image of the periphery of the vehicle;
For each of the plurality of feature points, a feature amount calculation unit that calculates a feature amount based on the luminance of the feature points;
Based on sensor information, a light source direction estimation unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image;
For each of the plurality of feature points of the environment map, the feature amount of the feature point, the estimated direction of the light source, and the direction of the light source obtained in advance when the feature amount of the environment map is acquired Based on the feature amount, the feature amount for correcting the feature amount of the feature point of the environment map to be the feature amount of the feature point when the feature amount of the environment map is the direction of the light source A correction coefficient determination unit for determining a correction coefficient;
For each of a plurality of feature points of the environment map, a feature amount for correcting the feature amount of the feature point based on a feature amount of the feature point of the environment map and a feature amount correction coefficient corresponding to the feature point Correction part,
An estimation unit that estimates the position of the vehicle based on the corrected feature amount for each of the plurality of feature points of the environmental map and the feature amount for each of the plurality of feature points;
A vehicle position estimation device including: A sunshine condition estimating unit for estimating the sunshine condition of the vehicle based on the brightness obtained from the sensor information;
Further including
When the sunshine condition obtained by the sunshine condition estimator is a sunshine condition that affects a feature amount due to a difference in the direction of the light source, the correction coefficient determination unit A feature amount correction coefficient for correcting the feature amount of the feature point according to the estimated feature point, the direction of the light source, and the direction of the light source obtained in advance when the feature amount of the environment map is acquired The vehicle position estimation apparatus according to claim 1 or 2. A program for estimating the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points each having a known position are stored,
Computer
A feature point extraction unit that extracts a plurality of feature points from an image of the periphery of the vehicle;
For each of the plurality of feature points, a feature amount calculation unit that calculates a feature amount based on the luminance of the feature points;
A light source direction estimating unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image based on sensor information;
For each of the plurality of feature points, based on the feature amount of the feature point, the estimated direction of the light source, and the direction of the light source obtained in advance when the feature amount of the environmental map is acquired. A correction coefficient for determining a feature quantity correction coefficient for correcting the feature quantity of the feature point so as to be the feature quantity of the feature point in the direction of the light source when the feature quantity of the environment map is acquired Decision part,
For each of the plurality of feature points, a feature amount correcting unit that corrects the feature amount of the feature point based on a feature amount of the feature point and a feature amount correction coefficient of the feature point; and
The program for functioning as an estimation part which estimates the position of the said vehicle based on the said corrected feature-value about each of these feature points, and each feature-value of several feature points of the said environmental map. A program for estimating the position of a vehicle using an environmental map in which feature quantities at a plurality of feature points each having a known position are stored,
Computer
A feature point extraction unit that extracts a plurality of feature points from an image of the periphery of the vehicle;
For each of the plurality of feature points, a feature amount calculation unit that calculates a feature amount based on the luminance of the feature points;
A light source direction estimating unit that estimates a direction of a light source with respect to a shooting direction at the time of shooting the image based on sensor information;
For each of the plurality of feature points of the environment map, the feature amount of the feature point, the estimated direction of the light source, and the direction of the light source obtained in advance when the feature amount of the environment map is acquired Based on the feature amount, the feature amount for correcting the feature amount of the feature point of the environment map to be the feature amount of the feature point when the feature amount of the environment map is the direction of the light source A correction coefficient determination unit for determining a correction coefficient;
For each of a plurality of feature points of the environment map, a feature amount for correcting the feature amount of the feature point based on a feature amount of the feature point of the environment map and a feature amount correction coefficient corresponding to the feature point Correction part, and
A program for causing the vehicle to function as an estimation unit that estimates the position of the vehicle based on the corrected feature amount for each of the plurality of feature points of the environmental map and the feature amount for each of the plurality of feature points. .