JP2011113364A - Apparatus, program and system for estimating surrounding environment - Google Patents

Abstract

[PROBLEMS] To accurately estimate the surrounding environment.
A surrounding environment estimation device 1 includes a camera 1a and a surrounding environment estimation unit 10, and is installed on a moving body to estimate a roadside shape, a situation, and the like as a surrounding environment. The surrounding environment estimation unit 10 estimates the surrounding environment by measuring the distance to the surrounding object. In this case, the position of the camera 1a and the shooting direction are recognized, and a reference path composed of reference path points that are one or more known positions is set. And based on the image image | photographed with the camera 1a, the distance from the reference | standard path | route to the peripheral object image | photographed in the image is calculated | required.
[Selection] Figure 1

Description

  The present invention relates to a surrounding environment estimation device that estimates a surrounding environment, a surrounding environment estimation program that causes a computer to execute control for estimating the surrounding environment, and a surrounding environment estimation system.

  In order to perform maintenance or the like to keep the map data up-to-date, changes in the roadside shape, which is the surrounding environment of the vehicle, may be examined. In this case, instead of actually walking and investigating the site, an image of the roadside shape of the site is taken using an in-vehicle image and later examined.

  As a conventional technique, Patent Document 1 proposes a technique for estimating a roadside shape using an in-vehicle image. Further, Patent Document 2 proposes a technique for estimating a roadside shape by using only the center portion of an in-vehicle image as a line sensor.

JP 2005-56295 A JP 2007-148809 A

  Estimating the surrounding environment such as buildings and road shapes excluding temporarily moving objects such as parallel vehicles on the road, parked vehicles or pedestrians on the sidewalk as roadside shapes It is also useful for creating.

  For example, in this case, a peripheral object is photographed with an in-vehicle camera, a distance between the photographed peripheral object and the in-vehicle camera position is measured, a plurality of these measurement results are collected and compared, and a difference is investigated. If a region having a large difference is regarded as a region where there is a temporary moving object (obstacle), a stationary environment in which the moving object is removed in the surrounding environment can be estimated by such a method.

  However, conventionally, there has been a problem that it is difficult to accurately estimate the surrounding environment excluding moving objects using the measurement result in this way. The measurement point, that is, the camera position and the camera line-of-sight direction depend on the traveling path of the vehicle on which the camera is mounted, the vehicle type, and the like. The above problems often occur when the measurement point or camera line-of-sight direction when acquiring one measurement data and the measurement point or camera line-of-sight direction when acquiring the other measurement data are different. This is because the measured data cannot be simply compared.

The present invention has been made in view of these points, and an object of the present invention is to provide a surrounding environment estimation device that accurately estimates the surrounding environment in an environment where a moving object exists.
Another object of the present invention is to provide a surrounding environment estimation program and a surrounding environment estimation system for accurately estimating the surrounding environment in an environment where a moving object exists.

  In order to solve the above-mentioned problem, a surrounding environment estimation device for estimating a surrounding environment is provided. The surrounding environment estimation device includes a camera and a surrounding environment estimation unit that estimates the surrounding environment by measuring a distance between the surrounding object and the camera.

  Here, the surrounding environment estimation unit recognizes the position and shooting direction of the camera, sets a reference path including reference path points that are one or more known positions serving as a reference, and is an image captured by the camera. Based on this, the distance from the reference path to the peripheral object photographed in the image is obtained.

  The surrounding environment can be estimated with high accuracy.

It is a figure which shows the structural example of a surrounding environment estimation apparatus. It is a figure which shows the component of a surrounding environment estimation part. It is a figure which shows an example which is measuring the peripheral object along a road from the mobile body which drive | works a road. It is a flowchart which shows an example of operation | movement of a data converter. It is a figure which shows the calculation flowchart of an optimal mapping position. It is a figure which shows the creation example of an image map. It is a figure which shows an image map. It is a flowchart which shows an example of operation | movement of a distance image conversion part. It is a flowchart which shows the operation example of a data comparison part. It is a flowchart which shows the operation example of a data comparison part.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of a surrounding environment estimation apparatus.
The surrounding environment estimation device 1 includes a camera 1a and a surrounding environment estimation unit 10, and is a device that estimates, for example, roadside shape and spatial information such as a situation as the surrounding environment. The camera 1a that measures the measurement data used for estimation may be a camera that is fixedly installed around the road. Further, the surrounding environment estimation device 1 has no particular problem even if it includes a device that is fixedly installed around the road. By including a fixed camera and a fixed device, the amount of measurement data to be used can be increased. In the following description, it is assumed that the camera 1a or the entire apparatus is installed and used in a moving body such as an automobile.

  The surrounding environment estimation unit 10 estimates the distance and direction from the surrounding object as the estimation of the surrounding environment using the position of the camera 1a and the shooting direction. An arbitrary coordinate system position and direction may be used as the position and shooting direction of the camera 1a. However, as an example, explanation will be made using latitude and longitude and azimuth. The surrounding environment estimation unit 10 further sets a reference route that connects reference route points that are one or more arbitrary known positions (latitude and longitude) serving as a reference.

  And the distance from the camera 1a to the surrounding object image | photographed in the image obtained based on the image image | photographed with the camera 1a is converted into the distance from a reference | standard path | route. In this way, by converting the distance from the captured camera to the distance from the set reference path, it is possible to absorb differences in the shooting direction and shooting position, and estimate the surrounding environment with high accuracy from the measurement results. can do.

  Next, the configuration of the surrounding environment estimation unit 10 will be described. FIG. 2 is a diagram illustrating components of the surrounding environment estimation unit 10. The surrounding environment estimation unit 10 includes a surrounding measurement unit 11, a position acquisition unit 12, a reference route setting unit 13, a data conversion unit 14, a distance image conversion unit 15, a data comparison unit 16, and an output unit 17.

  The peripheral measurement unit 11 measures the distance between the peripheral environment (peripheral object) photographed by the camera 1a and the moving body on which the camera 1a is mounted, and the line-of-sight direction of the camera 1a with respect to the peripheral object. The position acquisition unit 12 acquires the current position of the moving body (= the current position of the camera 1a).

  The reference route setting unit 13 sets a reference route / direction (hereinafter simply referred to as a reference route) that defines a camera movement position and a camera line-of-sight direction as a reference on the road when the moving body passes. The data conversion unit 14 converts the measurement data of the peripheral object measured on the traveling path by the moving body so as to become the measurement data from the reference path.

  The distance image conversion unit 15 converts the measurement data converted by the data conversion unit 14 or existing measurement data into an image map as a two-dimensional map. The existing measurement data is, for example, distance data that has already been measured by a camera or the like fixed on another vehicle or road.

  When the image map is not used, the data comparison unit 16 compares the reference measurement data converted by the data conversion unit 14 with the existing measurement data, or the data converted by the data conversion unit 14. Further, the surrounding environment is estimated by performing a comparison process between the image maps converted by the distance image conversion unit 15. When using an image map, the image map generated from the reference measurement data converted by the data converter 14 is compared with the image map generated from the existing measurement data (corresponding image location). The surrounding environment is estimated by comparing the pixel values of For example, the output unit 17 displays and outputs the processing result, or writes the processing result data to a storage unit (not shown).

  Next, each component will be described in detail. The peripheral measurement unit 11 estimates and measures the distance between the peripheral object and the own mobile body based on the image information of the camera (camera 1a) mounted on the mobile body.

  For example, by comparing the amount of pixel movement in the captured image of a camera that shoots the front, back, left, and right of a car that is a moving body with the estimated movement change amount of the camera, the surrounding environment and the moving body equipped with the camera And estimate the distance.

  As a measurement and estimation method using such a camera image, for example, there are a method using a stereo camera image or two images of the same camera with a time difference (laser, millimeter wave, light, etc. without using a camera image). A direct sensor measurement method may be used).

  Next, the position acquisition unit 12 will be described. The position acquisition unit 12 acquires the current position of the moving body (current position of the camera). For example, position information in an arbitrary position coordinate system typified by latitude and longitude is acquired using a GPS (Global Positioning System) mounted on the moving body, a vehicle speed counter, and other arbitrary position measurement and position acquisition methods.

  Next, the reference route setting unit 13 will be described. The reference route setting unit 13 sets a reference route for comparing the measurement results on each road, and holds the reference route data. The reference route data is, for example, position / direction information of a plurality of points for generating a reference route, and includes position coordinate information such as latitude and longitude.

  The reference path serves as a reference for the camera position and the camera viewing direction (hereinafter, the camera viewing direction on the reference path is also referred to as a camera reference direction). Further, at least one reference route may be provided for one road, but a plurality of reference routes may be set.

  The reason why a plurality of reference routes are set instead of one is that there is a possibility that different criteria may be selected depending on the forward and return roads and each lane. For example, even if the road is the same, the road shape may be greatly different between the outward trip and the lane, and when these are known in advance, a reference route corresponding to the road shape is set. When the reference route is similar to the actual road shape, a highly accurate comparison is possible particularly on a curved road.

  Such a reference route can be prepared in advance in a database based on information such as road shape. Further, the camera movement position and the camera line-of-sight direction (camera direction) of the car that first traveled on the road can be used as the camera position and the camera line-of-sight direction of the reference route. In this case, there is an advantage that the service can be easily started and used because no special data is required for implementation.

  Next, the operation of the data converter 14 will be described with reference to FIGS. The data conversion unit 14 converts the distance data measured in a certain camera line-of-sight direction on the route of the vehicle so that the moving object becomes the distance data measured in the camera reference direction set by the reference route setting unit 13. Process.

  In this data conversion processing, a point on the reference path that minimizes the error of the converted distance data is selected as a point (mapping position) on the reference path that maps (maps) the measured distance data. By mapping the distance data measured by setting the reference path, it is possible to absorb the difference in position and the shooting direction, and it is possible to estimate the surrounding environment with high accuracy from the measurement result in the subsequent processing unit.

  FIG. 3 is a diagram illustrating an example in which peripheral objects along the road are measured from a moving body traveling on the road. A reference route is set separately from the vehicle's own vehicle route (traveling route). Here, for the sake of simplicity, the camera reference direction is a direction perpendicular to the reference path. Similarly, the camera direction when the moving body is measuring the distance from the surrounding object is a direction perpendicular to the own vehicle path at the time of measurement.

  The reference route is a route passing through three points R, S, and T. The camera reference direction at the moving point S on the reference path is determined as a perpendicular direction to the line segment RS with the previous moving point R. The vehicle route is a route that passes through two points M and N. The camera direction of the moving point N at the time of measurement is a perpendicular direction to the line segment MN. Of course, the camera direction (or camera reference direction) may be a direction other than the above. Here, the three points R, S, and T are equally spaced for convenience, but they may be at different intervals.

  The own vehicle, which is a moving body, detects the measurement point A by the peripheral measurement unit 11 at the own vehicle position N on the road, and obtains the distance (AN distance) between the measurement point A and the moving body. Then, the data conversion unit 14 converts the distance AN into data on the reference route (data conversion is performed so that the distance measured from the reference route is used).

FIG. 4 is a flowchart showing an example of the operation of the data conversion unit 14.
[S1] A reference route point around the vehicle position N is searched. For the vehicle position N acquired by the position acquisition unit 12, a reference route point near the position N is searched from the reference route data. In this example, it is assumed that the reference point S is searched.

  Here, the reference route data is held in the reference route setting unit 13. However, if the route information is grouped or hierarchized at nearby route points in advance, the data conversion unit 14 can easily perform a search and can perform a high-speed search. Become.

  For example, it is conceivable to group reference route points at intersection intervals, and further group at intersections having road units or close position information. By performing grouping at intersection intervals, it is possible to easily search for a reference route even after the vehicle turns right or left.

  In addition, although the reference | standard path | route point S of FIG. 3 was made into the vertex of the broken line which comprises a reference | standard path | route, it may actually be in the middle of a broken line. For example, an arbitrary point other than the vertex of the reference route can be acquired as a reference route point by obtaining an intersection of a straight line from the measurement point A to the vehicle position N and a neighboring reference route segment group. In this way, since it is possible to acquire a reference route point that is closer to the vehicle position N, the search time for the optimum mapping position on the reference route may be shortened.

[S2] A straight line L parallel to the camera reference direction at the reference route point S detected by searching around the vehicle position N is drawn from the measurement point A toward the reference route to obtain an intersection B with the reference route.
[S3] In order to convert the distance AN between the measurement point A and the moving body into data on the reference path, an optimum mapping position is calculated so that the conversion error is minimized on the reference path starting from the intersection B. To do.

  Since the intersection B is calculated from the camera reference direction at the reference route point S in the vicinity of the vehicle position N, the camera reference direction on the line segment ST to which the intersection B belongs and the camera reference direction used in calculating the intersection B (It can be seen from FIG. 3 that the camera reference direction at the point S is different from the camera reference direction on the line segment ST). For this reason, a better measurement result mapping position is calculated from the intersection B as a starting point (in the example of FIG. 3, the state where the point C is obtained as the optimum mapping position is shown).

  First, mapping position candidate points Bn (B1 to B3 in FIG. 3) obtained by shifting the position little by little before and after the reference path around the intersection B are obtained. Thereafter, an angle β formed by the line segment ABn and the camera reference direction of the reference path line segment ST to which the candidate point Bn belongs is obtained.

  Further, the position of the candidate point is shifted by searching for a position where the angle β is as close to zero as possible. The candidate point that minimizes the angle β, that is, the candidate point B3 in FIG.

  In addition, as a method of determining the optimum position while shifting the position, there are, for example, an iterative solution method such as Newton's method, a direct solution method, etc. as an approximate solution calculation method in numerical analysis calculation, and these can be used.

  Here, the three-dimensional coordinates of the candidate point Bn are (Bnx, Bny, Bnz), the three-dimensional coordinates of the measurement point A are (Ax, Ay, Az), and a reference direction normalization vector V that is a normalization vector of the camera reference direction. Is (Vx, Vy, Vz), the angle β can be described by the following equation (1a) (acos is arccos). D is a normalization vector from the candidate point Bn to the measurement point A, and is represented by Expression (1b).

In FIG. 3, the final candidate point C is stored in the line segment ST to which the intersection B obtained first belongs, but the candidate point search is not necessarily limited to the same line segment.
[S4] The measurement data at the measurement point A measured at the vehicle position N is converted into measurement data at the mapping position C on the reference route. That is, distance data measured in a certain camera direction (measurement direction) at the vehicle position N is converted into distance data measured in the camera reference direction D at the mapping position C on the reference route.

  For the distance data from the mapping position C, the distance between the measurement point A and the mapping position C is calculated, and this calculated distance is the distance obtained by data conversion into the reference path at the mapping position C on the reference path.

  [S5] An angle β formed by the vector from the mapping position to the measurement point A and the reference direction vector at the mapping position is calculated, and the angle is held as an error when data is converted to the reference path.

  Here, if the angle β is zero, the straight line from the final candidate point C to the measurement point A is on the same straight line as the reference direction vector D of the final candidate point C. The line segment distance AC of C is used as distance data.

  When the angle β is not zero, the line segment distance AC is used as it is, or ACcos β is obtained from the angle β and used, but it should be noted that an error occurs anyway. The error β and the value resulting from β (cos β, etc.) can be stored as an inclusion error in the reference path mapping of the measurement point A, and can be used in the subsequent comparison process. For example, when there are different measurement results, it can be used as a weight when determining the final measurement estimated value, and the accuracy can be further improved.

  Further, the candidate points converted by the reference path mapping and the measured values after conversion are all stored as they are and can be used as a database and used by the distance image conversion unit 15 and the data comparison unit 16 described later. Further, the average values and the minimum / maximum values may be held by collecting the measured values in the vicinity to some extent and used.

  The finer the measured values, the more accurately the surrounding environment can be grasped, but the amount of data increases and the amount of comparison processing also increases. For this reason, it is also effective to collect the measurement data in order to compress the measurement data amount or to remove minute noise in the measurement data value.

  Thus, since the measurement information on an arbitrary route can be adaptively converted into the measurement information on the reference route using the above method, the difference between the reference route and the vehicle route, etc. Use of data comparison processing and the like can be easily realized. Further, by holding the error at the time of measurement and conversion at the same time as the measurement value, more accurate numerical comparison can be made in consideration of the conversion error.

FIG. 5 is a diagram showing a flowchart for calculating the optimum mapping position.
[S11] Candidate points Br and Bl serving as both ends of a search range that is sufficiently shifted left and right around the intersection B are obtained.

  [S12] A candidate point Bc located at the midpoint of the candidate points Br and Bl on the reference route is obtained. Further, on the reference route, a candidate point Bcr located at the midpoint of Br and Bc and a candidate point Bcl located at the midpoint of Bl and Bc are obtained.

  [S13] The angles βcr and βcl formed by the vectors from Bcr and Bcl to the measurement point A and the reference direction vector at each candidate point are calculated, and it is determined whether or not βcr has an absolute value smaller than βcl. If it is smaller, go to Step S14, and if it is not smaller, go to Step S17.

  [S14] It is determined whether the angle βcr is smaller than a specified value or exceeds a specified number of search repetitions. If the above condition is not satisfied, the process goes to step S15. If the condition is satisfied, the process goes to step S16.

[S15] Since an optimum candidate exists between Br and Bc, the candidate point Bc is set as the candidate end Bl and Bcr is set as the midpoint candidate point Bc for the next search. Return to step S13.
[S16] Bcr is the optimum mapping position C on the reference route.

  [S17] It is determined whether or not the angle βcl is smaller than a prescribed value or exceeds a prescribed number of repeated searches. If the above condition is not satisfied, the process goes to Step S18. If the condition is satisfied, the process goes to Step S19.

[S18] Since an optimum candidate exists between Bl and Bc, the candidate point Bc is set as the candidate end Br and Bcl is set as the midpoint candidate point Bc for the next search. Return to step S13.
[S19] Bcl is the optimum mapping position C on the reference path.

Next, the distance image conversion unit 15 will be described. The distance image conversion unit 15 performs a process of converting (developing) the measurement data converted for the reference path into a two-dimensional image map.
Specifically, a plane that rises from the ground toward the sky along the reference path is defined, and the measurement data is set to any pixel value (luminance, color) for that plane (with respect to the unevenness of the plane). Convert to an image map converted to.

  For example, when the numerical value of the measurement data is quantized to a luminance value of 0 to 255 and the measurement data retains not only the distance from the reference path position but also the height information from the ground, within the defined plane A pixel that matches the height information is selected and set as a corresponding luminance value.

FIG. 6 is a diagram showing an example of creating an image map. Consider a case where measured points a to e exist using a pixel change (flow) at the center (vertical axis) of an image taken by the camera 1a.
The measurement values up to these measurement points are converted into measurement values on the reference path by the data converter 14. In FIG. 6, it is considered that the converted position on the reference route is converted to the same point m having a different height direction. Furthermore, it is assumed that the measured value is almost the same as the measured value from the camera 1a.

  The distance image conversion unit 15 selects a pixel corresponding to the point m with respect to the image plane along the reference path, using the converted position and measurement value. Then, the measurement values at the measurement points a to e are converted into the luminance of 0 to 255, and a gray scale image is created as the luminance of each selected pixel.

  In FIG. 6, a measurement point a having a large (far) measurement value obtained by measuring the sky is assumed to have a large luminance value (for example, luminance value = 255), and measurement points b and c of intermediate measurement values obtained by measuring a house are: The luminance is set to an intermediate value (for example, luminance value = 132), and the measurement points d and e having a small measured value obtained by measuring the nearest parked vehicle are set as values having a small luminance (for example, luminance value = 12). FIG. 7 shows an image map 15a created based on the luminance values at the measurement points a to e.

Note that the method of converting the measurement value into the luminance is not limited to the above, and the luminance may be determined step by step using a threshold value, or the luminance may be determined using some multidimensional function.
If the luminance is used, the existing image processing algorithm can be used in the subsequent comparison processing in the data comparison unit 16 and the processing complexity can be reduced. However, instead of the luminance, color or appropriate conversion is possible. You may convert into the numerical value classified into several steps as a numerical value.

  On the other hand, the error generated when converted into the measurement value of the reference path, that is, the value (cosβ) using the value of the angle β may be held as the above image map or as another image map. It may be held.

  For example, as the former, the error may be normalized to a value of 0 to 1, and the error may be held as the transparency of the image map. In this way, by developing the error on the two-dimensional map in the same format as the measured value image map, it is possible to execute calculation including the error in the data comparison described later, and to improve the comparison accuracy. become.

  Note that, similarly to the data holding unit after conversion described in the data conversion unit 14, the pixel resolution of this two-dimensional image map is also arbitrary, and if it is held as fine pixels, fine measurement values can be plotted as they are, By adopting an average, maximum value, minimum value, etc. of a plurality of measurement values for a large pixel, it may be used to see the tendency of that portion.

  In particular, in the case where it is intended to collectively hold in a large unit, in the data conversion unit 14, rather than searching and collecting a plurality of measurement values at the plot positions near the reference path, a single pixel image is obtained. It is possible to perform high-speed processing by using a processing function for a plurality of pixels in a pixel block unit of image processing. An example of the processing function is an averaging function.

FIG. 8 is a flowchart showing an example of the operation of the distance image conversion unit 15.
[S21] The pixel position Gm on the image map corresponding to the reference path point m is obtained.
[S22] The measured value at the reference path point m is converted to a value between 0 and 255.

[S23] The error during measurement value conversion at the reference path point m is normalized to a value of 0.0 (maximum error) to 1.0 (minimum error = none).
[S24] In a pixel group included in an area of an arbitrary size centered on the pixel Gm on the image map, a measured conversion value (0 to 255) for the current luminance and an error value (0.0 to 1.0) is set.

  In this way, the distance image conversion unit 15 does not hold the position of the measurement result as a two-dimensional map, but as a two-dimensional map, thereby reducing the amount of data during use and speeding up the comparison. It becomes possible. Further, if the error value is held in the same format as the image map of the measurement value, it is possible to easily compare the error dust.

  Next, the data comparison unit 16 will be described. The data comparison unit 16 converts the measurement data of the own mobile body (own vehicle) and the existing measurement data (measurement data acquired by another mobile body (other vehicle), etc.) converted on the reference route by the data conversion unit 14, Alternatively, the image maps obtained by further converting them by the distance image converting unit 15 are compared.

  First, a case where comparison is performed using measurement data without using an image map will be described. If the difference between two measurement results (converted measurement values) is greater than or equal to a threshold at a point that is sufficiently close on the same reference path, temporary peripheral objects such as moving objects and obstacles exist during either measurement It is determined that a temporary change in the surrounding environment has occurred.

At this time, in addition to the measurement error, one of the measurement results may include an error of the vehicle position acquired by the position acquisition unit 12 or an error when converted by the data conversion unit 14. .
In this case, if an error due to the angle β used when calculating the position C on the reference path is held in advance, the comparison may be performed in consideration of the error value. That is, using the held β instead of the distance AC, AC ± ACcos β is regarded as a registered value and compared as a value having an error width.

  In addition, as a result of comparison, one of them may not be validated, and a certain measured value may be obtained by collecting a plurality of measurement results. Specifically, it is conceivable that the measurement result is weighted with (1-cosβ) as the reliability and an average value is taken.

  When (1-cos β) is a reliability, 1 is the maximum reliability, and the reliability decreases as β increases. As a plurality of measurement results are accumulated, the average value weighted with reliability can theoretically converge to the distance to the actual surrounding environment, particularly the surrounding stationary environment without a temporary moving object.

  Note that the data comparison unit 16 can not only estimate the surrounding environment by comparing a plurality of measurement data, but also estimate the surrounding environment using a single measurement result value. That is, if you have information about the reference route (information such as the reference route is a route that passes near the center of the road) and information about the road shape (information such as the width of the road and the number of lanes) The surrounding environment can also be estimated using a single measurement result value.

  For example, it is assumed that the reference route passes near the center on the road and the width W of the road is known. In this case, when an object located outside the road, that is, an object located far from the sidewalk is measured, the measured value on the reference route should be larger than the road width W / 2. On the other hand, when a parked vehicle or the like on the road is measured, the measured value is smaller than the road width W / 2. Therefore, by taking these things into consideration, it is possible to estimate the surrounding environment using a single measurement result value.

  Next, a case where comparison is performed using an image map will be described. When comparison is made using an image map obtained by quantizing the measurement result and applied to the pixel luminance, pixel comparison of the images may be performed. That is, the luminance difference between the corresponding pixels in the image is obtained, and if the difference is equal to or greater than the threshold, the large (or small) luminance value is discarded, or a temporary peripheral moving object exists in the corresponding pixel portion I think.

  Further, it may be interpreted that a peripheral object exists when a vector of the reference path point where the pixel is located is extended by a distance corresponding to the pixel luminance. In the above description, the data comparison unit 16 performs the data comparison as a measurement value or an image map on the reference path. However, the present invention is not limited to this.

  For example, the conversion of the data conversion unit 14 is applied in reverse, and the data measured elsewhere is converted into the vehicle route via the reference route for comparison, or the data measured elsewhere is used instead of the conversion to the reference route. A similar method may be used for direct conversion to the own vehicle route for comparison.

  By acquiring the data converted into the own vehicle route, the comparison result is easy to understand intuitively, and can be displayed as it is on the navigation screen displaying the own vehicle route as a basis.

  In addition to collecting a plurality of measurement results and using them for grasping a more accurate surrounding situation, average measurement values on the reference path are accumulated, and the conversion of the data conversion unit 14 is applied in reverse. The reference data for reference can be converted into data on the route of the vehicle as a moving body and used as a surrounding situation map.

  Next, a specific example when the data comparison unit 16 performs data comparison processing using an image map will be described. FIG. 9 is a flowchart showing an operation example of the data comparison unit 16. The determination process regarding the presence or absence of a moving object that temporarily exists is shown.

First, the distance image conversion unit 15 creates an image map (MapN). In this case, the pixel Gc on the image map corresponding to the optimum mapping position C on the reference route with respect to the measurement value A at the vehicle position N, the pixel brightness (I _N ) of the measurement data after data conversion, and the transparency α The value (α _N ) is obtained, and an image map (MapN) that summarizes the measurement results of the vehicle is created.

[S31] A partial image around the pixel Gc is cut out from the image map (MapN) and an image map (hereinafter referred to as an accumulated image map (MapN-1)) that collects the results of measurement and creation by other vehicles.
[S32] A difference image between the partial images N and N-1 is calculated. In the difference image, the luminance difference ΔI at the same pixel position is set as a new luminance, and the error is set as δα. ΔI can be expressed by equation (2), and δα can be expressed by equation (3).

ΔI = I _N −I _N−1 (2)
δα = ((α _N ² + α _N-1 ² )) ^1/2 (3)
[S33] It is determined whether or not the absolute value of ΔI in the pixel Gc is larger than a specified value X (= a reference value obtained by converting the width of a temporary peripheral moving object to be searched from the distance to the pixel luminance). If it is larger, the process proceeds to step S34, and if it is not larger, the process is terminated.

  [S34] A reference pixel obtained from a difference image obtained by converting a size of a temporary peripheral moving object to be searched for a specified value X from the size to the number of pixels in the specified region range of the pixel at the vehicle position N An average value of absolute values of the luminance difference ΔI of the peripheral pixel group in (range) is acquired.

[S35] It is determined whether or not the average value is greater than a specified value X. If it is larger, the process goes to step S36, and if it is not larger, the process ends.
[S36] It is determined whether ΔI is negative. If negative, go to step S37, otherwise go to step S38.

  [S37] At the time of measurement at the vehicle position N, it is determined that there is a high possibility that a temporary peripheral moving object exists. In this case, the reflection to the image map is not performed. Note that, as a use example of the determination result, a notification that there is a high possibility that a peripheral moving object is present, or a peripheral edge of the difference image is acquired and It is considered as a contour.

  [S38] It is determined that there is a high possibility that a temporary peripheral moving object exists in the result of accumulating the measured values of the other vehicles. Examples of use of the judgment result include notification that there is a high possibility that a surrounding moving object exists, the value of the accumulated image map (MapN-1) of another vehicle is replaced with the image map (MapN) of the own vehicle, or a difference image For example, the peripheral edge of the object is acquired and regarded as the outline of the peripheral object.

  FIG. 10 is a flowchart illustrating an operation example of the data comparison unit 16. The data update / replacement processing held as image data is shown. First, an image map (MapN) is created in the distance image conversion unit 15 as in FIG.

[S41] A partial image in a range corresponding to the image map (MapN) created by the own vehicle is cut out from the accumulated image map (MapN-1) obtained by collecting the measurement results of other vehicles.
[S42] It is determined whether to update or replace the partial image N-1 of the stored image map (MapN-1) using the value of the partial image N of the image map (MapN). In the case of updating, the process goes to step S43, and in the case of replacement, the process goes to step S44.

  [S43] The own vehicle measurement value is reflected and held in the accumulated image map (MapN-1). In this case, the new luminance Ia, the new error αa, and the new accumulated data number Sa are updated for the same position pixels of the partial image N-1 and the partial image N as shown in equations (4) to (6). S is the number of accumulated data of the corresponding pixel of the partial image N-1.

Ia = (S × I _N-1 + I _N ) ÷ (S + 1) (4)
αa = (S × α _N−1 ² + α _N ² ) ÷ (S + 1) (5)
Sa = S + 1 (6)
[S44] The accumulated image map (MapN-1) is replaced using the own vehicle measurement value. New luminance Ia = I _N , new error αa = α _N , and new accumulated data number Sa = 1.

  As described above, the surrounding environment estimation apparatus 1 sets a reference route serving as a reference on each road, maps a measurement result (subject position) to the reference route, and compares measurement data. Then, assuming that there is a temporary moving object with a large difference, for example, processing such as rejecting the corresponding measurement data is performed to obtain a comprehensive distance measurement result.

  In conventional measurements, the survey point / direction (camera position / direction) differs depending on the car route, etc., making it difficult to simply compare the measured values. Therefore, the reliability of the estimation accuracy of the surrounding environment is low. Met.

  On the other hand, the surrounding environment estimation device 1 determines a reference route for each road, maps and compares the measured values on the moving route to the reference route, and further compares portions where the measurement results are clearly different during the comparison process. A determination process such as “an area where there is a possibility that a peripheral moving object (a peripheral obstacle) may exist” is performed.

As a result, differences in shooting direction and shooting position can be absorbed, measurement results can be compared with high accuracy, and the estimation accuracy of the surrounding environment can be significantly improved.
Furthermore, the data processing amount can be greatly reduced by expanding the measurement result into a two-dimensional image map that is a value in the height direction on the reference path, and maintaining and using it. Comparison processing (neighboring environment estimation processing) High speed is possible.

  The processing function of the surrounding environment estimation apparatus 1 can be realized by a computer. In that case, a program (peripheral environment estimation program) describing the processing contents of the functions that the surrounding environment estimation apparatus 1 should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium.

  Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic storage device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Optical discs include DVD, DVD-RAM, CD-ROM / RW, and the like. Magneto-optical recording media include MO (Magneto-Optical disc).

  When distributing the program, for example, a portable recording medium such as a DVD or a CD-ROM in which the program is recorded is sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  In addition, at least a part of the above processing functions can be realized by an electronic circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device).

  In addition, as a product to which the surrounding environment estimation device 1 is applied, a roadside measurement device installed in a moving body, a moving body equipped with a camera 1a, and a center having the function of the surrounding environment estimation unit 10 are communicated. The present invention can be applied to various forms such as a traffic system in which measurement data is updated and used or effective measurement data is extracted from the accumulation result in the center.

As mentioned above, although embodiment was illustrated, the structure of each part shown by embodiment can be substituted by the other thing which has the same function. Moreover, other arbitrary structures and processes may be added.
(Appendix 1) In the surrounding environment estimation device for estimating the surrounding environment,
Recognizing the camera position and shooting direction, setting a reference path including one or more reference path points that are known positions, and based on the image captured by the camera, the peripheral object captured in the image A surrounding environment estimation unit for obtaining a distance from the reference route to the surrounding environment.

  (Additional remark 2) When the said camera or the said surrounding environment estimation apparatus and the said camera are installed in a mobile body, the said surrounding environment estimation part sets the path | route of the said mobile body which passed first as the said reference | standard path | route. The surrounding environment estimation apparatus according to appendix 1, which is characterized.

(Supplementary Note 3) When the surrounding environment estimation unit is installed on a moving object, the camera or the surrounding environment estimation device and the camera are installed.
By performing a data conversion process for converting the distance between the surrounding object and the moving body measured on the traveling path of the moving body into a distance on the reference path, the distance from the reference path to the surrounding object is determined. Find the distance
The surrounding environment estimation apparatus according to claim 1, wherein in order to perform the data conversion process, a mapping position on the reference path is calculated so as to minimize an error by mapping a point on the reference path. .

  (Additional remark 4) The said surrounding environment estimation part hold | maintains at least the conversion error at the time of performing the said data conversion process as an error accompanying a measurement, The surrounding environment estimation apparatus of Additional remark 3 characterized by the above-mentioned.

(Additional remark 5) The said surrounding environment estimation part groups the reference path point which forms the said reference path for every surrounding area, and is hold | maintained, The surrounding environment estimation apparatus of Additional description 1 characterized by the above-mentioned.
(Additional remark 6) The said surrounding environment estimation part hold | maintains the distance from the said reference | standard path | route to the said surrounding object as a two-dimensional map, The surrounding environment estimation apparatus of Additional remark 1 characterized by the above-mentioned.

  (Additional remark 7) The said surrounding environment estimation part estimates the change of a surrounding environment by comparing the distance from the said reference | standard path | route to the said surrounding object and the present measurement distance. The surrounding environment estimation apparatus according to 1.

(Additional remark 8) In the surrounding environment estimation program which makes a computer perform control which estimates a surrounding environment,
The computer recognizes the position and shooting direction of the camera, sets a reference path including one or more reference path points that are known positions, and is captured in the image based on the image captured by the camera. Obtaining a distance from the reference path to the surrounding object;
A surrounding environment estimation program characterized by causing processing to be executed.

(Supplementary note 9) In the surrounding environment estimation system for estimating the surrounding environment,
A camera,
Recognizing the position and shooting direction of the camera, setting a reference path including one or more reference path points that are known positions, and based on the image captured by the camera, the peripheral imaged in the image A surrounding environment estimation unit for obtaining a distance from the reference route to the object;
An output unit for outputting a processing result of the surrounding environment estimation unit;
The surrounding environment estimation system characterized by having.

1 Ambient environment estimation device 1a Camera 10 Ambient environment estimation unit

Claims (6)

In the surrounding environment estimation device for estimating the surrounding environment,
Recognizing the camera position and shooting direction, setting a reference path including one or more reference path points that are known positions, and based on the image captured by the camera, the peripheral object captured in the image A surrounding environment estimation unit for obtaining a distance from the reference route to the surrounding environment.   The surrounding environment estimation unit, when the camera or the surrounding environment estimation device and the camera are installed on a moving body, sets a route of the moving body that first passes as the reference route. Item 1. The surrounding environment estimation apparatus according to Item 1. The surrounding environment estimation unit, when the camera or the surrounding environment estimation device and the camera are installed on a moving body,
By performing a data conversion process for converting the distance between the surrounding object and the moving body measured on the traveling path of the moving body into a distance on the reference path, the distance from the reference path to the surrounding object is determined. Find the distance
2. The surrounding environment estimation according to claim 1, wherein in order to perform the data conversion process, a mapping position on the reference path is calculated so as to minimize an error by mapping a point on the reference path. apparatus.   The surrounding environment estimation unit estimates a change in the surrounding environment by comparing the calculated distance from the reference route to the surrounding object and a current measurement distance. Ambient environment estimation device. In a surrounding environment estimation program that causes a computer to execute control that estimates the surrounding environment,
The computer recognizes the position and shooting direction of the camera, sets a reference path including one or more reference path points that are known positions, and is captured in the image based on the image captured by the camera. Obtaining a distance from the reference path to the surrounding object;
A surrounding environment estimation program characterized by causing processing to be executed. In the surrounding environment estimation system that estimates the surrounding environment,
A camera,
Recognizing the position and shooting direction of the camera, setting a reference path including one or more reference path points that are known positions, and based on the image captured by the camera, the peripheral imaged in the image A surrounding environment estimation unit for obtaining a distance from the reference route to the object;
An output unit for outputting a processing result of the surrounding environment estimation unit;
The surrounding environment estimation system characterized by having.

Images