WO2018123641A1 - Travelable area detection device and travel assistance system - Google Patents

Abstract

The present invention achieves: a travelable area detection device that can detect breaks in road edges ahead of a moving body and determine whether there are travelable areas that the moving body can travel from the detected breaks in the road edges; and a travel assistance system that uses the travelable area detection device. The travelable area detection device 1: detects the position of a roadside of a travel area from a stereoscopic image generated on the basis of a plurality of images acquired by a stereo camera device 2; and uses position information for detected road edges to estimate the change in height of breaks between the road edges and the change in gradient of areas that extend from the breaks in the road edges to the opposite side of a travel lane. The present invention can thereby determine whether the breaks in the road edges are areas that can be traveled by a vehicle V.

Description

RUNNING AREA DETECTION DEVICE AND RUNNING SUPPORT SYSTEM

The present invention relates to a travelable area detection device and a travel support system.

Conventionally, based on information obtained from a camera or radar mounted on a vehicle such as an automobile that is a moving body, a white line or a roadside (a sidewalk, a roadside belt, a curb, etc.) A system has been developed that detects the boundary of the vehicle and uses the detection result for driver assistance.

Patent Document 1 discloses an in-vehicle system in which a road end break is detected by an in-vehicle camera that captures the periphery of a vehicle, and the target section is set as a travelable area based on edge information of the break section.

JP 2016-18371 A

However, in the technique disclosed in Patent Document 1, road edge interruption is detected, but calculation of the distance of the road edge interruption section and whether or not the area on the opposite side of the roadway of the road edge interruption section can travel. There is a problem that it is difficult to detect a reliable travelable region.

For this reason, a travelable region detection device capable of detecting a road end break in front of a vehicle that is a moving body and determining whether or not there is a travelable region in which the vehicle can travel from the break. And a driving support system using the same has been desired.

The present invention has been made in view of the above-described problems of the prior art. The object of the present invention is to detect a road end break in front of the moving body, and the moving body travels from the detected road end break. It is to realize a travelable area detection device capable of determining whether there is a travelable area that can be performed and a travel support system using the travelable area detection device.

In order to solve the above-described problems, the present invention is configured as follows.

In the travelable area detection device,
Based on the captured image, a road edge detection unit that detects a road edge existing on the road on which the mobile body travels and outputs road edge information;
A search range determination unit that determines a search range based on road edge information detected by the road edge detection unit;
Based on the search range determined by the search range determination unit, a road surface gradient estimation unit that estimates the road surface gradient of the road-side travelable determination region as viewed from the mobile body,
A travelable region determination unit that determines whether or not the travelable determination region is a region in which the moving body can travel from at least the road surface gradient estimated by the road surface gradient estimation unit.

In the driving support system,
An imaging device for imaging the front of the moving body;
Based on the captured image captured by the imaging device, a road edge detection unit that detects a road edge existing on a road on which the moving body travels and outputs road edge information; and a road detected by the road edge detection unit A search range determination unit that determines a search range based on edge information, and a road surface gradient in the travelable determination region on the road edge side as viewed from the mobile body is estimated based on the search range determined by the search range determination unit. A road surface gradient estimation unit; and a travelable region determination unit that determines whether or not the travelable determination region is a region in which the mobile body can travel based on at least the road surface gradient estimated by the road surface gradient estimation unit. A travelable area detection device;
A travel support control device that assists travel by controlling the operation of the moving body according to the determination result of the travelable region detection device.

According to the present invention, it is possible to detect a road edge break in front of the mobile body and determine whether or not there is a travelable region in which the mobile body can travel from the detected road edge break. A travelable region detection device and a travel support system using the travelable region detection device can be realized.

1 is a schematic configuration diagram of a travel support system to which a travelable area detection device according to an embodiment of the present invention is applied. It is a flowchart of a driving | running | working area | region determination process. It is an overhead view which shows the road end point group detected by the driving | running | working area | region detection apparatus. It is a figure which shows the road edge point group and search range which were detected by the driving | running | working area | region detection apparatus.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

An embodiment of the present invention is an example when applied to a vehicle as a moving body.

FIG. 1 is a schematic configuration diagram of a travel support system to which a travelable area detection device according to an embodiment of the present invention is applied.

In FIG. 1, the driving support system is mounted on a vehicle V (shown in FIG. 3) such as an automobile, and mainly includes a plurality of (two in this embodiment) cameras that capture the front of the vehicle V. A stereo camera device (imaging device) 2, a travelable region detection device 1 that detects a travelable region around the vehicle V from a plurality of images captured in synchronization with each camera of the stereo camera device 2, and travel Based on the detection result of the possible region detection device 1, various control devices (for example, an accelerator control unit 31, a brake control unit 32, a speaker control unit 33, a steering control unit 34, etc.) mounted on the vehicle V are controlled to correspond to the corresponding vehicle. And a travel support control device 3 that supports travel by controlling the operation of V.

The stereo camera device 2 is installed toward the front of the vehicle V, for example, in front of the upper part of the windshield in front of the vehicle V, and as a pair of imaging units that capture the image of the front of the vehicle V and acquire image information. Left camera (left image acquisition unit) 10 and right camera (right image acquisition unit) 11.

Each of the left camera 10 and the right camera 11 has an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and the left and right directions of the vehicle (direction perpendicular to the longitudinal direction of the vehicle). Are installed so as to image the front of the vehicle V from positions separated from each other.

The travelable region detection device 1 is a device that detects a travelable region based on the image information of the imaging target region in front of the vehicle V acquired in time series by the stereo camera device 2 in a predetermined cycle.

The travelable area detection device 1 detects a travelable area from a road edge that is a boundary with a sidewalk, a roadside belt, a curb, etc. on the side of the travel road, and outputs the detection result to the travel support control device 3. To do.

The travelable area detection device 1 controls the stereo camera device 2 (for example, controls the imaging timing and exposure amount of each camera), RAM (Random Access Memory) which is a temporary storage area, ROM (Read Only Memory) that stores programs and various initial values, CPU (Central Processing Unit) recognition information that controls the entire system, etc. External IF (Interface) image processing LSI (Large Scale Integration) that outputs outside the system, etc. Each component is communicably connected via a bus.

As shown in FIG. 1, the travelable area detection device 1 includes a parallax image generation unit 12, a road edge detection unit 13, a search range determination unit 14, a road surface gradient estimation unit 15, and a travelable region determination unit 16. It has.

The left image acquisition unit 10 and the right image acquisition unit 11 of the stereo camera device 2 acquire the captured left image and right image, respectively, and transmit the acquired left image and right image to the parallax image generation unit 12.

The travel support control device 3 is based on the detection result (determination result) received from the travelable region detection device 1, and the accelerator control unit for assisting the operation of the speaker control unit 33 of the vehicle V and the travel of the vehicle V. The control amount of 31, the control amount of the brake control unit 32, and the control amount of the steering control unit 34 are calculated to adjust the accelerator, the brake, the steering, and the like.

In addition, below, the case where the driving | running | working area | region detection apparatus 1 detects the driving | running | working area | region which exists ahead of the applicable vehicle V is demonstrated concretely.

Since there is a change in the height of the road edge (such as a sidewalk, curbstone, or median) on the road of the vehicle, there is a change in height (for example, a change in the height of the road edge or slope), so the height of the interrupted section It is possible to determine whether or not there is a travelable region where the vehicle can travel from the interrupted section by using the change and the gradient change of the road surface in the area opposite to the traveling lane of the interrupted section. It is considered possible.

Next, the travelable area determination process executed by the travelable area detection device 1 will be described with reference to FIGS. 2, 3, and 4. FIG. 2 is a flowchart of the travelable area determination process, and FIG. FIG. 4 is a diagram showing a road end point group and a search range detected by the travelable area detection device 1. A travelable area determination process will be described with reference to the flowchart shown in FIG. This process is started when the vehicle starts running as an example.

2 is processing of the parallax image generation unit 12, steps S2, S3, and S4 are processing of the road edge detection unit 13, and steps S5 and S6 are processing of the search range determination unit 14. is there.

Step S7 is a process of the road surface gradient estimation unit 15, and steps S8 and S9 are a process of the travelable area determination unit 16.

2, a parallax image is generated using the left image and the right image acquired by the left image acquisition unit 10 and the right image acquisition unit 11, and the generated parallax image is processed in step S2.

Specifically, based on the left image and the right image acquired by the left image acquisition unit 10 and the right image acquisition unit 11, a parallax (positional deviation between images) is calculated by block matching processing, and the calculated parallax is calculated. Based on this, a parallax image including three-dimensional distance information (spatial information) is generated.

More specifically, in step S1, in block matching processing, one image (for example, the left image) is divided so as to include a plurality of pixels by a first block having a predetermined shape, and the other image (for example, the right image) is divided. The second block is the same size, shape and position as the first block.

Then, the second block is shifted by one pixel at a time in the horizontal direction to calculate the correlation value for the two luminance patterns in the first block and the second block at each position, and the correlation value is the smallest, that is, the correlation Search for the highest position (corresponding point search).

In addition, as a calculation method of a correlation value, SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), NCC (Normalized Cross Correlation), a gradient method, etc. can be used, for example.

In the case of search, when the position where the correlation is highest is specified, the distance between the specific pixel in the first block and the specific pixel in the second block at the specified position is calculated as parallax, A parallax image can be generated by executing the same step for all the pixels as a single step.

Step S2 uses the parallax image acquired in Step S1 to extract road edge parts that can be candidates for road edges (eg, sidewalks / curbs, guardrails, continuous poles, walls, etc.) on the road, and the extracted road edges Is processed in step S3.

Describing step S2 in detail, in the parallax image acquired in step S1, using the fact that the distance change near the object that is the road edge candidate is smaller than the distance change of the flat part, Detect feature points. For example, the image data is scanned in the vertical direction to detect a step that can be regarded as a roadside object.

Next, in step S3, the detected road edge is distributed to the left road edge and the right road edge of the vehicle V.
Furthermore, as shown in FIG. 3, the road end point group 100 other than the roadside objects such as other vehicles and pedestrians (the illustrated example is a case of the vehicle) is excluded, and the road end point group used for the travelable area search Only 101 is extracted, and only the extracted road edge information (output from the road edge detector 13) is processed in step S4.

Here, the road edge used for the search for the travelable area is a boundary between the travel lane and the road edge excluding an obstacle (for example, another vehicle) on the travel lane of the vehicle V.

Next, in step S4, when there is a road edge break at the extracted road edge and it is determined that the road edge break is a travelable area candidate of the vehicle V, the extracted road edge position information is obtained. Processed in step S5. In determining the travelable area candidate of the vehicle V, the travelable area candidate is determined when the distance between the road end end point 103 and the road end start point 102 in FIG.

In step S5, the processing area used for road surface slope estimation from the road edge discontinuity section (between point 102 and point 103) detected in step S4, that is, the search area (area area on the road edge side as viewed from the vehicle). ) And the coordinates of the determined search range are processed in step S6.

Specifically, as shown in FIG. 4, the straight line 400 from the start point 102 of the road edge discontinuity section to the discontinuation end point 103 is drawn in the horizontal direction from the point 102 and the point 103 to the opposite side of the traveling road. The rectangular area 600 formed by the straight lines 200 and 300 and the straight line 500 at the screen edge is set as the search range. The setting of the road surface gradient estimation filter size described in step S5 of FIG. 2 means the setting of the rectangular area 600.

Next, in step S6, when a shield (tree, guardrail, etc.) is detected within the rectangular area 600 determined in step S5, and the road gradient cannot be estimated, the road surface is determined in step S17. The information in step S5 is processed in step S7 only when the premises estimation is impossible and the road surface gradient can be estimated.

In step S7, the gradient of the road surface of the rectangular area 600 that is the search area determined in step S5 is estimated, and the estimated gradient information is processed in step S8.

The specific estimation procedure in step S7 scans the rectangular area 600 which is a search area (running determination area) in the horizontal direction, and acquires road surface height information. As one step, the road surface height information is obtained from the near side (the vehicle V side with respect to the road edge break), and the road surface gradient in the rectangular region 600 is estimated.

In step S8, it is determined from the change in the road surface gradient estimated in step S7 whether the road surface gradient is within a range in which the vehicle V can travel. If it is determined that the vehicle cannot travel, the vehicle cannot travel. The determination is made in step S18. Only when it is determined that the rectangular area 600 can travel, the information in step S7 is processed in step S9.

In step S9, when the vehicle V enters the road edge section from the roadway, there is an area where the vehicle V can stop without protruding into the traveling lane, that is, an obstacle (such as a wall or a step). The presence / absence, presence of a hole, and whether or not the vehicle can travel are determined.

If it is determined that there is an object that becomes an obstacle, a hole, or a region where the host vehicle is not allowed to travel, it is determined in step S18 that the region is not travelable.

If it is determined in step S8 that there is no obstacle, no hole, and the vehicle can travel, the travelable region is determined in step S19. Is transmitted to the driving support device 3.

Then, the driving support control device 3 determines the road surface gradient estimation impossibility determination in step S17, the improper travel area determination in step S18, and the step based on the shielding determination in step S6 and the determination results in step S8 and step S9. Based on the result of the determination of the travelable area in S19, the travel support of the vehicle V is implemented.

As described above, in the travelable region detection device according to the embodiment of the present invention, the position of the road edge in the travel region is detected from the parallax images generated based on the plurality of images acquired by the stereo camera device 2. Then, based on the detected position information of the road edge, a change in the height direction of the break between the road edges 102 and 103 and a change in the gradient of the region extending from the road edge break to the opposite side of the traveling lane are estimated. Thus, it can be determined whether or not the road edge break is in the travelable region of the host vehicle V.

As a result, for example, even when a road end break on the road of the host vehicle V is detected, a travelable area can be estimated from the road end break, and high-precision driving support for automatic driving is provided. can do.

In other words, a road area detection is possible in which a road edge break in front of the moving body is detected, and it is possible to determine whether or not there is a driveable area in which the mobile body can travel from the detected road edge break. A device and a driving support system using the device can be realized.

In addition, although the example mentioned above is an example which mounts the driving | running | working area | region detection apparatus of this invention on a vehicle, this invention is applicable not only to a vehicle but another mobile body. For example, the present invention can be applied to a traveling robot (rescue robot) used for disasters.

In the above-described example, an imaging element such as a CCD or CMOS is taken as an example of the stereo camera device, but a laser radar can also be used as the imaging device.

In the above example, the parallax image generation unit 12 generates a parallax image and detects a roadside object. However, the roadside object can be detected by image recognition processing other than the parallax image. It is.

DESCRIPTION OF SYMBOLS 1 ... Travel possible area | region detection apparatus, 2 ... Stereo camera apparatus, 3 ... Travel assistance control apparatus, 10 ... Left image acquisition part, 11 ... Right image acquisition part, 12 ... Parallax Image generation unit, 13 ... road edge detection unit, 14 ... search range determination unit, 15 ... road surface gradient estimation unit, 16 ... travelable region determination unit, 31 ... accelerator control unit, 32 ... Brake control unit, 33 ... Speaker control unit, 34 ... Steering control unit, 100 ... Road end point group other than roadside objects such as other vehicles and pedestrians, 101 ... Road end point group, 102: Start point of break section, 103: End point of break section, 200, 300 ... Straight line drawn in the horizontal direction, 400 ... Straight line to break point, 500 ... Screen edge Straight line, 600 ... rectangular area (search area)

Claims (12)

Based on the captured image, a road edge detection unit that detects a road edge existing on the road on which the mobile body travels and outputs road edge information;
A search range determination unit that determines a search range based on road edge information detected by the road edge detection unit;
Based on the search range determined by the search range determination unit, a road surface gradient estimation unit that estimates the road surface gradient of the road-side travelable determination region as viewed from the mobile body,
A travelable region determination unit that determines whether or not the travelable determination region is a region in which the mobile body can travel from at least the road surface gradient estimated by the road surface gradient estimation unit;
A travelable area detecting device comprising: The travelable area detecting device according to claim 1,
A parallax image generation unit that generates a parallax image from a plurality of images is further provided, and the road edge detection unit detects a road edge present on the road based on the parallax image generated by the parallax image generation unit. A travelable area detecting device characterized by the above. The travelable area detecting device according to claim 1,
The travelable region detection device, wherein the travelable region determination unit determines that the travelable region cannot travel when the detected road gradient is greater than a predetermined value. . The travelable area detecting device according to claim 1,
The travelable area determination unit determines whether there is an obstacle, a hole, and an area where the moving body can travel in the travelable determination area, and there is an obstacle. Or a travelable area detecting device characterized in that, when it is determined that a hole exists or the mobile body is not in an area where the mobile body can travel, the mobile body determines that the travelable determination area cannot travel. . The travelable area detecting device according to claim 4,
The travelable area determination unit is configured such that the detected road gradient is equal to or less than a predetermined value, and there is no obstacle, no hole, and the moving body in the travelable determination area. The travelable region detecting device, wherein when the region is a travelable region, the mobile body determines that the travelable determination region can travel. In the travelable region detection device according to any one of claims 1, 2, 3, 4, and 5,
The travelable region detecting device, wherein the moving body is a vehicle. An imaging device for imaging the front of the moving body;
Based on the captured image captured by the imaging device, a road edge detection unit that detects a road edge existing on a road on which the moving body travels and outputs road edge information; and a road detected by the road edge detection unit A search range determination unit that determines a search range based on edge information, and a road surface gradient in the travelable determination region on the road edge side as viewed from the mobile body is estimated based on the search range determined by the search range determination unit. A road surface gradient estimation unit; and a travelable region determination unit that determines whether or not the travelable determination region is a region in which the mobile body can travel based on at least the road surface gradient estimated by the road surface gradient estimation unit. A travelable area detection device;
A driving support control device for supporting the driving by controlling the operation of the moving body according to the determination result of the driving region detection device;
A driving support system comprising: In the driving support system according to claim 7,
The travelable area detection device includes a parallax image generation unit that generates a parallax image from a plurality of images, and the road edge detection unit exists on a road based on the parallax image generated by the parallax image generation unit. A driving support control system for detecting a road edge. In the driving support system according to claim 7,
The travel support system according to claim 1, wherein the travelable area determination unit determines that the mobile body cannot travel when the detected road gradient is greater than a predetermined value. In the driving support system according to claim 7,
The travelable area determination unit determines whether there is an obstacle, a hole, and an area where the moving body can travel in the travelable determination area, and there is an obstacle. Or a travel support system, wherein when it is determined that there is a hole or the region is not capable of traveling, the mobile unit determines that the travelable determination region is not travelable. The driving support system according to claim 10,
The travelable area determination unit is configured such that the detected road gradient is equal to or less than a predetermined value, and there is no obstacle, no hole, and the moving body in the travelable determination area. The travel support system according to claim 1, wherein when the vehicle is in a travelable region, the mobile body determines that the travelable determination region can travel. In the driving support system according to any one of claims 7, 8, 9, 10, and 11,
The traveling support system, wherein the moving body is a vehicle.

Images