JP2014089548A - Road surface level difference detection method, road surface level difference detection device and vehicle equipped with the road surface level difference detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road surface level detecting device capable of detecting not only an obstacle in front but also a level difference on a road surface.
[Solution]
In the road surface level difference detection method of the present invention, at least a first image and a second image obtained by stereo shooting of a road surface are projected on XY plane coordinates, and specific coordinates (X, Y) are centered on the plane coordinates. A detection area is set, a parallax v1 is calculated when the image of the detection area is at a road surface position, and a comparison area centered on coordinates (X−v1, Y) obtained by subtracting the parallax v1 from the second image is set. The height of the detection region from the road surface is detected by comparing the image of the detection region with the image of the comparison region.
[Selection] Figure 4

Description

  The present invention relates to a road surface step detection method, a road surface step detection device, and a vehicle including a road surface step detection device that detect a road surface step from a plurality of image data obtained by imaging a road surface with a stereo camera.

  Recently, the target scenery outside the vehicle is imaged with a camera mounted on the car, and the captured image is processed to determine the distance from the car to the object, predicting the risk of collision with the vehicle or guardrail ahead. Thus, a method of performing control such as braking the vehicle has been put into practical use.

  The distance measurement technology using such images is based on the principle of triangulation from the technology that estimates the distance to a target object from the monocular image using the relationship with the camera position and the stereo images captured by multiple cameras. It is roughly divided into the technology that calculates the distance. Among these, the technique for obtaining the distance from the stereo image based on the principle of triangulation obtains the distance from the relative shift of the position of the same object in the left and right images, and therefore, an accurate distance can be obtained.

  For example, Patent Literature 1 discloses an image storage unit that stores images input by a plurality of cameras, a feature extraction unit that extracts a plurality of white lines existing on a road surface, and an arbitrary on a road surface from the extracted white lines. An obstacle detection apparatus comprising a parameter calculation unit that obtains a relational expression that is established between projection positions of each point on each image, and a detection part that detects an object having a height from the road surface using the relational expression Disclosure.

  According to the obstacle detection device of Patent Document 1, even when there is a change in the slope of the road surface, the road surface is recognized from the movement of the two white lines, and the obstacle present on the road surface is detected at high speed and with high accuracy. Can be detected.

JP 2001-76128 A

  However, the obstacle detection device of Patent Document 1 is applied to a vehicle that travels on a road surface without a lane, such as a senior car or an electric wheelchair, in order to recognize a road surface by extracting a white line such as a lane. In this case, there is a problem that it is difficult to detect an obstacle because the road surface cannot be recognized correctly.

  The present invention has been made in view of such problems, and provides a road surface level difference detection method capable of detecting a level difference of a level difference from a road surface even when it is difficult to identify the road surface.

  In order to solve the above-described problem, the road surface level difference detecting method of the present invention projects at least a first image and a second image obtained by stereo shooting a road surface onto XY plane coordinates, and specific coordinates (X , Y) is set as the center, the parallax v1 is calculated when the image of the detection area is at the road surface position, and the coordinates (X−v1, Y) obtained by subtracting the parallax v1 from the second image are the center. The comparison area is set, and the image of the detection area is compared with the image of the comparison area to detect the height of the detection area from the road surface.

  Further, the height difference between adjacent detection areas is obtained from the heights of a plurality of detection areas, and it is determined that there is a step between the detection areas if the height difference is equal to or greater than a threshold value. .

  In addition, the height difference between adjacent detection areas is obtained from the heights of the plurality of detection areas, and there is an inclination between the detection areas when the height difference continuously changes between the plurality of detection areas. It is characterized by determining.

  In addition, the road surface level difference detecting device of the present invention is taken with at least a first camera and a second camera for taking a stereo image of a road surface, a first image taken with the first camera, and the second camera. The second image is projected onto XY plane coordinates, a detection area centered on specific coordinates (X, Y) is set on the plane coordinates, and the parallax v1 when the image of the detection area is at the road surface position is calculated. Then, a comparison area centered on the coordinate (X−v1, Y) obtained by subtracting the parallax v1 from the second image is set, and the image of the detection area is compared with the image of the comparison area. And a height calculation unit for detecting the height of the.

  A vehicle according to the present invention includes the road surface level difference detecting device described above.

  According to the present invention, a step can be easily detected from each area of a road surface in an image using an image captured by a stereo camera.

It is a block diagram of a road surface level | step difference detection apparatus. It is a layout diagram of a stereo camera. It is the image imaged with the selethio camera. It is a flowchart of the height detection process in a calculating part. It is explanatory drawing which shows the distance calculation method. It is explanatory drawing which shows the parallax calculation method. It is explanatory drawing which shows the parallax calculation method. It is explanatory drawing which shows the road surface height calculation method. It is an example of a road surface height calculation result. It is a senior car which is a vehicle provided with a road surface level difference detection device.

  An embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration of a road surface level difference detecting device 1 according to the present invention. The road surface level difference detection device 1 according to the present invention includes two cameras 11 and 12 that capture a stereo image, and a calculation unit 20 that calculates a stereo image. In addition, the calculation unit 20 includes a height detection unit 30 that calculates the height of a region where a step is desired to be detected from the stereo image, and a step detection unit 40 that determines the presence or absence of a step between the detection regions from the height of each detection region. It is composed of Furthermore, an output device 50 such as an audio speaker or a display device is provided to notify the operator of the presence or absence of a step as required.

  FIG. 2A is a top view showing the arrangement of the stereo camera, and FIG. 2B is a side view showing the mounting position of the stereo camera. The two cameras 11 and 12 have the same specifications, and have a predetermined horizontal angle of view as shown in FIG. 2A. For example, a predetermined interval g is set to the left and right in the front part of a senior car or an electric wheelchair vehicle. Installed separately. Also, as shown in FIG. 2B, the cameras 11 and 12 are installed at a predetermined height hc from the road surface, have a predetermined vertical field angle and depression angle, and the optical axis of the lens images the road surface side. It has become downward.

  If the horizontal and vertical angles of view of the cameras 11 and 12 are too large, the step of the road surface to be imaged becomes small and the detection accuracy of the step is lowered. Conversely, if the angle of view is too small, the detection range of the step becomes narrow. It is necessary to set appropriately according to conditions. The depression angle is preferably set so that the proportion of the road surface is increased. The mounting height hc of the cameras 11 and 12 is preferably set as high as possible in order to widen a detectable range from a small step to a large step.

  The specifications and arrangement of the cameras 11 and 12 have, for example, a horizontal field angle and a vertical field angle corresponding to a 35 mm lens, a mounting interval g is 15 to 25 cm, a mounting height hc is 60 to 80 cm, 10 It is installed at a depression angle of ~ 25 °. In the following description of the present embodiment, half of the horizontal and vertical angles of view are θ1 and θ2, respectively, and the depression angle is θ3. In addition, as shown in FIG. 2, an embodiment in which the cameras 11 and 12 are installed on the left and right will be described. However, the cameras 11 and 12 can be installed in the vertical and diagonal directions. The detection method is the same.

  FIG. 3 shows two images of the stereo camera, and the road surface including the sidewalk is captured. 3A is a first image captured by the left camera 11, and FIG. 3B is a second image captured by the right camera 12. Further, FIG. 3C is obtained by extracting only the boundary line between the sidewalk and the road surface by superimposing the first image and the second image. As shown in FIG. 3C, the image is taken at a position where the boundary line is shifted in the left image and the right image.

  This parallax in the left-right direction is parallax, and on a flat road surface, the parallax decreases at a constant rate from the near side to the depth side. The present invention detects the height of the detection area from the road surface by comparing the parallax v1 on the flat road surface and the actual parallax v2 obtained by imaging the step detection area.

  FIG. 4 is a flowchart of the height detection process in the calculation unit 20. When the calculation unit 20 assumes that the detection region is on the road surface from the Y coordinate and the position information of the camera with respect to the detection region centered on an arbitrary coordinate (X, Y) of the first image. The parallax v1 is obtained, a comparison area centered on the coordinate (X−v1, Y) shifted by the parallax v1 in the second image is determined, and the height from the road surface of the detection area is determined from the parallax v2 of the detection area and the comparison area. Ask.

  With the processing from step 1 to step 3 shown in FIG. 4, the detection region is detected from the Y coordinate value and the camera position information for the detection region centered on an arbitrary coordinate (X, Y) of the first image. The parallax v1 with the second image when it is assumed that the vehicle is on the road surface is obtained.

  In order to obtain the parallax v1, it is first necessary to calculate the distance d1 from the camera to the focal plane of the image. FIG. 5 is an explanatory diagram illustrating a distance calculation method. FIG. 5A shows a step detection region in which the first image is converted into a coordinate space 13 having a vertical center of ± w pixels and a horizontal side of ± h pixels with the coordinate center (0, 0) as the origin P. These coordinate points (X, Y) are shown in the coordinate space 13. FIG. 5B is a side view showing the focal plane A1 of the cameras 11 and 12 and camera position information.

  In step 1, a coordinate point (X, Y) of an arbitrary detection region for detecting a step in the coordinate space 13 is selected. The coordinate space 13 corresponds to a focal plane that is a plane perpendicular to the optical axis of the camera 11 shown in FIG. When there is no distortion of the lenses of the cameras 11 and 12, since all the objects on the focal plane have the same parallax, the coordinate point (X, Y) in the coordinate space 13 and the origin P also have the same parallax.

  In step 2, the distance d1 from the camera to the origin P of the focal plane A1 when the coordinate point (X, Y) is assumed to be on the road surface is obtained. Here, since it is complicated to calculate the distance d1 using the coordinate point (X, Y) as a base point, the same focal point as the coordinate point (X, Y) is used by utilizing that the coordinate points on the focal plane all have the same parallax. A distance d1 to the origin P of the focal plane A1 is calculated using the coordinate point Q (0, Y) on the plane A1 as a base point.

  FIG. 6 is an explanatory diagram illustrating a parallax calculation method. FIG. 6A is a side view of the camera 11 and shows a downward angle θy when the coordinate point Q is viewed from the camera 11. Assuming that half of the vertical angle of view of the camera 11 is θ2, since the height of the coordinate point Q in FIG. 5A is Y, θy can be obtained by the following equation 1.

θy = arctan (tan θ2 × Y / h) (Formula 1)
Further, as shown in FIG. 5B, when the camera mounting height is hc and the camera is mounted downward at a depression angle θ3, the distance d1 ′ from the camera 11 to the coordinate point Q is
d1 ′ = hc / sin (θ3 + θy) (Formula 2)
It can be obtained more.

Therefore, the distance d1 from the camera 11 to the origin P of the focal plane A1 is
d1 = d1 ′ × cos (θy) (Formula 3)
It can be obtained more.

  Next, in step 3, the parallax v1 between the first image and the second image when the road surface is on the focal plane A1 is obtained. In the case of a lens without distortion, all the points on the focal plane A1 are considered to have the same parallax. Therefore, the parallax v1 may be obtained using the distance d1 obtained in the above (Equation 3). The distance d1 may use another value or need to be corrected depending on lens distortion or the like.

  FIG. 6B is a top view of the left and right cameras 11 and 12. As shown in FIG. 6B, the origin P located at the focal plane A1 of the left camera 11 is seen by the right camera 12 in the direction of the angle θx from the center. This θx is obtained by the following equation, where g is the distance between the left and right cameras.

θx = arctan (g / d1) (Formula 4)
At this time, the origin P looks like the coordinates of the origin (0, 0) in the first image on the left side as shown in FIG. 7A, and the parallax in the second image on the right side as shown in FIG. 6B. When the number of pixels is v1, it appears at the point of (−v1, 0). As shown in FIG. 6A, v1 is obtained by the following equation when θ1 is half of the horizontal field angle of the camera.

v1 = w × tan θx / tan θ1 (Formula 5)
Since the number of parallax pixels v1 is the same at the point P and the point (X, Y), it appears at the position (Xr, Y) in the right image. here,
Xr = X-v1 (Formula 6)
It is.

  Next, in Step 4, it is determined whether or not the object shown in the coordinates of the detection area (X, Y) of the left first image is at the same height as the road surface. At this time, it is confirmed whether the same object as the object shown in the coordinates (X, Y) of the left first image is shown in the position of the comparison area (Xr, Y) of the right second image. do it.

  There are various methods for confirming whether or not they are the same object. For example, the luminance of several pixels around the target point of the left and right images may be extracted and compared. If they match within the range of error factors such as camera noise, the point can be determined to be the same height as the road surface. If it is determined that the two do not match and are shifted to the left or right, it can be determined that the position is higher or lower than the road surface according to the parallax.

For example, when it is determined that the object at the position (X, Y) in the first image on the left side is at the coordinates (Xr−v2, Y) in the second image on the right side, When the distance is d2, from (Expression 4) and (Expression 5),
v1 = (w × g) / (d1 × tan θ1)
v1 + v2 = (w × g) / (d2 × tan θ1)
If v1 is deleted,
d2 = (d × w × g) / (w × g + v2 × tan θ1) (Expression 7)
As shown in FIG. 8, the height hs of the object on the road surface is
hs = hc × (d1−d2) / d1 (Formula 8)
Thus, the height hs of the road level difference can be known. That is, when the parallax v2 of the object is positive (larger than the parallax v1 of the road surface), the distance d2 to the object is smaller than the distance d1 to the road surface as shown in FIG. 8A, and hs is a positive value. It can be judged that it is higher than the road surface. Conversely, when v2 is negative (smaller than the parallax v1 on the road surface), the distance d2 to the object is larger than the distance d1 to the road surface as shown in FIG. 8B, and hs is a negative value. It can be judged that it is lower than the road surface. In this way, the height difference from the road surface of the coordinate point (X, Y) in the first image is known.

  In step 6, the above procedure is repeated at other coordinate points at appropriate intervals, and the process ends when the detection of the height from the road surface within the required range in the image is completed.

  The details of the flowchart shown in FIG. 4 are as described above, and these processes are processed by the arithmetic unit 20 shown in FIG. Specifically, it may be realized as software on a PC or a microcomputer, or may be realized as hardware using an FPGA or ASIC. A configuration in which the remainder is partially processed by hardware and software is also possible.

  FIG. 9 is an example of a result of applying the above method to the stereo image of FIG. The gradation is displayed according to the height of each detection area, the area having the same height as the road surface is displayed in gray, and the area lower than the road surface is displayed in black.

  Next, the level difference detection unit 40 of the calculation unit 20 will be described. In the detection result of the height detection unit 30 as shown in FIG. 9, the height from the road surface is compared between the detection areas adjacent in the vertical and horizontal directions. It is judged that. The height difference threshold for determining a step is set so as to ensure safety even when a senior car or a wheelchair falls, for example. As an example, in the detection result of FIG. 9, the boundary determined as a step is indicated by a broken line portion.

  Further, when it is assumed that the height difference between adjacent detection areas is small and a road surface, it is possible to obtain the inclination angle of the road surface from the distance between the detection areas and the height difference. Thereby, when the road surface is inclined uphill, downhill, or left and right, it is possible to determine whether the road surface can be secured.

  FIG. 10 shows an application example of a senior car 60 as an example of a vehicle provided with the road surface level difference detecting device 1 of the present invention. The road surface level difference detection device 1 is provided in front of the handle 61 of the senior car 60 at a height hc from the road surface. The road surface level difference detection device 1 includes an output device 30 such as a speaker 31 and a display device 32 in order to notify the driver of the senior car 60 of the level difference detection result. When a road level difference is detected by the road level difference detection device 1, a buzzer sound or voice guidance is output from the speaker 53, or a character or a graphic is displayed on the display device 54 to notify the existence of the level difference on the road surface. It is possible to avoid dangers such as falling wheels and falling of the senior car 60.

  Further, the step detecting device 1 of the present invention may be provided not only in front of the senior car 60 but also in the rear. As a result, it is possible to avoid danger such as falling wheels even when reversing with poor visibility. Further, the use of the level difference detection device 1 of the present invention is not limited to the senior car 60, and is suitably used in vehicles that need to detect road level differences, for example, various vehicles ranging from wheelchairs to forklifts. It is possible.

  For example, in a wheelchair, by providing the step detection device 1 of the present invention in the front and rear and left and right, even if the direction is changed on the spot, the risk of falling or falling over the step around the wheelchair is prevented. can do. In addition, with a forklift, even if the forward field of view may be blocked during cargo transportation, it is possible to detect a short cargo placed on the road surface as a step and avoid a collision.

DESCRIPTION OF SYMBOLS 11, 12 Camera 20 Calculation part 30 Height detection part 40 Level difference detection 50 Output device

Claims (5)

Projecting at least a first image and a second image obtained by stereo shooting of a road surface onto XY plane coordinates;
A detection area centered on specific coordinates (X, Y) is set on the plane coordinates,
Calculating the parallax v1 when the image of the detection area is at the road surface position;
A comparison area centered on coordinates (X-v1, Y) obtained by subtracting the parallax v1 from the second image;
A road surface level difference detection method comprising: comparing an image of the detection area with an image of the comparison area to detect a height of the detection area from the road surface. From the height of the plurality of detection areas, to determine the height difference between the adjacent detection areas,
The road surface level detection method according to claim 1, wherein if the level difference is equal to or greater than a threshold value, it is determined that there is a level difference between the detection areas. From the height of the plurality of detection areas, to determine the height difference between the adjacent detection areas,
The road surface level difference detection method according to claim 1, wherein it is determined that there is an inclination between the detection areas when the difference in height continuously changes between the plurality of detection areas. At least a first camera and a second camera for stereo shooting of the road surface;
The first image captured by the first camera and the second image captured by the second camera are projected onto XY plane coordinates, and specific coordinates (X, Y) are centered on the plane coordinates. And a parallax v1 when the image of the detection area is at the road surface position is calculated, and the coordinates (X−v1, Y) obtained by subtracting the parallax v1 from the second image are set as the center. A road surface level difference comprising: a height calculation unit configured to set a comparison area, compare the image of the detection area with the image of the comparison area, and detect the height of the detection area from the road surface. Detection device. A vehicle comprising the road surface level difference detecting device according to claim 4.