JP2009265891A - Obstacle recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely and certainly recognize an obstacle which is perpendicular to the ground by an easy method through simple processing of photographic images. <P>SOLUTION: The photographic images for nearly the same road surface areas photographed by a photographing means 3 of one's own traveling vehicle 1 from a plurality of different directions are obtained by an image acquisition means of an image processing means 4. Then each obtained image is converted into a differential binary image and line widths are broadened by a differential binarization means and an expansion means in the image processing means 4. Then line widths is restored to original widths by shrinking the different parts in the expanded image in a calculating means and a shrinking means in the image processing means 4, and generates an image which contains only an obstacle image that is perpendicular to the ground. The recognition means of the image processing means 4 recognizes an obstacle based on the generated image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an obstacle recognition apparatus that processes a captured image and recognizes an obstacle on a traveling path of a host vehicle.

  Conventionally, when realizing pre-crash safety (reduction of damage, collision avoidance, etc.) of a vehicle, it is important how to recognize obstacles in front of and behind the traveling vehicle.

As a device for recognizing the obstacle, conventionally, after one direction (for example, the front of the vehicle) is photographed at regular intervals by a camera mounted on the vehicle, projective transformation is performed on the captured images at two times before and after. A device that takes a difference, detects a feature point having a time lag from the difference, detects an optical flow of a motion vector from the feature point, and recognizes an obstacle (a road surface vertical object) such as a vehicle in front of the host vehicle Has been proposed (see, for example, Patent Document 1).
Japanese Patent No. 3463858 (for example, claim 1, paragraphs [0014], [0041]-[0045], [0118], FIGS. 1 to 3)

  In the case of the conventional device described in Patent Document 1, by detecting the optical flow of the motion vector from the difference image after the projective transformation of the captured images at two times before and after, the road marking or the like can be detected without erroneous recognition. An obstacle is detected. In this case, using a photographed image viewed from one direction such as in front of the own vehicle, the obstacle is recognized from a slight difference accompanying a change in the distance and shape of the vehicle such as the preceding vehicle and the preceding vehicle in the image. Because of this configuration, not only complicated image processing is required to detect the optical flow, but also obstacles are not easily recognized, and obstacles may not be recognized depending on the situation. There is a problem that things cannot be recognized.

  Therefore, the applicant of the present application is a photographing unit capable of photographing a road surface viewed from a plurality of directions of the traveling vehicle, and an image obtaining unit for obtaining photographed images from a plurality of different directions of the photographing unit for substantially the same road surface area. An identification unit that extracts and identifies a predetermined monitoring range included in a captured image from a plurality of directions acquired by the image acquisition unit, and an image that differs between the captured images in the monitoring range depending on the identification result of the identification unit An obstacle recognition apparatus having a recognition means for recognizing a portion as an obstacle perpendicular to the road surface has already been filed (Japanese Patent Application No. 2007-186549).

  The obstacle recognition device of the already-applied application is, for example, a photographed image (front image) of a certain road surface area in front of the own vehicle photographed by the photographing means at time tm, and photographed at time tn when the own vehicle passes through the road surface region. For the captured image (rear image) of the same road surface area behind the host vehicle taken by the means, the identification means extracts almost the same monitoring range and identifies obstacles on the road from the difference in the images (difference in shade) To do.

  That is, if there is an obstacle ahead of the host vehicle, there is an obstacle in the monitoring range of the front image, but there is no obstacle in the rear image monitoring range. On the contrary, if there is an obstacle behind the host vehicle, there is no obstacle in the monitoring range of the front image, but there is an obstacle in the monitoring range of the rear image. And when an obstacle exists ahead or behind the host vehicle, the obstacle is included only in the monitoring range of either the front image or the rear image, and the contents of the monitoring range of both images are greatly different. Therefore, the obstacle recognition device of the already-applied application can easily detect an obstacle in front of or behind the vehicle from the difference in light and shade (brightness and darkness) of a simple image without performing complicated image processing to detect an optical flow. It can be recognized stably and reliably.

  However, in the case of the obstacle recognition device of the above-mentioned application, for example, the front image and the rear image have different shades even in a portion such as the same road marking on the road surface due to a change in the degree of light hit at the time of photographing. In addition, even when there is a slight difference between images when detecting a difference in shading, a positional deviation occurs. Therefore, there is a possibility that a part such as the road sign is erroneously recognized as an obstacle.

  An object of the present invention is to recognize an obstacle accurately and reliably without erroneous recognition of a portion such as a road marking by simple processing of a captured image.

  In order to achieve the above-described object, the obstacle recognition device of the present application includes a photographing unit capable of photographing a road surface in a plurality of directions as viewed from the traveling vehicle, and a plurality of directions of the photographing unit for substantially the same road surface region. Image acquisition means for acquiring a photographed image, differential binarization means for differentiating and binarizing each captured image acquired by the image acquisition means, and each of the differential binarization means after the differential binarization Expansion means for performing expansion processing on a photographed image, calculation means for outputting images of different portions between the captured images after expansion processing of the expansion means, and contraction processing for the image output from the calculation means And a recognizing means for recognizing an obstacle perpendicular to the road surface from the image after the shrinking process of the shrinking means.

  The obstacle recognition device of the present application further includes projective conversion means for performing projective transformation on each captured image obtained by the image acquiring means, based on each captured image after the projective conversion of the projective conversion means, The computing means outputs the image of the different portion (claim 2).

  In the case of the first aspect of the present invention, each captured image acquired by the image acquisition means is differentiated and binarized by the differential binarization means and converted into a differential binarized image.

  The differential binarized image has a wide line width due to the expansion process of the expansion means. Therefore, for example, even if the same road marking portion of the photographed front image and the rear image has a slight positional shift or lightness difference, those errors are absorbed and substantially matched by differential binarization and expansion repair, An image output from the calculation means to the contraction means (for example, an image of a different portion between the front image and the rear image) is an image in which the portion of the road marking has almost disappeared.

  Furthermore, when the line width is restored to the original by the contraction process of the contraction means, the thinned portion of the road marking disappears in the image after the contraction process, for example, either one of the photographed front image and rear image Only the image portion of the obstacle perpendicular to the road surface of the vehicle or the like that exists only in the area remains.

  Therefore, the recognition means can reliably recognize the obstacle from the image after the contraction process of the contraction means, and the road surface can be accurately and reliably recognized without erroneously recognizing road markings on the road surface by simple processing of the captured image. Obstacles such as vertical vehicles can be recognized.

  According to the invention of claim 2, by projective transformation of the projective transformation means, for example, for the front image and the rear image, an image obtained by bird's-eye view (bird's-eye view) of each photographing range can be obtained. Therefore, there is an advantage that the process of superimposing the front image and the rear image, the expansion process, and the like can be performed more easily.

  Next, in order to describe the present invention in more detail, an embodiment will be described in detail with reference to FIGS.

  FIG. 1 shows a block configuration of an obstacle recognition device 2 mounted on the host vehicle 1, and FIG. 2 shows a configuration example of an imaging means of the obstacle recognition device 2. FIG. 3 is a flowchart for explaining the operation of the obstacle recognition apparatus 2. 4, 5, and 7 to 10 show examples of images for explaining the processing of the obstacle recognition apparatus 2, and FIG. 6 is an explanatory diagram of projective transformation.

  As shown in FIG. 1, an obstacle recognition device 2 mounted on a vehicle 1 has a photographing unit 3, an image processing unit 4 having a microcomputer configuration for processing and identifying the photographed image, and a photographed image and the like can be rewritten. The data storage means 5 stores the data.

  The photographing means 3 is composed of a monochrome or color monocular camera capable of photographing a road surface viewed from a plurality of directions of the traveling vehicle 1 and outputs a frame image (or field image) taken every moment.

  The plurality of directions are all or a part of the front, rear, left, and right of the vehicle 1 (two or more directions). In the present embodiment, in order to simplify the description, The shooting directions are the two directions before and after the most practical vehicle 1.

  In addition, in order to perform two-way road surface photographing in front of and behind the vehicle 1, the photographing means 3 includes, for example, a monocular camera 3a in front of the inner mirror for monitoring the front of the vehicle 1, and a rear monitoring as shown in FIG. And a monocular camera 3b above the back door.

  The monocular cameras 3a and 3b are the same, and for example, a moving image is divided into still images at 1/30 second intervals and output. In addition, the installation angle of the monocular cameras 3a and 3b is set to an angle including an infinite point at the top of the image so that many white lines on the side of the vehicle can be used when projective transformation is performed.

  Next, the image processing means 4 forms the image acquisition means, projective transformation means, differential binarization means, expansion means, calculation means, contraction means, and recognition means of the present invention. The recognition processing program in steps S1 to S9 is repeatedly executed.

  The image acquisition means is means for acquiring photographed images from a plurality of different directions of the photographing means 3 for substantially the same road surface area, and FIG. 4 is taken by the monocular camera 3a at time ta in steps S1 and S2 of FIG. A captured image (hereinafter referred to as a forward image) F1 of the road surface area ahead of the host vehicle 1 is captured and held in the data storage means 5, and is delayed by a minute time Δt determined from the host vehicle speed or the like of a vehicle speed sensor (not shown). At the time tb, a captured image (hereinafter referred to as a rear image) B1 of substantially the same road surface area behind the host vehicle 1 in FIG. 4 captured by the monocular camera 3b is captured and held in the data storage means 5. 4 includes only the road marking a, and the rear image B1 includes the following vehicle b as an obstacle perpendicular to the road surface and includes the road marking a and the vehicle b.

  In the case of this embodiment, the projective transformation means performs projective transformation on the front image F1 and the rear image B1 in step S3 of FIG.

  As shown in FIG. 6, this projective transformation is a well-known coordinate transformation in which a point P (x, y) on a plane L is projected as a point P0 (u, v) on another plane L0 with respect to the projection center O. Yes, in a road environment where the photographing road surface of the monocular cameras 3a and 3b can be assumed to be a plane, it can be projectively transformed into an image viewed from an arbitrary viewpoint.

  The projection is performed on the assumption that the road surface is a plane with respect to the front image F1 obtained by photographing the front situation with the monocular camera 3a at the time ta and the rear image B1 obtained by photographing the rear situation with the monocular camera 3b at the time tb later than the time tb. When converted, the front image F2 and the rear image B2 after the projective transformation shown in FIG. 4 are obtained.

  Since the obstacle such as the vehicle b is recognized by superimposing the images using the front image F2 and the rear image B2, the front image F2 and the rear image F3 are actually normalized so that they can be superimposed. It is desirable to form. Therefore, (i) the same area is imaged by the monocular cameras 3a and 3b. (Ii) Using the angles and heights of the monocular cameras 3a and 3b, an image viewed from directly above is generated by projective transformation on the assumption that the road surface is a complete plane. At this time, the white lines on both sides of the lane are always extracted using the Hough transform, and the angle is adjusted so as to be parallel. (Iii) The width and direction of the lanes of the front image F2 and the rear image B2 are matched with the straight line component of the white line obtained by the Hough transform. (Vi) Since the distance moved from the own vehicle 1 can be grasped from the vehicle speed or the like, the front image F2 and the rear image B2 are accurately aligned in an area slightly larger than the traveling lane by template matching or the like.

  When the front image F2 and the rear image B2 obtained in this way are overlapped to obtain a difference in density, the difference image FB2 of FIG. 7 is obtained, but the difference image FB2 is dark in the area close to the monocular cameras 3a and 3b. Therefore, the shade varies greatly depending on the direction of the monocular cameras 3a and 3b with respect to illumination light such as sunlight. In addition, the degree of change in the shade varies depending on the material of the road surface. Therefore, there is a possibility that an obstacle such as the vehicle b existing only in one of the front image F2 and the rear image B2 cannot be recognized from the difference image FB2. The same applies to the case where an obstacle is recognized from the difference image of the shade between the front image F1 and the rear image B1 before projective transformation.

  The differential binarization means is means for differentiating and binarizing the captured images from a plurality of directions acquired by the image acquisition means or their projective transformation images, and in the case of this embodiment including the projective transformation means, FIG. In step S4 of FIG. 3, the front image F2 and the rear image B2 are differentiated to enhance the density, and then binarized with a predetermined threshold to form the differentiated binarized front image F3 and rear image B3 of FIG. . In the differentiation process, it is preferable to obtain the differential values of the front image F2 and the rear image B2 after performing smoothing so that unnecessary differential values based on the unevenness of the road surface are not generated.

  The expansion means performs an expansion process on the front image F3 and the rear image B3 in step S5 of FIG. 3 to increase the line width formed by the respective logic “1” pixels, and after the expansion process of FIG. A front image F4 and a rear image B4 are formed. In the expansion process, for example, each of the front image F3 and the rear image B3 is divided into appropriate small areas of, for example, 4 pixels or 8 pixels, and if even one of the pixels in each small area is “1”. This is realized by setting all the pixels in the small area to “1”.

  The computing means computes an exclusive OR of the front image F4 and the rear image B4 in step S6 of FIG. 3, and the synthesized image (exclusive image of FIG. 5) is obtained as an image of a portion having different shades of the front image F4 and the rear image B4. Logical OR image) FB4 is output.

  The lower side of the front images F2, F3, and F4 is the own vehicle 1, and the upper side of the rear images B2, B3, and B4 is the own vehicle 1 side.

  Then, in the portion of the road sign a of the composite image FB4, the front image F4 and the rear image B4 overlap, most of the portions cancel each other, and the end portions that do not overlap remain thin. Further, different portions of the front image F1 and the rear image B1, that is, the portion of the vehicle b are thick outlines that have been subjected to expansion processing. In addition, instead of the exclusive OR operation, an operation for obtaining the difference between the images F3 and B3 may be performed. In this case, for example, by setting the pixel whose operation result is “−1” to “0”. The same image as the composite image FB4 is obtained.

  The contraction means performs a contraction process opposite to the expansion process on the composite image FB4 in step S7 of FIG. 3, narrows the line width of the composite image FB4, and remains thin in the road marking a portion of the composite image FB4. The composite image (contraction image) FB5 of FIG.

  The recognition means recognizes the portion of the vehicle b in the composite image FB5 as an obstacle perpendicular to the road surface in step S8 of FIG. 3, and recognizes the recognition result as a collision prediction processing means (not shown) for reducing damage to the own vehicle 1 or avoiding collision. Z)).

  That is, in general, when a differential binarized image is subjected to an expansion process and a contraction process, the points and lines of the differential binarized image expand and contract to return to their original state. Then, when a straight line having a coordinate distance of d between two differential binary images is expanded by d + α and synthesized, the two differential value regions overlap with each other with a width 2α. Further, when an exclusive logical sum of two differential binarized images expanded by d + α times is taken, an edge (a different image portion) of a recognition target object that exists only in one of the differential binarized images is expanded. Even if they do not overlap, image portions such as road markings that remain in common in both differential binarized images overlap and become thin. Further, when the images of different parts obtained by taking the exclusive OR of both differential binary images are contracted d + α times, the thinned image parts disappear and the edge part of the recognition target object returns to the original. Remain.

  Then, the obstacle recognition device 2 of the present embodiment uses the above-described expansion and contraction characteristics with respect to the differential image, and performs an expansion process on the front image F3 and the rear image B3 that are differential binarized images of the same road surface. A composite image FB4 of exclusive OR of the front image F4 and the rear image B4 is obtained. At this time, the composite image FB4 remains in the recognition target vehicle b, which exists only in the rear image B3, without overlapping even if it is inflated, and the common road marking a existing in both the front image F3 and the rear image B3. The parts overlap and narrow. Further, the composite image FB4 is subjected to the same number of contraction processes as the expansion process, the road sign a disappears and the vehicle b returns to its original state to obtain a composite image FB5, and the vehicle b as an obstacle is obtained from the composite image FB5. recognize.

  In this case, in the composite image 5 after the contraction process, the thin remaining portion of the road sign a disappears, and only the portion of the vehicle b as an obstacle that exists only in one of the front image F1 and the rear image B1 remains. . Therefore, the recognizing means can accurately and accurately recognize the vehicle b as an obstacle, and the obstacle perpendicular to the road surface can be recognized by simple processing of the captured image without erroneously recognizing the road marking a on the road surface. It can be recognized accurately and reliably.

  In addition, since the projection conversion means is provided, expansion processing, exclusive OR processing, and the like can be easily performed based on the front image F2 and the rear image B2 obtained by bird's-eye view of the imaging ranges of the front image F1 and the rear image B1.

  In addition, when the exclusive OR of the front image F3 and the back image B3 which are differential binarized images is taken, a composite image FB3 of FIG. 8 is obtained. Then, as is clear from the comparison between the composite image FB3 and the composite image FB5, the road marking a portion remains only by taking the exclusive OR of the front image F3 and the rear image B3, resulting in erroneous recognition of the obstacle. .

  This is because the road surface is not a perfect plane, and the projection transformation image is distorted as it is far from the ideal plane, so the white line extracted by the Hough transformation is distorted, and the front image F3 and the rear image B3 are accurately superimposed. This is because it becomes difficult.

  Next, when an experiment was conducted in a situation where no vehicle was present before and after the host vehicle 1, the results of FIGS. 9 and 10 were obtained. 9 and 10, F11, F21, and F41 are front images corresponding to the front images F1, F2, and F4, B11, B21, and B41 are rear images corresponding to the rear images B1, B2, and B4, and FB41 and FB51 are synthesized. It is a composite image corresponding to images FB4 and FB5.

  If there is no vehicle b or the like that is an obstacle, even if the front image F41 and the rear image B41 include the expansion processing image of the road sign a that is differentiated and binarized, the expansion processing and exclusive OR are performed. It was confirmed that the combined image FB51 finally obtained by the processing and the contraction processing is an image in which the image portion is almost completely erased.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention.

  For example, in the embodiment, the projective transformation is performed before the differential binarization, but the projective transformation may be performed at any timing such as after the differential binarization.

  Next, in the above-described embodiment, the present invention is applied to the case where the road surface is photographed from the front and the rear of the own vehicle 1, but the photographing direction is not limited to the front and the rear. There may be two directions with the side, or three or more directions of front, back, left, and right.

  The present invention can be applied to obstacle recognition of various vehicles.

It is a block diagram of one embodiment of the present invention. It is explanatory drawing of the structural example of the imaging means of FIG. It is a flowchart for operation | movement description of FIG. It is explanatory drawing of the image formed by the one part process of FIG. 1, when a vehicle exists only behind the own vehicle. It is explanatory drawing of the image formed by the process of the remaining one part of FIG. 1, when a vehicle exists only behind the own vehicle. It is explanatory drawing of the projective transformation of FIG. It is explanatory drawing of the example of a difference image of the front image and back image after the projective transformation of FIG. It is explanatory drawing of the example of a difference image of the front image and back image after differentiation binarization of FIG. It is explanatory drawing of the image formed by the one part process of FIG. 1, when a vehicle does not exist ahead and back of the own vehicle. It is explanatory drawing of the image formed by the remaining processes of FIG. 1, when a vehicle does not exist ahead and back of the own vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Own vehicle 2 Obstacle recognition apparatus 3 Imaging | photography means 4 Image processing means b Vehicle as an obstruction

Claims (2)

Photographing means capable of photographing the road surface in multiple directions as seen from the traveling vehicle;
Image acquisition means for acquiring captured images from a plurality of different directions of the imaging means for substantially the same road surface area;
Differential binarization means for differentiating and binarizing each captured image acquired by the image acquisition means;
Expansion means for performing expansion processing on each captured image after differential binarization by the differential binarization means;
A computing means for outputting an image of a different part between each captured image after the expansion processing of the expansion means;
Contraction means for performing contraction processing on the image output from the calculation means;
An obstacle recognition apparatus comprising: a recognition means for recognizing an obstacle perpendicular to the road surface from the image after the shrinking process of the shrinking means. The obstacle recognition apparatus according to claim 1,
Further comprising projective transformation means for performing projective transformation on each captured image after acquisition by the image acquisition means,
The obstacle recognizing apparatus, wherein the computing means outputs the image of the different part based on each photographed image after the projective transformation of the projective transforming means.