JP2006339960A - Object detection apparatus and object detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method detecting an object that exists nearby the vehicle. <P>SOLUTION: A viewpoint conversion section 103a performs viewpoint conversion of images picked up by a right rear camera 101 and a left rear camera 102 to generate a bird's-eye image on a road surface as a projection surface. Then a solid-body detection section 103b extracts edges from a plurality of generated bird's-eye images and superposes them, detects end points of edges present in the superposed images, and groups edge end points whose existing positions are close to each other. When a plurality of edges extend in different directions from respective edge points existing in each group, it is decided that the edges existing in the group are edges detected from the solid body. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an object detection apparatus and an object detection method capable of detecting a three-dimensional object existing around a vehicle.

  The following object detection apparatus is known from Patent Document 1. According to this object detection device, two images are taken from different directions, the luminance value of one image taken is compared with the luminance value of the other image, and the three-dimensional object is located at a position where the respective luminance values are different. It is determined that it exists.

Japanese National Patent Publication No. 10-505176

  However, in the conventional apparatus, since the three-dimensional object is detected by comparing the luminance values of the two images, there is a problem in that erroneous detection may occur when the location where the light hits in each image is different. It was happening.

  The present invention creates a plurality of bird's-eye images with a road surface as a projection plane by converting viewpoints of images around the vehicle imaged from different angles by a plurality of imaging means, and extracts edges from the created bird's-eye images Then, by superimposing multiple bird's-eye images from which edges are extracted, edge endpoints that exist in the superimposed image are detected, edge endpoints that are close to each other are grouped, and edges that exist in each grouped group An edge having an angle between edges extending from the end point being a predetermined value or more is determined to be an edge detected from a three-dimensional object.

  According to the present invention, since a solid object can be detected based on the angle between edges extending from the edge end points included in each group in the superimposed image, the solid object can be detected with high accuracy.

  FIG. 1 is a block diagram showing a configuration of an embodiment of an object detection apparatus according to the present embodiment. The object detection device 100 is mounted on a vehicle and installed on the right rear side of the vehicle, a right rear camera 101 that captures an image, a left rear camera 102 that is installed on the left rear side of the vehicle and captures an image, a CPU, a memory, And other peripheral circuits, and a control device 103 that executes various processes to be described later.

  For example, as shown in FIG. 2, the right rear camera 101 and the left rear camera 102 are installed so as to capture the rear of the vehicle from the left and right sides of the vehicle, and continuously capture images of the rear of the host vehicle at regular time intervals. That is, the images behind the vehicle are continuously taken from different angles. In the present embodiment, the case where the right rear camera 101 and the left rear camera 102 capture the rear of the vehicle from different angles will be described. However, the present invention is not limited to this, and the front of the vehicle may be captured from different angles. You may make it image the vehicle side from a different angle.

  The control device 103 includes a viewpoint conversion unit 103a and a three-dimensional object detection unit 103b. The viewpoint conversion unit 103a generates viewpoint images (bird's-eye views) by converting viewpoints of the images captured by the right rear camera 101 and the left rear camera 102. In other words, each image captured by the right rear camera 101 and the left rear camera 102 is converted into a bird's eye view displayed as if looking down from the sky by changing the viewpoint with the road surface as a projection plane. In addition, about the method of converting each image into a bird's-eye view image, since it is a well-known method, description is abbreviate | omitted.

  In the present embodiment, for example, a case will be described in which an image behind the vehicle as illustrated in FIG. 3 is captured by the right rear camera 101 and the left rear camera 102. In the example shown in FIG. 3, a solid object 3a such as a pole and a white line 3b on the road surface indicating the parking position of the vehicle exist behind the host vehicle.

  When such an image of the rear of the vehicle shown in FIG. 3 is picked up and inputted by the right rear camera 101 and the left rear camera 102, the viewpoint conversion unit 103a performs viewpoint conversion of each image as shown in FIG. Create an overhead image. 4A is a diagram illustrating a bird's-eye view image after the viewpoint conversion of the image captured by the right rear camera 101, and FIG. 4B is a viewpoint conversion of the image captured by the left rear camera 102. It is a figure which shows the subsequent bird's-eye view image.

  The three-dimensional object detection unit 103b performs image processing on the two overhead images converted by the viewpoint conversion unit 103a in this way, and detects a three-dimensional object such as a three-dimensional structure or a person existing in the overhead image. Thus, a three-dimensional object existing behind the vehicle is detected. First, an edge extraction filter is applied to each bird's-eye view image converted by the viewpoint conversion unit 103a to extract an edge from each bird's-eye view image. Thereafter, in each pixel from which the edge is extracted, the continuity of the edge with the adjacent pixel is detected, and only the edge constituting the straight line is extracted. As a result, it is possible to remove noise such as dots that do not form a straight line from the overhead image.

  Further, an edge (horizontal edge) extending in the horizontal direction in the image is deleted from the overhead image from which the edge has been extracted. Thereby, as shown in FIGS. 5A and 5B, a bird's-eye view image including only vertical edges can be obtained.

  Next, two overhead images including only the respective vertical edges created based on the images captured by the right rear camera 101 and the left rear camera 102 created by the above-described processing are superimposed. That is, taking into account the installation positions and installation angles of the right rear camera 101 and the left rear camera 102 in the vehicle and the camera parameters specific to each camera, the respective positions in the real space on each overhead image match each other. The overhead image is superimposed.

  FIG. 6A is a diagram illustrating a specific example in a case where the overhead image including only two vertical edges illustrated in FIGS. 5A and 5B is superimposed. In this way, it is possible to obtain a bird's-eye view image that is superimposed so that the positions of the three-dimensional object 3a and the white line 3b included in each captured image match in real space.

  In the image after superimposition (superimposed image) shown in FIG. 6A, edge end points are detected, and a plurality of adjacent edge end points are grouped. For example, in FIG. 6A, there are two edge end point groups indicated by reference numerals 6a and 6b. Then, only a group in which edges extending from edge end points existing in each group have an angle of a predetermined value or more is extracted.

  For example, in the example shown in FIG. 6A, since the edge extending from the two edge end points existing in the group 6b extends substantially in parallel on the superimposed image, the group 6b is not subject to extraction. It becomes. As a result, the edge detected from the white line 3b on the road surface as the projection plane at the time of viewpoint conversion extends substantially in parallel on the superimposed image regardless of the imaging angle difference between the right rear camera 101 and the left rear camera 102. In consideration of the presence, the edge extracted from the paint on the road surface existing in the superimposed image can be excluded with high accuracy.

  On the other hand, the edge extending from the two edge end points existing in the group 6a extends at a predetermined angle or more on the superimposed image, and is therefore an extraction target. Accordingly, each edge detected from the three-dimensional object is considered to extend in a different direction on the superimposed image on which the overhead image is superimposed due to the difference in imaging angle between the right rear camera 101 and the left rear camera 102. Thus, it is possible to accurately extract an edge that is highly likely to be extracted from a three-dimensional object existing in the superimposed image.

  Furthermore, when a plurality of edges extend in different directions from any one edge end point existing in the group 6a including the edge extracted from the three-dimensional object, a plurality of edges extending from the edge end point The edge is determined to have a high possibility of being extracted from the same three-dimensional object. That is, the edge extending from the same edge end point extends in two different directions due to the difference in imaging angle between the right rear camera 101 and the left rear camera 102, but the end point on each superimposed image is Since they match, it can be determined that there is a high possibility that the edges are detected from the same object.

  Then, in the extracted group 6a, an angle change between the frames of the angles θ1 and θ2 formed by the edges extending in different directions from the same end point, that is, the angles θ1 and θ2 formed in FIG. 6B is detected. As a result, when the angles θ1 and / or θ2 formed by the respective edges change between frames, each edge is configured from the rear camera 101 and the left rear camera 102 on the overhead view image as the host vehicle travels. Since the viewing angle of the object to be changed changes, it is possible to determine that the object constituted by the edge whose angle is changed is the same three-dimensional object.

  On the other hand, when the angles θ1 and / or θ2 formed by the respective edges do not change between the frames, the angle at which the object that each edge constitutes from the rear camera 101 and the left rear camera 102 changes on the overhead image. Therefore, it can be determined that the object constituted by these edges is an edge detected from the road surface, such as paint on the road surface.

  For example, in the example shown in FIG. 6B, changes in the magnitudes of θ1 and θ2 are detected between frames, and each edge extending from the end point of each edge is detected from a three-dimensional object existing on the road. It can be determined that the edge is a sharp edge. As a result, when there is a three-dimensional object within the imaging range of the right rear camera 101 and the left rear camera 102, the appearance of the three-dimensional object from the respective cameras is different, and the superimposed image is highly accurate. 3, that is, a three-dimensional object existing on the road within the imaging range of the right rear camera 101 and the left rear camera 102 can be detected.

  FIG. 7 is a flowchart showing processing of the object detection apparatus 100 in the present embodiment. The process shown in FIG. 7 is executed by the control device 103 as a program that starts when the power of the object detection device 100 is turned on by turning on the ignition switch of the vehicle.

  In step S10, reading of images captured from each of the right rear camera 101 and the left rear camera 102 is started, and the process proceeds to step S20. In step S20, as described above, the viewpoint conversion unit 103a performs viewpoint conversion on the images captured by the right rear camera 101 and the left rear camera 102 to create an overhead image. Thereafter, the process proceeds to step S30, and the three-dimensional object detection unit 103b applies an edge extraction filter to each overhead view image subjected to viewpoint conversion, extracts an edge from each overhead image, and proceeds to step S40.

  In step S40, in each pixel from which an edge has been extracted, the continuity of the edge with an adjacent pixel is detected, and only the edges that form a straight line are extracted. Thereafter, the process proceeds to step S50, an edge (horizontal edge) extending in the horizontal direction in the image is deleted from the overhead image, and the process proceeds to step S60. In step S60, as described above, a superimposed image is created by superimposing a bird's-eye view image including only vertical edges created based on images captured by the right rear camera 101 and the left rear camera 102, respectively. Thereafter, the process proceeds to step S70.

  In step S70, as described above, edge end points are detected in the superimposed image, and a plurality of adjacent edge end points are grouped. Thereafter, the process proceeds to step S80, and it is determined whether or not the edge extending from the edge end point existing in each group has a predetermined angle or more. If it is determined that each edge does not have a predetermined angle or more, that is, is substantially parallel, the process proceeds to step S120 described later. On the other hand, if it is determined that each edge has a predetermined angle or more, the process proceeds to step S90.

  In step S90, for an arbitrary edge endpoint existing in each group, an edge extracted from the image captured by the right rear camera 101 from the edge endpoint and an edge extracted from the image captured by the left rear camera 102 It is determined whether or not each extends in a different direction. When it is determined that each edge does not extend in a different direction from any edge end point, the process proceeds to step S120 described later. On the other hand, when it is determined that each edge extends in a different direction from an arbitrary edge end point, the process proceeds to step S100.

  In step S100, it is determined whether or not the angle formed by an edge in which each edge extends in a different direction from an arbitrary edge end point changes between consecutively imaged frames. If it is determined that the angle formed by the edge does not change between frames, the process proceeds to step S120, where it is determined that the edge is paint on the road surface. On the other hand, when it is determined that the angle formed by the edge changes between frames, the process proceeds to step S110, and it is determined that the object constituted by the edge is a three-dimensional object.

  Then, it progresses to step S130 and it is judged whether the ignition switch of the own vehicle was turned off. If it is determined that the ignition switch of the host vehicle is not turned off, the process returns to step S20 and the process is repeated. On the other hand, if it is determined that it is turned off, the process is terminated.

According to the present embodiment described above, the following operational effects can be obtained.
(1) A viewpoint image is created by converting viewpoints of images captured by the right rear camera 101 and the left rear camera 102, and only a vertical edge is extracted from the overhead image, and a real space on each overhead image is extracted. Each overhead image was superimposed so that the same position might correspond. Then, by grouping edge end points that are close to each other in the bird's-eye view image, and an edge extending from the edge end point existing in each group has a predetermined angle, the edge is separated from a three-dimensional object. It was determined that the possibility of extraction was high. Accordingly, each edge detected from the three-dimensional object is considered to extend in different directions on the superimposed image on which the overhead image is superimposed due to the difference in imaging angle between the right rear camera 101 and the left rear camera 102. Thus, it is possible to accurately extract an edge that is highly likely to be extracted from a three-dimensional object existing in the superimposed image.

  For example, as in the conventional three-dimensional object detection method, two images are taken from different directions, the luminance value of one image is compared with the luminance value of the other image, and the positions where the respective luminance values are different. In the case where it is determined that there is a three-dimensional object, there is a possibility that erroneous detection may occur when the location where the light hits is different in each image. On the other hand, according to the above-described embodiment, instead of determining a three-dimensional object based on the luminance of the image, the three-dimensional object is detected based on the angle of the edge extracted in the superimposed image. It is possible to detect a three-dimensional object with high accuracy regardless of how it hits.

(2) Further, when edge edges that are close to each other in the bird's-eye view image are grouped, and the edges extending from the edge endpoints existing in each group are substantially parallel, the edges are painted on the road surface. It was determined that the edge was extracted from. As a result, the edge detected from the road surface as the projection plane at the time of viewpoint conversion extends substantially in parallel on the superimposed image regardless of the difference in imaging angle between the right rear camera 101 and the left rear camera 102. In consideration of this, paint on the road surface existing in the superimposed image, for example, an edge extracted from a white line can be excluded with high accuracy.

(3) As a result of grouping edge end points that are close to each other in the bird's-eye view image, when a plurality of edges extend in different directions from the edge end points existing in each group, those edges are from the same object. Judged to be an extracted edge. As a result, the edges extending from the same edge end point extend in two different directions due to the difference in imaging angle between the right rear camera 101 and the left rear camera 102, but the end points on the respective superimposed images Taking into account that they match, edges detected from the same three-dimensional object can be detected with high accuracy.

(4) When an angle formed by edges extending in different directions from the same end point is calculated and the angle formed between frames changes, an object constituted by the edge formed by the changed angle is a three-dimensional object. It was determined to be. As a result, when there is a three-dimensional object within the imaging range of the right rear camera 101 and the left rear camera 102, the appearance of the three-dimensional object from the respective cameras is different, and the superimposed image is highly accurate. A three-dimensional object existing in the can be detected.

(5) An edge that is not detected as a three-dimensional object in the superimposed image is determined to be an edge detected from paint on the road surface. As a result, in the superimposed image created based on the overhead view image obtained by changing the viewpoint of the respective images captured by the right rear camera 101 and the left rear camera 102 with the road surface as the projection plane, the paint on the road surface is a plane. Considering that the object can be handled, it is possible to accurately separate the solid object and the road surface paint existing in the superimposed image by determining that the object other than the three-dimensional object is the road surface paint.

-Modification-
The object detection apparatus according to the above-described embodiment can be modified as follows.
(1) In the embodiment described above, the right rear camera 101 and the left rear camera 102 are provided as imaging means, and processing for detecting a three-dimensional object existing behind the host vehicle based on images captured by these cameras. explained. However, the present invention is not limited to this. For example, a right front camera and a left front camera may be provided to detect a three-dimensional object existing in front of the vehicle, and a plurality of cameras for imaging the entire periphery of the vehicle are installed. Then, a three-dimensional object existing around the vehicle may be detected.

(2) In the above-described embodiment, an example has been described in which, when all the following conditions (A) to (C) are satisfied, it is determined that the object formed by the extracted edge is a three-dimensional object.
(A) An edge extending from an edge end point existing in each group has a predetermined angle or more.
(B) Each of the edges extending from an arbitrary edge end point existing in each group extends in a different direction.
(C) An angle formed by an edge in which each edge extends in a different direction from an arbitrary edge end point varies between consecutively imaged frames.
However, when any one or two of (A) to (C) is satisfied, it may be determined that the object formed by the extracted edge is a three-dimensional object.

(3) In the above-described embodiment, the example in which the object detection device is mounted on the vehicle has been described. However, the present invention is not limited to this, and the object detection device may be mounted on another moving body.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

  The correspondence between the constituent elements of the claims and the embodiment will be described. The three-dimensional object detection unit 103b corresponds to an image superimposing unit, a grouping unit, and an angle change detection unit. The above description is merely an example, and when interpreting the invention, there is no limitation or restriction on the correspondence between the items described in the above embodiment and the items described in the claims.

It is a block diagram which shows the structure of one Embodiment of the object detection apparatus in this Embodiment. It is a figure which shows the example of installation in the vehicle of the right rear camera 101 and the left rear camera 102. FIG. It is a figure which shows the specific example of the object which exists in the vehicle back. It is a figure which shows the specific example of the bird's-eye view image produced by changing the viewpoint of the image imaged with each of the right rear camera and the left rear camera. It is a figure which shows the specific example of the bird's-eye view image which extracted only the vertical edge. It is a figure which shows the specific example of a superimposed image. It is a flowchart figure which shows the process of the object detection apparatus 100 in this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Object detection apparatus 101 Right rear camera 102 Left rear camera 103 Control apparatus 103a View point conversion part 103b Three-dimensional object detection part

Claims (4)

Viewpoint conversion means for creating a plurality of overhead images having a road surface as a projection plane by converting viewpoints of images around the vehicle imaged from different angles by a plurality of imaging means;
Edge extraction means for extracting edges from a plurality of overhead images created by the viewpoint conversion means;
Image superimposing means for superimposing a plurality of overhead images obtained by extracting edges by the edge extracting means;
Grouping means for detecting edge endpoints existing in the image superimposed by the image superimposing means, and grouping edge endpoints whose existence positions are close to each other;
A solid object detecting means for determining that an edge having an angle between edges extending from edge end points existing in each group grouped by the grouping means is equal to or larger than a predetermined value is an edge detected from the solid object. An object detection device characterized by. The object detection apparatus according to claim 1,
In the case where a plurality of edges extend in different directions from one edge end point existing in an arbitrary group grouped by the grouping unit, the three-dimensional object detection unit has an edge extending from the edge end point. An object detection apparatus for determining that an edge is extracted from the same three-dimensional object. The object detection device according to claim 2,
The three-dimensional object detection means includes a plurality of edges extending in different directions from the one edge end point between frames when a plurality of edges extend in different directions from one edge end point existing in the group. Including an angle change detecting means for detecting a change in angle between,
When a change in angle between edges is detected between frames by the angle change detection means, it is determined that a plurality of edges extending in different directions from one edge end point are edges extracted from the same three-dimensional object. An object detection apparatus characterized by: A plurality of bird's-eye images are created by converting the viewpoint of each of the images around the vehicle imaged from different angles by a plurality of imaging means and using the road surface as a projection plane,
Edges are extracted from the created overhead images,
Superimpose multiple bird's-eye images extracted edges,
Detect the edge endpoints that exist in the superimposed image, group the edge endpoints that are close to each other,
An object detection method, wherein an edge having an angle between edges extending from edge end points existing in each grouped group is determined to be an edge detected from a three-dimensional object.

Images