JP6464783B2 - Object detection device - Google Patents

Description

  The present invention relates to a technique for detecting a specific object from an image captured by a camera provided in a vehicle.

  In Patent Literature 1, in a vehicle periphery monitoring device that recognizes an object from a captured image by a camera mounted on the vehicle, recognition processing for the object is started based on a relative positional relationship between the host vehicle and the object. A technique for determining timing and an image processing range (recognition frame) for recognizing an object is described. With such a technique, it is said that recognition of an object can be started at an accurate timing and can be recognized with high accuracy.

JP 2005-347945 A

  According to the above-described prior art, although the timing and the recognition frame for starting the recognition process for the object can be accurately determined based on the relative positional relationship between the host vehicle and the object, the result of the recognition process is reliable. No consideration is given to assessing sex. Therefore, when there are a plurality of objects that appear to be objects in the recognition frame, there is a possibility that an object other than the object to be recognized is erroneously detected as the object.

  In general, the recognition frame is set to a large size with a margin in consideration of an installation error of the sensor mounting position and an error in which the posture of the sensor changes due to vibration. In addition, position acquisition means for acquiring in advance position information in the real space where the object is expected to exist (in the prior art, a map database storing the position information of the object in the real space and the position information of the object in the real space by GPS And a recognition means (a camera in the prior art) that actually recognizes the object at the current time is configured as a separate sensor. For this reason, the position information acquired by the position acquisition unit cannot be precisely associated with the recognition unit. Therefore, the recognition means must set a recognition frame larger than the actual image area of the target object, and the possibility that an object other than the target object is included in the recognition frame increases.

  For example, when a traffic signal to be followed by the host vehicle is assumed as an object, in addition to the traffic signal that is the object, a plurality of traffic signals including a traffic signal at another adjacent road or one intersection ahead is recognized. It is possible to detect within. In such a case, there is a possibility that erroneous vehicle control or information provision may be performed based on a recognition result for a traffic signal different from the traffic signal that the host vehicle should follow. Also, when an object with a pattern similar to the traffic signal as the target object appears in the recognition frame, such as the taillight or brake light of the preceding vehicle, an object that is not a traffic signal is erroneously detected as the target object. there is a possibility.

  The present invention has been made to solve the above problems. The objective of this invention is providing the technique for improving the detection performance by image recognition by evaluating the reliability of the detection result of the target object by image recognition.

  An object detection device according to the present invention is an object detection device that detects a specific target object from an image around a vehicle imaged by an imaging unit provided in the host vehicle. Means, evaluation means, and judgment means.

  The image recognition means performs target detection processing by image processing on the image picked up by the image pickup means, and detects candidate targets that are candidate objects from the image. The image position acquisition means acquires image position information relating to a relative position of the image portion corresponding to the candidate target detected by the image recognition means with respect to the host vehicle in the real space for the image captured by the imaging means. The evaluation means includes the position in the real space of the image portion corresponding to the candidate target represented by the image position information, the current position of the host vehicle and the position of the target object to be detected specified based on the given map data. Based on these comparisons, the reliability of the candidate target detected by the image recognition means is evaluated. The determining means determines whether or not the candidate target is actually an object in accordance with the reliability evaluation result by the evaluating means.

  ADVANTAGE OF THE INVENTION According to this invention, it can be judged exactly whether the detected candidate target is a target object which should be recognized on the basis of the relative position of the own vehicle and a target object. In other words, if the position in the real space of the candidate target measured from the captured image matches the position in the real space of the object described in the map data, the object detection result is assumed to be correct. Can do. On the other hand, if the position in the real space of the candidate target measured from the captured image is separated from the position in the real space of the target object based on the map data, it can be estimated that the candidate target is a non-target object. it can. In this way, even if a plurality of candidate targets that are similar to the target are detected from the same image, it is accurately determined whether or not the detected candidate target is the target to be recognized. And detection performance by image recognition can be improved.

The block diagram showing the structure of an object detection apparatus. Explanatory drawing which represents the information content of map data typically. The flowchart showing the procedure of an object detection process. Explanatory drawing showing the example of a setting of a recognition frame. Explanatory drawing showing the candidate target (traffic signal candidate) detected within the recognition frame.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is possible to implement in various aspects.
[Description of configuration of object detection apparatus]
The configuration of the object detection apparatus according to the embodiment will be described with reference to FIG. The object detection apparatus is mainly configured by a computer including a processor (not shown) that performs image processing, and has a function of performing image recognition processing for detecting a specific target object from images captured by the imaging unit 10. Have. This object detection apparatus is mounted on a vehicle and embodied as an in-vehicle system that is responsible for detecting object information used for various vehicle controls such as automatic driving.

  The imaging unit 10 is configured with an in-vehicle camera that images the front of the vehicle. The imaging unit 10 is embodied by a stereo camera configured by a pair of left and right cameras, for example. This stereo camera is arranged at a parallel equiposition to the traveling direction of the host vehicle, and images a common area (a front area of the host vehicle) at the same timing to obtain a stereo image composed of a pair of left and right images. . The imaging unit 10 is a sensor for acquiring an image used in image recognition processing by a candidate target detection unit 17 described later and image processing performed by an image distance estimation unit 18 described later. The image input unit 11 is an input interface that inputs image data representing an image captured by the imaging unit 10 to the candidate target detection unit 17.

  The position information input unit 12 has a function of inputting satellite positioning information by a GPS (Global positioning System) receiver or the like, and autonomous positioning information by a vehicle speed sensor, a gyro sensor, an acceleration sensor, or the like. The current position estimation unit 13 has a function of estimating the current position of the host vehicle based on information input from the position information input unit 12.

  The map database 14 is a storage device that stores map data in which information about road maps is recorded. The map data stored in the map database 14 includes information on each node corresponding to intersections, branch points, road shape points (position coordinates, etc.), information on road links connecting the nodes, and the like. . Further, the map data stored in the map database 14 includes position information related to the object detected by the object detection device of the present embodiment. In the present embodiment, an object presenting traffic information to be followed on a road is assumed as a target object. Specifically, the information contents to be presented such as traffic lights and electric bulletin boards change over time, road signs, lane markings, road markings, and the like.

  Here, the information content regarding the object of map data is demonstrated, referring FIG. As illustrated in FIG. 2, the map data includes nodes corresponding to intersections and road shape points and road links connecting the nodes as information constituting the roads.

  In addition, a road link connected to the node and an object presenting traffic information to be followed on the road represented by the link are defined in association with the node at the intersection. In the example of FIG. 2, it is assumed that traffic lights a and b, a stop line a and the like are defined as objects to be followed by a vehicle traveling on a link a connected to the node α at the intersection. And position information is defined about each object, such as a traffic light and a stop line. The position information of the target object is information indicating a relative position with respect to position coordinates (for example, latitude / longitude) of a reference point such as a node to which each target object is associated, and the size (width) of the target object. It is.

  For example, the direction perpendicular to the road extending direction (that is, the traveling direction of the vehicle) is the X axis, the direction coinciding with the road extending direction is the Y axis, and the height perpendicular to both the X axis and the Y axis. With the direction as the Z-axis, the position information regarding the traffic signal is defined as a deviation (ΔX, ΔY, ΔZ) with reference to the position coordinates of the corresponding node. In addition, the size (width) of the traffic signal is defined as W. Further, the position coordinates regarding the stop line are defined as deviations (ΔX, ΔY) with reference to the position coordinates of the corresponding node. In addition, the width of the stop line is defined as W.

  Returning to the description of FIG. The recognition timing determination unit 15 performs image recognition processing for detecting an object based on the current position estimated by the current position estimation unit 13 and the position information of the object described in the map data of the map database 14. Decide when to run. The recognition frame determination unit 16 is an image processing range for detecting an object from an image captured by the imaging unit 10 based on a positional relationship between the current position of the host vehicle and the object existing in the forward direction of the host vehicle. Determine the position and size of the (recognition frame).

  The candidate target detection unit 17 is well-known in the recognition frame range determined by the recognition frame determination unit 16 for the captured image input from the image input unit 11 at the timing determined by the recognition timing determination unit 15. Image recognition processing such as pattern matching is performed. By this image recognition processing, the candidate target detection unit 17 detects an image portion where a candidate target that is a candidate for the target exists from the captured image.

  The image distance estimation unit 18 estimates the relative position in real space between the candidate target detected by the candidate target detection unit 17 and the host vehicle from the image acquired by the imaging unit 10. Specifically, when the imaging unit 10 is configured by a stereo camera, the image distance estimation unit 18 uses a three-dimensional measurement technique such as a well-known stereo method to detect the distance between stereo images captured by the imaging unit 10. Get information. This distance information is associated with information representing the distance of the object in the real space in units of pixels constituting the image.

  Then, the image distance estimation unit 18 determines the relative position between the candidate target and the host vehicle based on the distance information acquired from the image and the position coordinates on the image of the candidate target detected by the candidate target detection unit 17. (Lateral direction deviation, depth direction deviation, height direction deviation) is estimated.

  The reliability evaluation unit 19 evaluates the reliability of the candidate target detected by the candidate target detection unit 17. The reliability representing the object likeness of the candidate target is determined based on the relative position of the candidate target estimated by the image distance estimation unit 18 and the vehicle and the object based on the position information of the object described in the map data. It is derived by comparing the relative position.

  The object determination unit 20 determines whether or not the candidate target is actually a target based on the evaluation result by the reliability evaluation unit 19. The vehicle control unit 21 recognizes traffic information indicated by the object determined by the object determination unit 20 (for example, a current signal color or a stop line), and a vehicle related to information presentation or travel safety based on the recognized traffic information. Execute control. Specifically, for example, the vehicle control unit 21 determines whether to enter or stop an intersection based on the current traffic signal display status according to the recognition result of the object, and based on the determination result, accelerates the vehicle.・ Automatically perform driving operations such as steering and braking, and present information to the driver.

[Description of object detection processing]
A procedure of processing executed by each unit included in the object detection device of the embodiment will be described with reference to a flowchart of FIG.

  In S <b> 100, the recognition timing determination unit 15 reads, from the map data database 14, surrounding map data based on the current position of the host vehicle estimated by the current position estimation unit 13. In S102, the recognition timing determination unit 15 specifies the link of the road corresponding to the current position of the host vehicle (that is, the road on which the host vehicle is currently traveling) based on the map data read in S101.

  In S104, the recognition timing determination unit 15 is within a specified distance ahead of the traveling direction of the road on which the vehicle is currently traveling based on the position information regarding the road and the object described in the map data read in S101. It is determined whether there is an object to be recognized. Here, based on the current vehicle position and the correspondence between the road and the object described in the map data, it is conceivable to narrow down the objects to be recognized only to the object to be followed at the present time. Alternatively, after all the objects are to be detected, a process for selecting an object to be followed at the present time after image recognition may be performed. The target object to be followed at this time is, for example, a traffic signal, a stop line, a road sign, or the like corresponding to the nearest intersection or road from the current position of the host vehicle.

  When there is no target object within the specified distance ahead in the traveling direction (S104: NO), the object detection apparatus ends this process. On the other hand, when the target is present within the specified distance ahead in the traveling direction (S104: YES), the object detection apparatus proceeds to S106. In S106, the recognition timing determination part 15 determines the timing which starts the image recognition process for detecting a target object. Specifically, based on the relative positional relationship between the object to be recognized determined in S104 and the host vehicle, the speed of the host vehicle, the road shape, and the planned course, for example, the target and the host vehicle The time when the distance is estimated to be a predetermined value is determined as the timing for starting the image recognition process.

  Here, the planned route is a route scheduled to travel in the future, such as a guide route set in the car navigation system or a planned travel route set in the automatic driving function of the vehicle. After the arrival of the timing determined in S106, the object detection apparatus starts image recognition processing for detecting the target object, and performs image recognition processing on image data input from the image input unit 11 at a predetermined cycle. Is continued for a predetermined period.

  At this time, in S108, the recognition frame determination unit 16 determines the position / size of a partial area (recognition frame) to be subjected to image recognition from the entire image. Specifically, the recognition frame determination unit 16 determines the relative position between the target object and the host vehicle based on the current position of the host vehicle at the time when the image is captured and the position information of the target object described in the map data. The relative positional relationship is calculated. The relative positional relationship between the object and the host vehicle is represented by a lateral deviation, a depth deviation, and a height deviation. Then, based on the relative positional relationship between the target object and the host vehicle, the speed of the host vehicle, the road shape, the planned route, etc., the position where the target object described in the map data is estimated to exist on the image is included. The range to be recognized is determined as a recognition frame.

  In determining the position and size of the recognition frame, the position information of the object described in the map data is information that can uniquely identify the lateral deviation, the depth direction deviation, and the height direction deviation with respect to the host vehicle. Ideally configured. However, if some of the location information of the object described in the map data is missing, use existing knowledge such as the regulations and specifications that will be the basis for the installation of the object, and The relative position may be estimated. The same applies to the evaluation of the reliability of the detected candidate target (described later).

  A specific example of the recognition frame determined in S108 is shown in FIG. In the example of FIG. 4, the traffic signal installed in the intersection nearest the front of the own vehicle is assumed as an object to be recognized. As illustrated in FIG. 4, the image shows a red signal (the latest red signal to be followed) indicated by a traffic signal installed at the nearest intersection in front of the host vehicle. In addition, a green light displayed by a traffic signal installed at an intersection ahead of the most recent intersection and the rear part of the preceding vehicle are shown.

  And the two recognition frames 1 and 2 (dashed line part) for recognizing the traffic signal installed in the nearest intersection ahead of the own vehicle are set with respect to this image. As in the case of FIG. 4, the recognition frame may be set over a plurality of locations on one image. Further, the shape of the recognition frame is not limited to a rectangle and may be various shapes.

  Returning to the flowchart of FIG. In S110, the candidate target detection unit 17 performs image recognition processing for detecting a candidate target that is a candidate for the target, on the image portion within the recognition frame set in S108. An example of the image recognition processing in S110 is shown in FIG. In addition, the example of FIG. 5 represents the result of the image recognition process with respect to the recognition frame illustrated in FIG. Here, the traffic signal installed in the intersection nearest the front of the own vehicle is assumed as an object to be recognized.

  As illustrated in FIG. 5, the latest red signal to be followed is detected as a candidate target from the recognition frame 1. On the other hand, from the recognition frame 2, as a candidate target, a green light at the previous intersection and a taillight of the preceding vehicle are detected. Among the plurality of candidate targets detected from the recognition frames 1 and 2, the latest red signal to be followed corresponds to the object to be recognized. On the other hand, the green light at the previous intersection and the taillight of the preceding vehicle are objects that do not correspond to the object to be recognized. These objects are assumed to have been detected by chance because an object whose characteristics on the image are similar to the object to be recognized is included in the recognition frame.

  Returning to the flowchart of FIG. In S112, the image distance estimation unit 18 acquires image position information representing a relative position in real space between each candidate target detected in S110 and the host vehicle from the image used in the image recognition process in S110. Specifically, the image distance estimation unit 18 acquires distance information in which information representing the distance of the candidate target in the real space is associated in units of pixels constituting the candidate target image. Then, based on the acquired distance information and the position coordinates on the image of the candidate target, the relative position between the candidate target and the host vehicle is calculated. The relative position between the candidate target and the host vehicle is represented by a lateral deviation, a depth direction deviation, and a height direction deviation based on the position of the host vehicle.

  In S114, the reliability evaluation unit 19 evaluates the reliability of each candidate target detected in S110. Here, the reliability evaluation unit 19 compares the relative position of the candidate target on the image calculated in S112 with the relative position of the object determined from the map data in S104 and the host vehicle. Calculate the deviation. And reliability evaluation is determined according to the magnitude | size of the calculated deviation. Specifically, the smaller the deviation between the relative position of the candidate target on the image and the relative position of the object described in the map data, the higher the reliability, and the larger the deviation, the lower the reliability. . Alternatively, the overall reliability is calculated by combining the pattern matching score indicating the degree of coincidence with the reference model and the reliability evaluation result when a candidate target is detected in the image recognition process. May be.

  In S116, the object discriminating unit 20 discriminates an object based on the reliability evaluation result in S114. Here, the object discriminating unit 20 identifies a candidate target having a reliability evaluation value equal to or higher than a specified value as an actual target object.

[Other devices applicable to the object detection apparatus of the embodiment]
Since the equipment related to the road is updated with time, the map data stored in the map database 14 may be appropriately updated based on the latest data acquired via an external computer network. Further, when the object detection device detects a target or an intersection not described in the map data stored in the map database 14, the information is provided to the data center that manages the map information, and the map information A configuration that prompts an update may be used. At that time, during automatic driving, the vehicle may be stopped so that it can be operated safely, such as by stopping at a safe place such as the shoulder of the host vehicle, or warning the driver.

[Modification]
In the above-described embodiment, the candidate target detection unit 17 performs image recognition processing using the image captured by the imaging unit 10, and in addition, the image distance estimation unit 18 performs candidate processing on the image by image processing. The example of calculating the relative position with the above has been described. In this example, an image recognition process and an image distance estimation process are performed using images acquired by the same sensor.

  Apart from this, the imaging unit 10 may be configured by a combination of a plurality of sensors such as a laser radar that measures the distance and direction of an object using a camera and laser light, for example. In this case, it is assumed that the distance information of the object acquired by the laser radar is associated with the coordinates of the image captured by the camera. When the imaging unit 10 is configured by a laser radar and a camera, the image distance estimation unit 18 is based on the correspondence between the object distance information acquired by the laser radar and the coordinates of the image captured by the camera. You may acquire the distance information of a captured image. In other words, this modification is configured to perform image recognition processing and image distance estimation processing using information acquired by a plurality of sensors.

[effect]
The object detection device according to the embodiment has the following effects.
It is possible to accurately determine whether or not the detected candidate target is an object to be recognized with reference to the relative position between the host vehicle and the object. That is, if the position in the real space of the candidate target measured from the image matches the position in the real space of the target object described in the map data, it can be estimated that the object detection result is correct. On the other hand, if the position in the real space of the candidate target measured from the captured image is separated from the position in the real space of the target object described in the map data, the candidate target is estimated to be a non-target object. be able to. In this way, even if a plurality of candidate targets that are similar to the target are detected from the same image, it is accurately determined whether or not the detected candidate target is the target to be recognized. And detection performance by image recognition can be improved.

  The object detection device of the embodiment is an object that presents traffic information to be followed on a running road, such as a traffic signal, an electric bulletin board, a road sign, a lane marking, and a road marking, and is used to evaluate reliability of image recognition. Based on these detections, these objects are detected. By doing in this way, the situation judgment regarding the intersection approach by automatic driving etc. can be performed exactly, for example.

  DESCRIPTION OF SYMBOLS 10 ... Imaging part, 11 ... Image input part, 12 ... Position information input part, 13 ... Current position estimation part, 14 ... Map database, 15 ... Recognition timing determination part, 16 ... Recognition frame determination part, 17 ... Candidate target detection 18, an image distance estimation unit, 19 a reliability evaluation unit, 20 an object determination unit, and 21 a vehicle control unit.

Claims (6)

An object detection device for detecting a specific object from images around a vehicle imaged by an imaging means (10) provided in the host vehicle,
Image recognition means (17) for performing target detection processing by image processing on the image picked up by the image pickup means and detecting candidate targets that are candidates for the target object from the images;
Image position acquisition means (18) for acquiring image position information related to the relative position of the image portion corresponding to the candidate target detected by the image recognition means in the real space with respect to the image captured by the image recognition means. When,
The position in the real space of the image portion corresponding to the candidate target represented by the image position information acquired by the image position acquisition means, the current position detected by the positioning means provided in the host vehicle, and the target The reliability of the candidate target detected by the image recognition means is evaluated based on the comparison with the position of the object to be detected that is specified based on given map data including the object position information regarding the position of the object. Evaluation means (19) for
Depending on the reliability of the evaluation result by the evaluation means, and a determining means (20) to the candidate target object actually determines whether a target object,
In the map data, a road and an object to be recognized on the road are defined in association with each other, and the object position information representing the position of each object is recorded.
The evaluation means is configured to identify the position of the object to be detected based on the object position information regarding the object associated with the road on which the host vehicle is traveling in the map data. That
Object detection apparatus according to claim. The object detection apparatus according to claim 1,
The image position acquisition unit performs image processing using an image captured by the same imaging unit as an imaging unit that acquires an image used for detection of a candidate target by the image recognition unit, thereby obtaining the image position information. To get the
An object detection device characterized by. In the object detection device according to claim 1 or 2,
The specific object is an object that presents traffic information to be followed on a road on which the host vehicle is traveling,
In the map data, a road and position information related to an object presenting traffic information to be followed on the road are stored in association with each other,
The evaluation means includes a position in an actual space of an image portion corresponding to the candidate target represented by the image position information, and a position of an object corresponding to a road on which the host vehicle is traveling in the map data. Evaluating the reliability of the candidate target detected by the image recognition means based on the comparison;
An object detection device characterized by. The object detection device according to claim 3,
The specific object is an object that presents traffic information to be followed on a road on which the host vehicle is traveling, and the information content to be presented changes with time.
An object detection device characterized by. The object detection device according to claim 4,
The specific object is a traffic signal;
An object detection device characterized by. In the object detection device according to any one of claims 1 to 5,
The image recognition means is configured to detect a plurality of the candidate targets from the image,
The evaluation means is configured to evaluate reliability for each of the detected candidate targets;
An object detection device characterized by.