JP2018163530A - Object detection device, object detection method, and object detection program - Google Patents

Abstract

An object detection device, a detection method, and a detection program capable of stably detecting an object even when external light is reflected on the object to be detected. In a vehicle detection unit 1001 of an external environment recognition device, a matching image generation unit 101 that generates a plurality of matching images from a target image, and an area having a higher luminance than the surroundings in each matching image And a strong contrast region determination unit 102 for determining whether or not a strong contrast region exists, and when it is determined that a strong contrast region exists, the brightness is corrected so that the contrast is reduced with respect to the strong contrast region. Then, based on the processing result of the matching process, the strong contrast correction unit 106 that creates the corrected matching image, the vehicle detection unit 103 that performs the matching process for determining the presence of the vehicle using the corrected matching image, And a vehicle position determining unit 108 for specifying the position of the vehicle in the image. [Selection] Figure 2

Description

  The present invention relates to an object detection device that detects an object in an image.

  In recent years, in order to avoid accidents such as collisions between vehicles and collisions between people and vehicles, when shooting a video of the situation around the vehicle with an in-vehicle camera and detecting dangers such as sudden approach of other vehicles or people In addition to outputting warnings to drivers, technologies to automatically control vehicle behavior, such as applying automatic braking, have been developed. In particular, in order to perform preventive safety and automatic driving control of a vehicle, it is necessary to constantly monitor the approach of surrounding vehicles.

  Various methods have been proposed as a vehicle detection method based on image analysis, and a plurality of vehicle shape features are abstractly expressed (vehicle feature model) and stored in advance, Image matching that checks whether there is a location that matches the abstracted vehicle shape feature, or whether the abstracted shape of any range in the image matches the abstracted vehicle shape The technique is widely known.

  By the way, when strong external light such as sunlight or street light is present in the surrounding area, it is reflected on the vehicle to be detected and a part of the vehicle is overexposed (a region having a strong contrast to the surroundings: both a strong contrast region) May occur). In such a case, if the image matching method is used, the image portion of the vehicle in which the overexposure occurs does not match the vehicle shape feature stored in advance, so that the vehicle exists even though the vehicle exists. There was a problem that was undetected.

  As a technique for this problem, Patent Document 1 discloses a light source condition (use of illumination such as a strobe) in the step of storing vehicle shape template data indicating a feature amount of a contour line indicating a part of a vehicle shape to be detected. A method for coping with unexpected strong contrast regions by adding template data created by intentionally changing the presence / absence of light, the state of sunlight, forward light, backlight, etc.) .

JP 2003-242595 A

  In the technique disclosed in Patent Literature 1 described above, a state in which external light is intentionally reflected on the vehicle is intentionally created and added to the template data to deal with whiteout of the image of the detected vehicle. . However, depending on the direction of the detection vehicle, the situation of surrounding buildings (existence of secondary reflection of external light), the time zone, etc., there are an infinite number of reflection patterns due to the influence of sunlight and streetlights. It is extremely difficult to use feature expression as template data. Even if some of the reflection patterns are made into templates, the reflection patterns exist infinitely, so the probability of matching with the actual state of the detected vehicle is low, and it is difficult to perform stable vehicle detection. The same situation occurs not only when the object to be detected is a vehicle, but also when other objects are detected.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to stably detect an object even when external light is reflected on the object to be detected. To provide technology.

  In order to achieve the above object, an object detection apparatus according to one aspect is an object detection apparatus that detects an object from a predetermined object image, and a matching process for determining the presence of the object from the object image A matching image creation unit that creates a plurality of matching images for use in the image processing, and whether or not there is a strong contrast region that has a higher brightness than the surroundings in the matching image for each of the plurality of matching images A strong contrast region determination unit that determines whether or not a strong contrast region exists in the matching image, and corrects the luminance so that the contrast is reduced with respect to the strong contrast region in the matching image. The presence of an object using a strong contrast correction unit that creates a corrected matching image and a matching image or a corrected matching image It includes a matching processing section that performs a matching process for determining, based on the processing result of the matching process, and the object position specifying unit for specifying a position of the object in the target image.

  According to the present invention, it is possible to stably detect an object even when external light is reflected on the object to be detected.

FIG. 1 is an overall configuration diagram of an in-vehicle system including an external environment recognition device. FIG. 2 is a functional configuration diagram of the vehicle detection unit of the external recognition apparatus. FIG. 3 is a diagram for explaining a fisheye camera image and distortion correction of the image. FIG. 4 is a diagram illustrating an example of generating a matching image. FIG. 5 is a diagram for explaining a matching process using a matching image. FIG. 6 is a diagram for explaining processing for detecting a strong contrast region. FIG. 7 is a diagram for explaining the correction of the contrast in the strong contrast region. FIG. 8 is a diagram showing a specific example of a method for correcting the contrast of the strong contrast region. FIG. 9 is a diagram illustrating a determination process for determining whether or not to correct a strong contrast region. FIG. 10 is a diagram for explaining the tracking of the position of the strong contrast region. FIG. 11 is a flowchart of a vehicle detection process performed by the external environment recognition apparatus.

  Embodiments will be described with reference to the drawings. The embodiments described below do not limit the invention according to the claims, and all the elements and combinations described in the embodiments are essential for the solution of the invention. Is not limited.

  FIG. 1 is an overall configuration diagram of an in-vehicle system including an external environment recognition device.

  The in-vehicle system 10 includes an external environment recognition device 1000 as an example of an object detection device, an in-vehicle camera 1010, a control unit 1011, a memory 1012, and a host vehicle control unit 1013. The external environment recognition apparatus 1000, the in-vehicle camera 1010, the control unit 1011, the memory 1012, and the host vehicle control unit 1013 are connected via a bus 1018, for example.

  The in-vehicle camera 1010 captures an image (for example, a moving image) around a vehicle (own vehicle) on which the in-vehicle system 10 is mounted. The in-vehicle camera 1010 is installed at an arbitrary location of the host vehicle so that the surrounding vehicle can be easily recognized. Specifically, the in-vehicle camera 1010 may be installed on a front bumper, a rear bumper, left and right side mirrors, etc. outside the vehicle. One on-vehicle camera 1010 may be installed for the purpose of recognizing a vehicle only in a specific area, or a plurality of in-vehicle cameras 1010 may be installed so that the entire periphery of the host vehicle can be recognized. Moreover, you may make it install the vehicle-mounted camera 1010 in the vehicle of the own vehicle. The in-vehicle camera 1010 is, for example, a fisheye camera, and captures surrounding images as fisheye camera images. Here, if the in-vehicle camera 1010 is a fisheye camera, the shooting range can be widened, and a wide range can be covered with a small number of images. Therefore, in the vehicle detection process, the number of images to be processed can be reduced. And the processing load of the vehicle detection process can be reduced.

  The memory 1012 temporarily holds an image taken by the in-vehicle camera 1010. The control unit 1011 inputs and outputs images between the in-vehicle camera 1010 and the external recognition device 1000, and various information (such as recognition result information and warning information described later) between the external recognition device 1000 and the host vehicle control unit 1013. Control input and output. The own vehicle control unit 1013 performs overall control of the entire vehicle. For example, when the vehicle control unit 1013 receives recognition result information, warning information, or the like from the external environment recognition device 1000, the vehicle control unit 1013 performs vehicle control (braking control or the like) based on the information.

  The external environment recognition apparatus 1000 includes a vehicle detection unit 1001, a surrounding state recognition unit 1004, an alarm processing unit 1008, and a driver notification unit 1009. In this embodiment, the external environment recognition apparatus 1000 is a computer including a processor and a memory, and each functional unit is stored in a memory included in the external environment recognition apparatus 1000 by a processor included in the external environment recognition apparatus 1000. It is configured by executing a program.

  The vehicle detection unit 1001 inputs an image captured by the in-vehicle camera 1010 and executes a vehicle detection process for detecting the vehicle. The vehicle detection process will be described later. Based on the processing result of the vehicle detection process, the vehicle detection unit 1001 detects a vehicle approach signal indicating the risk of approaching or colliding with the vehicle, vehicle position information indicating the position of the detected vehicle, or failure in vehicle detection. The detected FAIL signal is output to the peripheral state recognition unit 1004.

  The surrounding state recognition unit 1004 includes a surrounding recognition unit 1005 that executes a recognition process for recognizing a state around the host vehicle (including a state of the vehicle around the host vehicle). As the recognition process executed by the periphery recognition unit 1005, for example, an analysis process of an image taken by the in-vehicle camera 1010 is performed, and other vehicles and pedestrians including motorbikes and bicycles in the vicinity of the host vehicle and in a distant peripheral space. Detecting and recognizing the presence or absence of obstacles that hinder the running or parking of the host vehicle, and detecting the sudden approach of other vehicles and pedestrians including motorcycles and bicycles in the vicinity and far away of the host vehicle There are a process for predicting a collision with the host vehicle and a process for predicting a collision between the host vehicle and an obstacle. In this case, the peripheral recognition unit 1005 outputs information on the recognition result (recognition result information) to the alarm processing unit 1008. The recognition processing by the periphery recognition unit 1005 includes lane departure warning processing for outputting warning information indicating that the host vehicle has deviated from the lane during traveling to the warning processing unit 1008, and a blind spot from the driver of the host vehicle. There is a blind spot alarm process that outputs warning information indicating that another vehicle has entered the alarm processing unit 1008.

  In addition, the periphery recognition unit 1005 is accurate when the detection FAIL signal is input from the vehicle detection unit 1001 and it is difficult to detect the vehicle because the image input from the in-vehicle camera 1010 is dark. Since it is difficult to perform image recognition processing and misrecognition is likely to occur, control is performed to temporarily or continuously stop its own operation and the operation of the vehicle detection unit 1001. At this time, the periphery recognition unit 1005 outputs notification information (dirt notification information) indicating that the periphery recognition unit 1005 is in an operation stop state to the driver notification unit 1009.

  When the warning processing unit 1008 receives the recognition result information and the warning signal from the periphery recognition unit 1005, the warning processing unit 1008 transmits the received recognition result information and the warning signal to the own vehicle control unit 1013. Thereby, the own vehicle control part 1013 can perform suitable control based on recognition result information or a warning signal.

  The driver notification unit 1009 is connected to, for example, the LED 1014, the speaker 1015, the display 1016, the carna navigation system (car navigation) 1017, and the like. When the notification information to be notified to the driver is received from the periphery recognition unit 1005, the driver notification unit 1009 A process of notifying the driver of the notification information is performed by turning on the light, causing the speaker 1015 to output sound, causing the display 1016 to perform display output, and causing the carna navigation system (car navigation) 1017 to perform display output. Thereby, notification information can be appropriately notified to the driver.

  Next, the functional configuration of the vehicle detection unit 1001 of the external environment recognition apparatus 1000 will be described.

  FIG. 2 is a functional configuration diagram of the vehicle detection unit of the external recognition apparatus.

  The vehicle detection unit 1001 includes an image distortion correction unit 100, a matching image creation unit 101, a strong contrast region determination unit 102, a vehicle detection unit 103 as an example of a matching processing unit, and a vehicle as an example of an object tracking unit. A tracking unit 104, a vehicle position integration unit 105, a strong contrast correction unit 106, a strong contrast region tracking unit 107, and a vehicle determination unit 108 as an example of a determination determination unit and an approach determination unit are provided. Here, the vehicle tracking unit 104, the vehicle position integration unit 105, the strong contrast region tracking unit 107, and the vehicle determination unit 108 are examples of the object position specifying unit.

  The image distortion correction unit 100 periodically inputs an image of one frame of a moving image taken by the in-vehicle camera 1010. Here, in the present embodiment, the frame image is a fisheye camera image. The image distortion correction unit 100 converts an image of a predetermined area of the fisheye camera image into an image without distortion (distortion correction image: target image) and passes the image to the matching image creation unit 101.

  The matching image creation unit 101 sequentially creates a matching image that is smaller in size than this image and is used for matching for detecting a vehicle, from the image (distortion correction image) received from the image distortion correction unit 100, The created matching image is sequentially transferred to the strong contrast region determination unit 102.

  The strong contrast region determination unit 102 is a state in which a matching image has a high contrast region (for example, the brightness is relatively higher than that of surrounding pixels due to the influence of strong external light such as sunlight or a streetlight, and a part of the image is whiteout) It is determined whether or not there is an area including the area (1). When there is a strong contrast region in the matching image, the strong contrast region determination unit 102 passes the matching image to the strong contrast correction unit 106. On the other hand, when there is no strong contrast region in the matching image, the strong contrast region determination unit 102 It passes to the detection unit 103.

  When the strong contrast correction unit 106 receives a matching image having a strong contrast region from the strong contrast region determination unit 102, the strong contrast correction unit 106 performs a shadowing process (contrast for reducing the contrast on the strong contrast region in the matching image). The image (corrected matching image) obtained by performing the shadowing process on the strong contrast region is passed to the vehicle detection unit 103, and the strong contrast region tracking unit 107 is provided with position information (for example, a strong contrast region). , Position information in the distortion-corrected image).

  The vehicle detection unit 103 performs a matching process between the matching image received from the strong contrast region determination unit 102 or the corrected matching image received from the strong contrast correction unit 106 and the vehicle feature model, and the matching image (corrected matching image). ) To execute a vehicle detection process for determining whether or not a vehicle candidate having a high possibility of a vehicle exists. In the present embodiment, the vehicle detection unit 103 may determine that the object in the matching image (or the corrected matching image) is a vehicle based on the degree of matching between the matching image (or the corrected matching image) and the vehicle feature model in the matching process. A score (evaluation value) indicating sex is specified, and whether or not a vehicle candidate exists is determined based on whether or not this score is equal to or greater than a predetermined threshold. In the present embodiment, the vehicle candidate detected by the vehicle detection unit 103 is determined to be a vehicle through the determination by the vehicle determination unit 108.

  The vehicle detection unit 103 notifies the vehicle tracking unit 104 of positional information of the vehicle candidate in the distortion corrected image according to whether or not the vehicle candidate is detected for each matching image (or corrected matching image) created from the same distortion correction image.

  The vehicle tracking unit 104 tracks and tracks the position of the same vehicle candidate over time based on the position information of the vehicle candidate in the distortion corrected image corresponding to each of the plurality of frame images notified from the vehicle detection unit 103. The result (vehicle candidate tracking result) is notified to the vehicle position integration unit 105.

  The strong contrast region tracking unit 107 tracks the position of the same strong contrast region over time based on the position of the strong contrast region in the distortion corrected image corresponding to each of a plurality of frames passed from the strong contrast correcting unit 106. The tracking result (strong contrast region tracking result) is notified to the vehicle position integration unit 105.

  The vehicle position integration unit 105 integrates the vehicle candidate tracking result of the vehicle tracking unit 104 and the strong contrast region tracking result of the strong contrast region tracking unit 107 so that the object in the matching image (or the corrected matching image) is the vehicle. Adjust the score to indicate that Specifically, when the tracking result of the vehicle tracking unit 104 and the tracking result of the strong contrast region tracking unit 107 are tracking results corresponding to the same object, the vehicle position integration unit 105 is a vehicle. A predetermined value is added to a score indicating a certain possibility.

  The vehicle determination unit 108 determines whether or not the score adjusted by the vehicle position integration unit 105 is equal to or greater than a predetermined threshold (a value larger than the threshold when the vehicle detection unit 13 determines a vehicle candidate). Is equal to or greater than the threshold value, it is determined that the vehicle candidate is a vehicle and the position of the vehicle in the distortion correction image is determined. In other cases, the vehicle candidate is determined not to be a vehicle. When the vehicle determination unit 108 determines that the vehicle candidate is a vehicle, the vehicle determination unit 108 calculates the distance between the vehicle and the host vehicle based on the position in the image of the vehicle, and the distance from the host vehicle is determined in advance. It is determined whether or not the distance is shorter than the set distance (caution / alarm distance). If the distance to the host vehicle is shorter than the attention / alarm distance, the vehicle position information indicating the position of the vehicle and the vehicle Is output to the surrounding state recognition unit 104. Further, the vehicle determination unit 108 detects that the detection result by the vehicle detection process for the continuously input frame images is short-term between detection (a vehicle was detected) and non-detection (a vehicle was not detected). When the state is switched between (for example, every frame), that is, in a state that cannot actually occur, it is determined that the reliability of the vehicle detection process has decreased, and the appropriate vehicle A detection FAIL signal indicating that the detection cannot be performed is output to the peripheral state recognition unit 104.

  Next, processing performed by the image distortion correction unit 100 will be described in detail.

  FIG. 3 is a diagram for explaining a fisheye camera image and distortion correction of the image. FIG. 3A shows a 1-frame fisheye camera image of a moving image taken by the in-vehicle camera 1010 installed on the right side mirror of the host vehicle. FIG. 3B shows an example of an image obtained by correcting the distortion of the fisheye camera image.

  In the one-frame fisheye camera image 200 of the moving image taken by the in-vehicle camera 1010 shown in FIG. 3A, the left side is the front (front direction) of the host vehicle, and the right side is the rear (rear direction) of the host vehicle. It becomes. The road surface 204 is shown in the center of the fisheye camera image 200, the background 205 such as the sky is shown in the upper part, the body 203 of the host vehicle is shown in the lower part, and the approaching vehicle 202 from the rear is shown on the right side. It is reflected. Here, in the present embodiment, for example, in the fisheye camera image 200, a region 201 in which a part of the rear of the host vehicle (for example, the right rear side with respect to the host vehicle) is shown is a vehicle detection region that detects the vehicle. 201 is set.

  The fish-eye camera image 200 has a wide imaging range, but the image is distorted as shown in the figure. In particular, the closer to the top, bottom, left and right edges of the fisheye camera image 200 (the farther the region is), the lower the resolution of the image than the depression lens image.

  Thus, since the image is distorted, the fisheye camera image 200 is not suitable for use in vehicle detection for performing matching processing. Therefore, in the present embodiment, the image distortion correction unit 100 corrects the image of the vehicle detection area 201 of the fisheye camera image 200 to an image without distortion (distortion correction image) 206 as shown in FIG. ing. Examples of a method for correcting distortion in a fisheye camera image include a nonlinear local geometric correction method that converts a distortion characteristic of a known fisheye lens into an actual spatial mapping, and a linear local geometric correction method that simply represents a distortion coefficient. However, other methods may be used.

  In this embodiment, the image distortion correction unit 100 is configured such that the approaching vehicle 202, the background 205, and the road surface 204 illustrated in FIG. 3A are similar to the approaching vehicle 209, the background 207, and the road surface 208 illustrated in FIG. That is, the correction is made so that it looks equivalent to the actual spatial mapping.

  Next, processing by the matching image creation unit 101 will be described in detail.

  FIG. 4 is a diagram illustrating an example of generating a matching image. FIG. 4A illustrates a method for generating a matching image from an input image. FIG. 4B is a diagram illustrating an outline of vehicle detection based on a matching image.

  The matching image is an image for comparison with a vehicle feature model stored in advance in a memory (not shown) of the external recognition device 100 in a matching process described later. In this embodiment, the input is received from the image distortion correction unit 100. The image is smaller than the image. The presence or absence of the vehicle can be detected by comparing the vehicle feature extracted from the matching image with the vehicle feature model and checking the degree of matching.

  As shown in FIG. 4A, the matching image creating unit 101 sets the range to be cut out as the matching image 301 for the input image 300 (corresponding to the distortion corrected image 206 in FIG. 3B) from the upper left to the lower right. A plurality of matching images are created by moving in a raster scan direction. More specifically, the matching image creating unit 101 sets a range where the matching image is cut out as a range of a predetermined size with respect to the origin at the upper left, and then a range based on a position slightly shifted in the horizontal direction. This is repeated until the cut-out range reaches the right end of the input image 300. When the cut-out range reaches the right end of the input image 300, the result is obtained by setting the position slightly shifted in the vertical direction and using the left end as a reference range. As shown by a line 302. As a result, the matching image can be cut out over the entire input image 300, and an area that is not a target of vehicle detection processing, which will be described later, can be prevented from occurring in the input image 300. Note that the method of creating a matching image from the input image 300 is not limited to this, and any method may be used as long as an uncut region does not occur in the input image 300.

  By creating the matching image in this way, as shown in FIG. 4B, the vehicle 305 is included in a plurality of matching images cut out from the input image 300 including the vehicle 305, the background 306, and the road surface 307. Can be included. For this reason, a vehicle can be detected in the matching image 304 including the vehicle 305 by executing the vehicle detection process on each matching image. Therefore, regardless of the position of the input image 300, the presence of the vehicle can be detected appropriately.

  Next, the process by the vehicle detection unit 103 will be described in detail.

  The vehicle detection unit 103 obtains a score indicating the degree of coincidence between the image feature obtained from the matching image created by the matching image creation unit 101 or the corrected matching image created by the strong contrast correction unit 106 and the vehicle feature model. If it is specified and the score is equal to or greater than a predetermined threshold, it is recognized that a vehicle (vehicle candidate) exists. When the score is less than the predetermined value, the vehicle detection unit 103 recognizes that there is no vehicle (vehicle candidate).

  As an image feature used for vehicle detection processing by the vehicle detection unit 103, for example, a HaarLike feature can be used. Note that the image feature used for vehicle detection may use an expression method other than the HaarLike feature.

  Here, the HaarLike feature is a method of expressing a feature by replacing the image state related to the periphery of a plurality of edges of an object with black and white binarized pattern data. It is known that the binarized pattern data around the edges of the vehicle image have similar numerical values if the vehicles have similar shapes, and this feature can be used for vehicle detection using a matching image. .

  FIG. 5 is a diagram for explaining a matching process using a matching image. FIG. 5A is an example of the HaarLike feature of the vehicle feature model stored in the memory. FIG. 5B is an example of the HaarLike feature when there is no region (high contrast region) where light is reflected in the vehicle of the matching image. FIG. 5C is an example of the HaarLike feature in the case where there is a region (high contrast region) where light is reflected on the vehicle of the matching image.

  The HaarLike feature may be set at an arbitrary position on the image edge. However, in this embodiment, as the HaarLike feature of the vehicle feature model, for example, as shown in FIG. It is assumed that the setting is made in three places, a part 402 and a right part 403. Here, for the vehicle feature model, the upper part of the upper part 401 is 1, the lower part is 0, the left part of the left part 402 is 1, the right part is 0, the left part of the right part 403 is 0, and the right part is 1. Suppose that

  For example, as shown in FIG. 5B, in the case where there is no region (high contrast region) where light is reflected on the vehicle in the matching image, the HaarLike feature is the upper part of the upper portion 405 of the vehicle 404. 1, the lower stage is 0, the left stage of the left part 406 is 1, the right stage is 0, the left part of the right part 407 is 0, and the right stage is 1. In this case, each value of the vehicle feature model shown in FIG. 5A matches, and the vehicle detection unit 103 specifies a high score indicating the degree of matching between the matching image and the vehicle feature model, It is determined that there is a vehicle candidate.

  On the other hand, as shown in FIG. 5C, in the case where a region (high contrast region) 412 where the light 413 is reflected exists in the vehicle of the matching image, the HaarLike feature is displayed on the upper portion 409 of the vehicle 408. The upper part is 1, the lower part is 0, the left part of the left part 410 is 1, the right part is 0, the left part of the right part 411 is 1, and the right part is 1. In this case, since the image feature of the right part 411 is different from the image feature of the vehicle feature model of FIG. 5A, the vehicle detection unit 103 generates a score indicating the degree of matching between the matching image and the vehicle feature model. Although it is specified low, it is erroneously determined that no vehicle candidate exists even though the vehicle exists in the matching image. In the present embodiment, when a region (strong contrast region) 412 where the light 413 is reflected exists in the vehicle of the matching image, an erroneous determination is made by correcting the strong contrast region of the matching image. So that it is reduced. This point will be described later.

  Next, processing performed by the strong contrast region determination unit 102 will be described in detail.

  FIG. 6 is a diagram for explaining processing for specifying a strong contrast region.

  Here, the strong contrast region determination unit 102 detects the strong contrast region when the matching image 500 in which the strong contrast region 502 exists in a part of the vehicle 501 is input as shown in FIG. Processing will be described.

  In the matching image 500, the strong contrast region determination unit 102 sets a value of 1 for a relatively bright pixel in the image (a pixel having a luminance equal to or higher than a predetermined threshold), and a relatively dark pixel (a predetermined threshold). For a pixel whose luminance is lower than the value), a binary image 503 as shown in FIG. In the binarized image 503 in FIG. 6B, the pixel having the value 1 is white and the pixel having the value 0 is black. In the strong contrast region determination unit 102, the threshold value to be used may be constant or may be changed for each image.

  Next, the strong contrast region determination unit 102 masks (pixel value is 0) so as not to refer to the peripheral portion 505 of the binarized image 503 in order to suppress the influence of textures other than the vehicle such as the background. To do.

  Next, the strong contrast region determination unit 102 performs labeling on each bright portion (a lump portion of pixels having a value of 1) in the binarized image 503, identifies the area of each bright portion, and the area of the bright portion is The value is changed to 0 for the bright part below the predetermined threshold value. Next, the strong contrast region determination unit 102 has a bright portion (506 or the like) in which pixels having a predetermined level of luminance (for example, luminance that is 95% or more of the entire luminance distribution) calculated from the pixel histogram in the matching image 500 exist. ) Only, and change the value of the other bright parts to 0. The strong contrast region determination unit 102 uses a region in which the bright portion 506 (for example, a rectangular region including the bright portion 506) in the binarized image 503 in FIG. The region is identified as a strong contrast region resulting from light reflection. By such processing, it is possible to appropriately specify a strong contrast region having a large influence in the matching processing.

  Note that the method of specifying the strong contrast region in the matching image is not limited to the above, and any method may be used as long as a relatively bright region can be specified. For example, an area that includes pixels that are higher than the average luminance of the matching image and that are equal to or higher than a predetermined threshold value may be specified as the high contrast area. In addition, an area including pixels whose luminance difference from neighboring pixels is a predetermined ratio or more may be specified as a high contrast area.

  Next, the processing of the strong contrast correction unit 106 will be described in detail.

  FIG. 7 is a diagram for explaining the outline of the contrast correction in the strong contrast region.

  Here, the processing for correcting the contrast when the matching image 600 in which the strong contrast region 602 exists in a part of the vehicle 601 as shown in FIG. 7A is input to the strong contrast correction unit 106 will be described. To do.

  The strong contrast correction unit 106 performs contrast levels (shadows) other than the strong contrast region 604 of the matching image 600 with respect to the strong contrast region 604 including the bright portion specified by the strong contrast region determination unit 102 illustrated in FIG. A process (shadowing process) for correcting the luminance of the strong contrast area 604 so as to be equivalent to the image level) is performed. As shown in FIG. 7C, matching is performed based on the corrected image 606 in this area. A correction matching image 605 is created by incorporating it in the image 600.

  Next, the shadowing process of the strong contrast correction unit 106 will be described in detail.

  FIG. 8 is a diagram showing a specific example of a method for correcting the contrast of the strong contrast region.

  Here, the shadowing process is a process for correcting the luminance of the image so that the high contrast region in the matching image is equivalent to the contrast level other than the strong contrast region.

  First, the strong contrast correction unit 106 cuts out an image of the strong contrast region 701 detected from the matching image 700 and temporarily stores it in a memory (not shown). Next, a luminance histogram 704 is obtained for regions other than the strong contrast region 701 of the matching image 702.

  Next, the strong contrast correction unit 106, for the obtained luminance histogram (value 0 to 255), the first pixel group 705 belonging to a predetermined ratio (A%) from the lower luminance (value 0) and the higher luminance Are divided into a second pixel group 707 belonging to a predetermined ratio (C%) and a third pixel group 706 having a predetermined ratio (B%) other than that, and the average of the pixels of the third pixel group 706 Find the brightness (reference average brightness).

  Next, the strong contrast region 701 corrects the luminance of each pixel so that the average luminance of each pixel of the image of the strong contrast region 701 is equal to the reference average luminance of the pixels of the third pixel group 706. A contrast-corrected image 708 is created (this process corresponds to a shadowing process). As a result, the contrast corrected image 708 is an image that is darker than the image of the strong contrast region 701.

  Next, the strong contrast correction unit 106 creates a corrected matching image 709 by fitting the contrast correction image 708 into the strong contrast region 701 of the matching image 700. Here, in the processing described above, the luminance of the dark portion (first pixel group 705) and the bright portion (second pixel group 707) of the luminance histogram of the image other than the strong contrast region 701 is set to the strong contrast of the matching image 700. The reason for not using the average brightness other than the area 710 is that there may be an extreme brightness value portion between the dark portion and the bright portion, and this affects the calculated average brightness value. This is because there is a possibility that the image of the strong contrast region 701 cannot be corrected to an image with appropriate luminance. Note that the average luminance for all pixels other than the strong contrast region 710 of the matching image 700 may be set as the reference average luminance.

  Next, another process by the strong contrast region determination unit 102 will be described in detail.

  FIG. 9 is a diagram illustrating a determination process for determining whether or not to correct a strong contrast region.

  When performing the shadowing process on the strong contrast region by the strong contrast correction unit 106, if the area of the strong contrast region is too large with respect to the area of the matching image 800, the area other than the strong contrast region is too small. A large deviation occurs in the average luminance, and the average luminance (shadowing level) as a reference for correcting the high contrast region is not an appropriate value.

  As a countermeasure against this, in the present embodiment, the strong contrast region determination unit 102 obtains the area of the strong contrast region 802 to be subjected to the shadowing process with respect to the matching image 800, and the matching image 800 and the strong contrast region 802 are obtained. With reference to the area ratio, it is determined whether or not to execute the shadowing process. Specifically, the strong contrast region determination unit 102 determines that the area of the strong contrast region 802 is equal to or smaller than a predetermined area ratio (for example, 1 / N of the area of the matching image 800: N is a number greater than 1). Then, the strong contrast correction unit 106 performs shadowing processing on the strong contrast region 802 to create a corrected matching image 804. As a result, the vehicle detection unit 103 uses the corrected matching image 804 to perform matching processing. On the other hand, when the area of the strong contrast region 802 exceeds a predetermined area ratio, the matching image 800 is transmitted to the vehicle detection unit 103, and the matching processing is performed using the matching image 800 in the vehicle detection unit 103. Let it run.

  Next, processing by the strong contrast region tracking unit 107 will be described in detail.

  FIG. 10 is a diagram for explaining the tracking of the position of the strong contrast region.

  In the vehicle detection process of the present embodiment, in addition to the vehicle position obtained by performing vehicle detection using a single input image, a vehicle tracking process that connects the vehicle positions in images corresponding to a plurality of consecutive frames in time series. As a result, the detection accuracy of the vehicle position is improved. In the present embodiment, the detection accuracy of the vehicle position is further stabilized by detecting the position of the strong contrast region. In addition, for the strong contrast region, as in the case of tracking the vehicle position, the tracking process for connecting the positions of the strong contrast region in time series is performed by using images corresponding to a plurality of continuous frames, so that the position of the strong contrast region is As a result, the detection accuracy of the vehicle position is improved.

  The strong contrast region tracking unit 107, when there is a strong contrast region (901, 903, 905) in each of the distortion corrected images (900, 902, 904) corresponding to a plurality of consecutive frames, Are obtained by connecting the corresponding strong contrast regions between the distortion corrected images corresponding to the respective frames, and storing the positions of the respective strong contrast regions. The vehicle position integration unit 105 determines the final vehicle position based on the tracking result obtained by the vehicle tracking unit 104 and the tracking result of the strong contrast region obtained by the strong contrast region tracking unit 107 ( Determine. For example, if the moving position of the vehicle indicated by the tracking result obtained by the vehicle tracking section 104 and the moving position of the strong contrast area indicated by the tracking result obtained by the strong contrast area tracking section 107 are substantially the same. It can be determined that the possibility that the vehicle candidate tracked by the vehicle tracking unit 104 is a vehicle is high and the reliability of the detected position is high.

  Next, the operation of the vehicle detection process will be described.

  FIG. 11 is a flowchart of a vehicle detection process performed by the external environment recognition apparatus.

  The external environment recognition apparatus 1000 determines whether or not the ignition of the vehicle is turned on (step S1100), and when the ignition is not turned on, that is, when the ignition is off (step S1100: NO). Waits until the ignition is turned on.

  On the other hand, when the ignition is turned on (step S1100: YES), the external recognition apparatus 1000 initializes itself (step S1101). That is, the external environment recognition apparatus 1000 starts input of an image from the in-vehicle camera 1010, sets initial values of parameters related to vehicle detection processing, and template data (vehicle feature model) necessary for matching processing for detecting the vehicle. ) Is read out. Thereafter, the external environment recognition apparatus 1000 repeatedly executes a process (steps S1102 to S1120) for actually detecting the vehicle.

  The image distortion correction unit 100 inputs the fisheye camera image captured by the in-vehicle camera 1010 (step S1102), extracts the vehicle detection area in the fisheye camera image, and corrects the distortion of the image in the vehicle detection area. The process is executed, and the processed image (distortion corrected image) is transferred to the matching image creating unit 101 (step S1103).

  Next, the matching image creating unit 101 creates a plurality of matching images based on the distortion corrected image input from the image distortion correcting unit 100 (step S1104).

  Next, the strong contrast region determination unit 102 determines whether or not a strong contrast region exists in each matching image (step S1105). If the strong contrast region does not exist in the matching image as a result of the determination (step S1106: NO), the strong contrast region determining unit 102 uses the matching image as the vehicle detection target image (target image). (Step S1111).

  On the other hand, when a strong contrast region exists in the matching image (step S1106: YES), the strong contrast region determination unit 102 determines whether or not the area of the strong contrast region is 1 / N or less of the area of the matching image. Determination is made (step S1107). As a result, when the area of the strong contrast region is not 1 / N or less of the area of the matching image, that is, when the area of the strong contrast region is larger than 1 / N of the area of the matching image (step A1107: NO), The strong contrast region determination unit 102 passes the matching image as a vehicle detection target image (target image) to the vehicle detection unit 103 (step S1111).

  On the other hand, when the area of the strong contrast area is 1 / N or less of the area of the matching image (step S1107: YES), the strong contrast area determination unit 102 uses the luminance histogram of the area other than the strong contrast area of the matching image. Based on this, the shadowing level in the shadowing process is set, the shadowing process is executed for the strong contrast area (step S1109), and the corrected matching image obtained by correcting the strong contrast area is set as the target image to the vehicle detection unit 103. Pass (step S1110).

  Next, the vehicle detection unit 103 executes vehicle detection processing for all target images (step S1112), and determines whether or not a vehicle candidate exists (step S1113).

  As a result, when there are no vehicle candidates in all target images (step S1113: NO), the vehicle detection unit 103 advances the process to step S1102.

  On the other hand, when there is a vehicle candidate (step S1113: YES), the vehicle tracking unit 104 is the same in the distortion-corrected image corresponding to a plurality of frames based on the position information of the vehicle candidate notified from the vehicle detection unit 103. The position of the vehicle candidate is tracked as time elapses, and the position of the vehicle candidate is notified to the vehicle position integration unit 105 as the vehicle position P1 (step S1114).

  Next, the vehicle tracking unit 104 determines whether or not a vehicle candidate is detected in the corrected matching image that has been subjected to the shadowing process in the high contrast region (step S1115), and no vehicle candidate is detected from the corrected matching image. In the case (step S1115: NO), the process proceeds to step S1117. On the other hand, when a vehicle candidate is detected from the corrected matching image (step S1115: YES), the process is based on the position of the strong contrast region of the corrected matching image. Thus, the position of the same strong contrast region in the distortion-corrected image corresponding to a plurality of frames is tracked as time elapses, and the vehicle position integration unit 105 is notified of the position of the strong contrast region as the vehicle position P2 (step S1116). . It should be noted that whether or not the strong contrast regions between the distortion-corrected images of a plurality of frames are the same is that the shape of the strong contrast region is similar or the ratio of the aspect ratio of the strong contrast region is substantially the same. Etc. can be determined.

  Next, the vehicle position integration unit 105 integrates the vehicle result (vehicle position P1) from the vehicle tracking unit 104 and the tracking result (vehicle position P2) from the strong contrast region tracking unit 107, thereby matching images (or The score indicating the possibility that the object in the corrected matching image) is a vehicle is adjusted. Specifically, when the vehicle position integration unit 105 can determine that the vehicle position P1 of the vehicle tracking unit 104 and the vehicle position P2 of the strong contrast region tracking unit 107 are the same position, the vehicle detection unit 103 causes the vehicle detection unit 103 to The score when detected with respect to the candidate is increased by a predetermined value, and the vehicle determination unit 108 is notified. When there is no vehicle position P2 or when the vehicle position P1 and the vehicle position P2 cannot be regarded as the same position, the vehicle position integration unit 105 detects the vehicle candidate by the vehicle detection unit 103. The score is notified to the vehicle determination unit 108 as it is. Next, the vehicle determination unit 108 determines whether or not the vehicle candidate object is a vehicle based on the score notified from the vehicle position integration unit 105 (step S1117). Specifically, the vehicle determination unit 108 determines whether or not the notified vehicle candidate score is equal to or higher than a predetermined threshold (a value higher than the threshold used by the vehicle detection unit 103), and the score is the threshold. If it is above, while a vehicle candidate is decided as a vehicle, if a score is less than a threshold value, it will be decided that it is not a vehicle.

  Next, the vehicle determination unit 108 calculates the distance between the host vehicle and the detected vehicle based on the determined position of the vehicle (detected vehicle) (step S1118), and the detected vehicle is approaching the host vehicle and It is determined whether or not the distance to the vehicle is equal to or less than a predetermined threshold (step S1119).

  As a result, when the detected vehicle is approaching the host vehicle and the distance from the host vehicle is equal to or smaller than the predetermined threshold (step S1119: YES), the vehicle determination unit 108 detects the position information of the detected vehicle, the vehicle A vehicle approach signal indicating proximity is output to the peripheral state recognition unit 1004, and the process proceeds to step S1121. On the other hand, when the detected vehicle is approaching the host vehicle and the distance from the host vehicle is not equal to or smaller than the predetermined threshold (step S1119: NO), the vehicle determination unit 108 advances the process to step S1121.

  In step S1121, the vehicle detection unit 1001 determines whether or not a situation in which the operation of the external world recognition apparatus 1000 ends due to an ignition OFF (off) or the like (step S1121), and the situation in which the operation of the apparatus ends. If not (step S1121: NO), the process proceeds to step S1102, while if the situation is that the operation of the apparatus is to be terminated (step S1121: YES), the vehicle detection process is terminated.

  According to the above embodiment, the strong contrast region determination unit 102 determines whether or not a strong contrast region exists in the matching image for each of the plurality of matching images, and the strong contrast correction unit 106 includes the matching image in the matching image. If it is determined that a strong contrast region exists, the luminance is corrected so that the contrast is reduced with respect to the strong contrast region in the matching image, and a corrected matching image is created. Since the matching process for determining the presence of the vehicle is performed using the image, and the vehicle tracking unit 104 identifies the position of the vehicle in the target image based on the processing result of the matching process, Even if a strong contrast region exists, the presence of the vehicle can be determined appropriately.

  Further, according to the embodiment, based on the change in the position of the vehicle candidate tracked by the vehicle tracking unit 104 and the change in the position of the strong contrast region tracked by the strong contrast region tracking unit 107, the vehicle determination unit 108. However, since it is determined whether or not the vehicle candidate is a vehicle, it is possible to appropriately determine that the vehicle in the strong contrast region is a vehicle. In other words, only the matching process performed by the vehicle detection unit 103 can appropriately determine a vehicle that is determined to have a low possibility of being a vehicle as a vehicle.

  In addition, this invention is not limited to the above-mentioned embodiment, In the range which does not deviate from the meaning of this invention, it can change suitably and can implement.

  For example, in the above-described embodiment, a fish-eye camera is used as the in-vehicle camera, but the present invention is not limited to this and may be a non-fish-eye camera.

  Moreover, in the said embodiment, based on the position (change of position) of the vehicle detected by matching, and the position (change of position) of a strong contrast area | region, the score of whether an object is a vehicle is changed, Whether or not the object is a vehicle is determined based on the score, but the present invention is not limited to this, and it is determined whether or not the object is a vehicle based only on the score obtained by matching. You may do it.

  Moreover, in the said embodiment, although the external environment recognition apparatus 1000 was mounted in the vehicle-mounted system 10 and the example utilized for the application for sensing the circumference | surroundings of a vehicle was shown, this invention is not limited to this, An external environment recognition apparatus May be mounted in another system other than the in-vehicle system and used for other purposes, for example, for use in detecting an object from an image of a surveillance camera.

  Moreover, in the said embodiment, although the example which made the target object to detect detect a vehicle, this invention is not limited to this, For example, objects other than vehicles, such as a person, a bicycle, a train, an aircraft, a ship, are targets. It is good.

  In the above embodiment, an example is shown in which each function unit of the external recognition device is configured by a processor executing a program. However, the present invention is not limited to this, and part or all of each function unit is configured. May be configured by hardware such as an integrated circuit. In the above embodiment, the program constituting the functional unit may be provided by a recording medium on which a program code is recorded. In this case, the functional unit can be realized by the computer processor reading and executing the program of the recording medium. As a storage medium for supplying the program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, SSD (Solid State Drive), optical disk, magneto-optical disk, CD-R, magnetic tape, non-volatile A memory card, ROM, or the like may be used. Further, the program constituting the functional unit is distributed via a network to be stored in a storage unit such as a hard disk or a memory of a computer or a storage medium such as a CD-RW or CD-R, and stored in a processor included in the computer. The program code stored in the storage unit or the storage medium may be read and executed.

DESCRIPTION OF SYMBOLS 10 ... In-vehicle system, 100 ... Image distortion correction part, 101 ... Matching image creation part, 102 ... Strong contrast area | region determination part, 103 ... Vehicle detection part, 104 ... Vehicle tracking part, 105 ... Vehicle position integration part, 106 ... Strong contrast Correction unit 107 ... Strong contrast area tracking unit 108 ... Vehicle determination unit 1000 1000 External recognition device 1001 Vehicle detection unit 1004 Peripheral state recognition unit 1008 Alarm processing unit 1009 Driver notification unit 1010 Car camera

Claims (15)

An object detection device for detecting an object from a predetermined object image,
A matching image creating unit that creates a plurality of matching images for use in matching processing for determining the presence of the object from the target image;
For each of the plurality of matching images, a strong contrast region determination unit that determines whether or not there is a strong contrast region including a region having a higher brightness than the surroundings in the matching image;
When it is determined that a strong contrast region exists in the matching image, a correction matching image is executed by correcting the luminance of the strong contrast region in the matching image so that the contrast is reduced. A strong contrast correction unit for creating
A matching processing unit that performs a matching process for determining the presence of the object using the matching image or the corrected matching image;
An object position specifying unit for specifying the position of the object in the target image based on the processing result of the matching process;
An object detection apparatus comprising: The object detection device according to claim 1, wherein the strong contrast correction unit reduces the contrast based on luminance of a pixel group in a region other than the strong contrast region of the matching image. The strong contrast correction unit is configured to select a pixel group belonging to a predetermined ratio from a higher luminance of the pixel to a pixel group in a region other than the strong contrast region of the matching image, and a predetermined luminance from a lower luminance of the pixel. And a reference average brightness that is an average brightness of a plurality of pixels excluding the pixel group belonging to the ratio of, and so that the average brightness of the strong contrast area becomes the reference average brightness. The object detection apparatus according to claim 2, wherein the contrast is reduced by adjusting the luminance. The matching processing unit
When the area of the strong contrast region is measured and the area is equal to or less than a predetermined ratio with respect to the matching image, the matching processing is performed using the corrected matching image corrected for the strong contrast region. The object detection according to any one of claims 1 to 3, wherein when the area exceeds a predetermined ratio with respect to the matching image, the matching processing is performed using the matching image. apparatus. The strong contrast region determination unit binarizes the luminance of each pixel in the matching image with a predetermined luminance as a reference, and includes a region including a block having a predetermined size or larger among the high-brightness pixel blocks. The object detection device according to any one of claims 1 to 4, wherein the object detection region is a high contrast region. The strong contrast region determination unit sets a region including a block having a predetermined size or larger and a block including pixels having a luminance of a predetermined ratio or more in the luminance distribution of pixels in the matching image as a strong contrast region. The object detection device according to claim 5. The target object detection apparatus according to claim 1, wherein an image corresponding to each of a plurality of frame images in a moving image is the target image. The object detection apparatus according to claim 7, further comprising an object tracking unit that tracks a change in position of the object obtained from the plurality of object images. A strong contrast region tracking unit that tracks a change in the position of the strong contrast region obtained from a plurality of the target images;
An object assumed by the object based on a change in the position of the object tracked by the object tracking unit and a change in the position of the strong contrast region tracked by the strong contrast region tracking unit; The object detection device according to claim 8, further comprising a confirmation determination unit that determines whether the object is an object. The object assumed by the object when the position of the object tracked by the object tracking unit is the same as the position of the strong contrast area tracked by the strong contrast region tracking unit. A vehicle position integration unit that adds a predetermined value to a score indicating the possibility of being, and notifies the confirmation determination unit,
The object detection device according to claim 9, wherein the determination determination unit determines whether or not the object is an assumed object based on the score notified from the vehicle position integration unit. The image distortion correction part which extracts the image of the predetermined area | region of the fisheye camera image imaged with the fisheye camera, and produces | generates the said target image by correct | amending the distortion of the said image is further provided of Claims 1-10. The target object detection apparatus as described in any one. The target object detection apparatus according to any one of claims 1 to 11, wherein the target object is a vehicle. Based on the position of the target object in the target image, the distance to the target object is calculated, and when the distance to the target object is less than or equal to a predetermined value, the distance to the target object is less than or equal to a predetermined value The object detection device according to claim 12, further comprising an approach determination unit that outputs information indicating that. An object detection method by an object detection device for detecting an object from a predetermined object image,
The object detection device includes:
Create a plurality of matching images for use in the matching process for determining the presence of the target object from the target image,
For each of the plurality of matching images, it is determined whether or not a strong contrast region including a region having a higher brightness than the surroundings exists in the matching image.
When it is determined that a strong contrast region exists in the matching image, the luminance is corrected so that the contrast is reduced with respect to the strong contrast region in the matching image, and a corrected matching image is created.
Using the matching image or the corrected matching image, performing a matching process for determining the presence of the object,
A target object detection method for specifying a position of the target object in the target image based on a processing result of the matching process. An object detection program for causing a computer that constitutes an object detection device that detects an object from a predetermined target image to execute the program,
A matching image creating unit for creating a plurality of matching images for use in the matching process for determining the presence of the object from the target image;
For each of the plurality of matching images, a strong contrast region determination unit that determines whether or not there is a strong contrast region including a region having a higher brightness than the surroundings in the matching image;
When it is determined that a strong contrast region exists in the matching image, the luminance is corrected so that the contrast is reduced with respect to the strong contrast region in the matching image, and a strong matching image is generated. A contrast correction unit;
A matching processing unit that performs a matching process for determining the presence of the object using the matching image or the corrected matching image;
An object position specifying unit for specifying the position of the object in the target image based on the processing result of the matching process;
Object detection program to function as