JP2012141740A - Vehicle external world recognition device and vehicle system using the same - Google Patents

Abstract

An object of the present invention is to provide a vehicle external environment recognition device capable of detecting two similar types of objects with a small processing load, such as a pedestrian holding an umbrella during rainy weather and a pedestrian not having an umbrella.
A vehicle external environment recognition apparatus 1000 includes a first similarity MT0 of a part common to a first object and a second object from an image, and a first similarity which is a similarity of a specific part of the first object. The first object is calculated using the common similarity MT0 and the first characteristic similarity MT1 by calculating the characteristic similarity MT1 and the second characteristic similarity MT2 which is the similarity of the characteristic part of the second object. And the presence / absence of the second object are determined using the common similarity MT0 and the second specific similarity MT2, thereby reducing the processing load of the entire apparatus.
[Selection] Figure 1

Description

  The present invention relates to an external environment recognition device for a vehicle that detects an object such as a pedestrian around a host vehicle based on information from an image sensor such as a camera, and a vehicle system using the same.

  In order to reduce the number of casualties due to traffic accidents, the development of preventive safety systems that prevent accidents in advance is underway. In Japan, accidents that kill pedestrians account for about 30% of all traffic fatalities. In order to reduce such pedestrian accidents, preventive safety by detecting pedestrians in front of the vehicle The system is valid. A preventive safety system is a system that operates in a situation where there is a high possibility of an accident.For example, when there is a possibility of collision with an obstacle in front of the host vehicle, a warning is given to the driver to alert the driver. Pre-crash safety systems have been put into practical use that reduce the damage to passengers by automatic braking when inevitable situations occur.

  As a method for detecting a pedestrian in front of the host vehicle, a pattern matching method is used in which the front of the host vehicle is imaged with a camera and detected from the captured image using the shape pattern of the pedestrian. Pedestrians can consider various clothing and shape patterns, and a method has been proposed that changes the detection by estimating the shape of the pedestrian by measuring the environment of the vehicle.

  For example, Patent Document 1 describes a method of determining a traveling environment such as nighttime or rainy weather and changing a pedestrian template according to the environment.

  Patent Document 2 describes a method of changing a threshold value of a matching rate with a pedestrian template according to a determination result such as whether or not the vehicle is traveling on a motorway or day or night. .

  Furthermore, Patent Document 3 describes a method of determining whether or not the environment is easy to detect the shape of a pedestrian and determining a detection determination threshold based on the determination result.

JP 2009-122859 A JP 2010-0666815 A JP 2009-026233 A

  However, in the above method, there are pedestrians that cannot be detected depending on the environment. For example, in the method of switching templates described in Patent Document 1, assuming that it is raining, when switching from a normal pedestrian template to a pedestrian template holding an umbrella, walking without an umbrella when it rains There is a problem that the person cannot be detected.

  Even in the methods of recognizing the environment described in Patent Documents 2 and 3 and changing the detection threshold, if the detection threshold is lowered to reduce detection omission, there is a problem that false detection increases.

  In addition, a method of always executing all matching with a plurality of templates in order to detect various pedestrians at the same time is conceivable, but it is not realistic from the viewpoint of processing load.

  The above-mentioned problems are not limited to pedestrians, but occur for all objects that can be obstacles for vehicles.

  The present invention has been made in view of the above points, and an object of the present invention is for a vehicle that can detect a plurality of types of objects having some common shapes with a smaller processing load. An external recognition device and a vehicle system using the same are provided.

  An external environment recognition apparatus for a vehicle according to the present invention that solves the above problem calculates a common similarity that is a similarity between a common part of a first object and a second object from an image obtained by photographing the periphery of the vehicle, The first unique similarity that is the similarity of the unique part of the one object is calculated, the second unique similarity that is the similarity of the unique part of the second object is calculated from the image, and the calculated Determining the presence or absence of the first object using the common similarity and the first specific similarity, and determining the presence or absence of the second object using the calculated common similarity and the second specific similarity It is a feature.

  According to the present invention, the similarity between the parts common to the first object and the second object from the image, the first unique similarity that is the similarity between the unique parts of the first object, and the second Calculating the second unique similarity that is the similarity of the unique part of the object, determining the presence or absence of the first object using the common similarity and the first unique similarity, Since the presence or absence of the second object is determined using the second specific similarity, for example, the similarity of the entire first object is calculated to determine the presence or absence of the first object and the entire second object Compared with the case of calculating the degree of similarity and determining the presence or absence of the second object, the process of determining the degree of similarity is duplicated for the parts common to the first object and the second object. It is not necessary to perform this process, and it is only necessary to execute the process of judging the similarity once for such common parts. It is possible to reduce the entire apparatus processing load.

The block diagram of the external field recognition apparatus for vehicles in 1st embodiment. The schematic diagram which shows the example of the process in the process area | region setting part of FIG. The flowchart which shows the process from the common similarity calculation part of FIG. 1 to an object position calculation part. The figure which shows the weight of the Sobel filter used by the edge extraction of FIG. The schematic diagram of the local edge determination device used in the common similarity calculation part of FIG. The block diagram which shows the calculation method of the similarity by the common similarity calculation part of FIG. The block diagram which shows the calculation method of the similarity in the 1st specific similarity calculation part of FIG. 1, and the 2nd specific similarity calculation part. The block diagram of the external field recognition apparatus for vehicles in 2nd embodiment. The flowchart showing the process of the environment estimation part in 2nd embodiment. The flowchart showing the process of the environment estimation part in 2nd embodiment. The block diagram of the external field recognition apparatus for vehicles in 3rd embodiment. The flowchart showing the process of the environment estimation part in 3rd embodiment. The block diagram of the external field recognition apparatus for vehicles in 4th embodiment. The flowchart showing the process of the vehicle system using the external field recognition apparatus for vehicles of this invention. The schematic diagram which shows the example of calculation of the risk degree of the vehicle system using the external field recognition apparatus for vehicles of this invention.

<First embodiment>
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram of a vehicle external environment recognition apparatus 1000 according to the first embodiment.

  The vehicle external environment recognition device 1000 is incorporated into a camera device mounted on an automobile, an integrated controller, or the like, and a plurality of types in which some shapes are common to each other from images captured by the camera 1010 of the camera device. This embodiment is configured to detect a pedestrian who is not wearing an umbrella and a pedestrian who is holding an umbrella from an image obtained by imaging the front of the vehicle. Yes. A pedestrian who is not wearing an umbrella and a pedestrian who is holding an umbrella share the same shape of the lower half of the body.

  The vehicle external environment recognition device 1000 is configured by a computer having a CPU, a memory, an I / O, and the like, and a predetermined process is programmed and the process is repeatedly executed at a predetermined cycle.

  As shown in FIG. 1, the vehicle external environment recognition apparatus 1000 includes an image acquisition unit 1011, a processing region setting unit 1021, a common similarity calculation unit 1030, a first specific similarity calculation unit 1031, and a second It includes a specific similarity calculation unit 1032, a first object determination unit 1041, a second object determination unit 1042, and an object position calculation unit 1051, and further includes an object position acquisition unit 1111 depending on the embodiment. There is a case.

  The image acquisition unit 1011 captures data obtained by photographing the front of the host vehicle from a camera 1010 attached to a position where the front of the host vehicle can be imaged, and writes the data on the RAM as an image IMGSRC [x] [y]. Note that the image IMGSRC [x] [y] is a two-dimensional array, and x and y indicate the coordinates of the image, respectively.

  The processing area setting unit 1021 sets a processing area 23 (SX, SY, EX, EY) for detecting a pedestrian from the image IMGSRC [x] [y]. Details of the processing will be described later.

  Hereinafter, the common similarity calculation unit 1030, the first specific similarity calculation unit 1031, the second specific similarity calculation unit 1032, the first object determination unit 1041, and the second object determination unit 1042 detect pedestrians. Determination region 31 (see FIG. 5) (SXG [g], SYG [g], EXG [g], EYG [g]) so as to search within the processing region 23 (SX, SY, EX, EY) to be performed And a process of detecting a pedestrian who is not wearing an umbrella as the first object and a pedestrian who is holding an umbrella as the second object is repeatedly executed. Here, g is an ID number when a plurality of areas are set.

  The common similarity calculation unit 1030 calculates a similarity between a pedestrian not wearing an umbrella and a template TP0 of the lower half of the pedestrian, which is a common pattern of the pedestrian holding the umbrella. The common similarity calculation unit 1030 includes a template TP0 for the lower part of the pedestrian, and the determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g] of the image IMGSRC [x] [y]. ] Is calculated. Details of the processing will be described later.

  The first unique similarity calculation unit 1031 calculates the similarity with the template TP1 of the upper body of the pedestrian, which is a unique pattern of the pedestrian who is not wearing an umbrella. The first specific similarity calculation unit 1031 has a template TP1 of the upper body of the pedestrian, and the determination region 31 (SXG [g], SYG [g], EXG [g], Similarity MT1 with EYG [g]) is calculated. Details of the processing will be described later.

  The second unique similarity calculation unit 1032 calculates a similarity with the template TP2 of the upper body of the pedestrian including the umbrella, which is a unique pattern of the pedestrian wearing the umbrella. The second unique similarity calculation unit 1032 includes a template TP2, and the determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]) of the image IMGSRC [x] [y]. The similarity MT2 is calculated. Details of the processing will be described later.

The first object determination unit 1041 uses the common similarity MT0 with the template TP0 of the common part and the first unique similarity MT1 with the template TP1 peculiar to the pedestrian who does not wear the umbrella, to walk without the umbrella. The presence or absence of a person is determined. The determination is made, for example, when the condition that the value obtained by adding the common similarity MT0 and the first specific similarity MT1 is larger than the threshold TH1_MT using the threshold TH1_MT, that is,
(MT0 + MT1)> TH1_MT
If the common similarity MT0 is greater than the threshold TH1_MT0 and the first specific similarity MT1 is determined using the common similarity threshold TH1_MT0 and the first specific similarity threshold TH1_MT1. Is larger than the threshold value TH1_MT1, that is,
MT0> TH1_MT0 and MT1> TH1_MT1
If so, it may be determined that an object exists.

The second object determination unit 1042 uses the common similarity MT0 with the common part template TP0 and the second specific similarity MT2 with the template unique to the pedestrian wearing the umbrella, to thereby use the pedestrian holding the umbrella. The presence or absence of is determined. The determination is made, for example, when the condition that the value obtained by adding the common similarity MT0 and the second specific similarity MT2 is larger than the threshold TH2_MT using the threshold TH2_MT, that is,
(MT0 + MT2)> TH2_MT
If the common similarity MT0 is larger than the threshold TH2_MT0 and the second specific similarity MT2 is determined using the common similarity threshold TH2_MT0 and the second specific similarity threshold TH2_MT2. Is larger than the threshold value TH2_MT2, that is,
MT0> TH2_MT0 and MT2> TH2_MT2
If so, it may be determined that an object exists.

  The object position calculation unit 1051 calculates the position of the object detected by the first object determination unit 1041 and the second object determination unit 1042. When the object position acquisition unit 1111 is included, the object position information acquired from the object position acquisition unit 1111 is used as the object position. Details of the processing will be described later.

  The object position acquisition unit 1111 acquires a detection signal from a radar that detects an object around the own vehicle such as a millimeter wave radar or a laser radar mounted on the own vehicle, and the object position ( Relative distance PYR [b], lateral position PXR [b], and lateral width WDR [b]). Here, b is an ID number of each object when a plurality of objects are detected. The position information of these objects may be acquired by directly inputting a radar signal to the vehicle external environment recognition device 1000, or may be acquired by communicating with a radar using a LAN (Local Area Network). Also good. The object position detected by the object position acquisition unit 1111 is used by the processing region setting unit 1021.

[Processing area setting section]
Next, the contents of processing in the processing area setting unit 1021 will be described with reference to FIG.

  FIG. 2 is a diagram illustrating an example of processing performed by the processing region setting unit. The processing area setting unit 1021 selects a processing area 23 for performing pedestrian detection processing in the image IMGSRC [x] [y], and the start point SX and end point EX of the x coordinate (lateral direction), which are the range of the coordinates, A start point SY and an end point EY on the y coordinate (vertical direction) are obtained (SX, SY, EX, EY). The processing area setting unit 1021 may or may not use the position information from the object position acquisition unit 1111.

  First, a case where position information from the object position acquisition unit 1111 is used will be described. FIG. 2A is a diagram illustrating an example of processing performed by the processing region setting unit 1021 when position information from the object position acquisition unit 1111 is used.

  The processing region setting unit 1021 determines the position of the detected object 21 on the image from the relative distance PYR [b], the lateral position PXR [b], and the lateral width WDR [b] of the object 21 detected by the object position acquisition unit 1111. The object position area 22 shown (start point SXB and end point EXB of x coordinate (horizontal direction), start point SYB and end point EYB of y coordinate (vertical direction)) is calculated. A camera geometric parameter that associates the coordinates on the camera image with the positional relationship in the real world is calculated in advance by a method such as camera calibration, and the height of the object is assumed in advance, for example, 180 [cm]. Thus, the position on the image is uniquely determined.

  Further, for reasons such as mounting error of the camera 1010 or communication delay with the radar, the position on the image of the object 21 detected by the object position acquisition unit 1111 and the image of the same object 21 reflected in the camera image. There may be a difference in the position of. Therefore, a region obtained by correcting the object position region 22 (SXB, EXB, SYB, EYB) indicating the position of the object 21 on the image is calculated as a processing region 23 (SX, EX, SY, EY). The correction is performed by enlarging or moving the object position area 22 indicating the position of the object 21 on the image by a predetermined amount. For example, each point SXB, EXB, SYB, The position of EYB may be extended by a predetermined pixel vertically and horizontally. In this way, the processing area 23 (SX, EX, SY, EY) can be obtained. When processing a plurality of regions, processing regions 23 (SX, EX, SY, EY) are generated, and an object detection process described later is individually performed on each processing region 23.

  Next, processing for setting the processing region 23 (SX, EX, SY, EY) in the processing region setting unit 1021 without using the position information from the object position acquisition unit 1111 will be described. The method of setting the processing area 23 when the object position acquisition unit 1111 is not used is, for example, a method of setting a plurality of areas so as to search the entire image while changing the size of the area, a specific position, a specific position There is a method of setting an area limited to the size only. In the case of limiting to a specific position, for example, there is a method of limiting to a position where the host vehicle has advanced T seconds later by using the host vehicle speed.

  FIG. 2B is an example in the case of searching for a position where the host vehicle has advanced two seconds later using the host vehicle speed. The position and size of the processing area 23 are the road surface height (0 cm) at the relative distance to the position where the host vehicle is traveling in 2 seconds, and the assumed height of the pedestrian 21 (180 cm in the present embodiment). Thus, the range (SYP, EYP) in the y direction on the image IMGSRC [x] [y] is obtained using the camera geometric parameter. The range in the x direction (SXP, EXP) may not be limited, or may be limited by the predicted course of the vehicle. In this way, the processing area 23 (SX, EX, SY, EY) can be obtained.

[Similarity calculation unit]
Next, using FIGS. 3 and 4, the common similarity calculation unit 1030, the first specific similarity calculation unit 1031, the second specific similarity calculation unit 1032, the first object determination unit 1041, and the second object The flow of processing in the determination unit 1042 and the object position calculation unit 1051 will be described. FIG. 3 is a flowchart for explaining processing of the vehicle external environment recognition device.

  First, in step S301, an edge is extracted from the processing area 23 (SX, SY, EX, EY) of the image IMGSRC [x] [y]. Hereinafter, a method of calculating the edge image EDGE [x] [y] and the gradient direction image DIRC [x] [y] when the Sobel filter is applied as the differential filter will be described.

  As shown in FIG. 4, there are two types of Sobel filters: an x-direction filter 41 for obtaining a gradient in the x direction and a y-direction filter 42 for obtaining a gradient in the y direction. When obtaining the gradient in the x direction from the image IMGSRC [x] [y], for each pixel of the image IMGSRC [x] [y], a pixel value of a total of 9 pixels, that pixel and the surrounding 8 pixels, and the corresponding position A product-sum operation for the weight of the x-direction filter 41 is performed. The result of the product-sum operation is the gradient in the x direction at that pixel. The same applies to the calculation of the gradient in the y direction.

  If the calculation result of the gradient in the x direction at a position (x, y) of the image IMGSRC [x] [y] is dx, and the calculation result of the gradient in the y direction is dy, the gradient strength image DMAG [x] [y] The gradient direction image DIRC [x] [y] is calculated by the following equations (1) and (2).

DMAG [x] [y] = | dx | + | dy | (1)
DIRC [x] [y] = arctan (dy / dx) (2)

  The gradient strength image DMAG [x] [y] and the gradient direction image DIRC [x] [y] are two-dimensional arrays having the same size as the image IMGSRC [x] [y], and the gradient strength image DMAG The coordinates (x, y) of [x] [y] and the gradient direction image DIRC [x] [y] correspond to the coordinates (x, y) of the image IMGSRC [x] [y].

  The calculated gradient strength image DMAG [x] [y] is compared with the edge threshold value THR_EDGE, and the gradient strength image DMAG [x] [y] is larger than the edge threshold value THR_EDGE (DMAG [x] [y]> If it is THR_EDGE), 1 is stored in the edge image EDGE [x] [y], otherwise, that is, 0 if the gradient strength image DMAG [x] [y] is equal to or less than the edge threshold value THR_EDGE.

  The edge image EDGE [x] [y] is a two-dimensional array having the same size as the image IMGSRC [x] [y], and the coordinates (x, y) of the edge image EDGE [x] [y] are the image IMGSRC [ This corresponds to the coordinates (x, y) of x] [y].

  Prior to edge extraction, an image corresponding to the processing region 23 (SX, EX, SY, EY) is cut out from the image IMGSRC [x] [y] so that the size of the object in the image becomes a predetermined size. You may zoom in and out. In the present embodiment, the distance information and camera geometry used in the processing area setting unit 1021 are used, and all the objects having a height of 180 [cm] and a width of 60 [cm] in the image IMGSRC [x] [y] are all 16 pieces. The image is enlarged / reduced to a size of dot × 12 dots, and the edge is calculated.

  Further, the calculation of the edge image EDGE [x] [y] and the gradient direction image DIRC [x] [y] may be all zero outside the range of the processing region 23 (SX, EX, SY, EY).

  Next, in step S3011, determination areas 31 (SXG [g], SYG [g], pedestrian determination are performed in the edge image EDGE [x] [y] and the edge direction image DIRC [x] [y]. EXG [g], EYG [g]).

  As described in step S301, in the present embodiment, all the objects having a height of 180 [cm] and a width of 60 [cm] in the image IMGSRC [x] [y] are 16 dots × 12 dots in the image IMGSRC [x] [y]. The edge image EDGE [x] [y] is generated by enlarging / reducing the image so that the size becomes. Therefore, when the size of the determination area 31 is 16 dots × 12 dots and the processing area 23 (SX, EX, SY, EY) in the edge image EDGE [x] [y] is larger than 16 dots × 12 dots, A plurality of determination areas 31 are set so as to be spread at regular intervals in the edge image EDGE [x] [y].

  Next, with g = 0, all the determination areas 31 (SXG [g], SYG [g], EXG [g], EXG [g]) set by the processing area setting unit 1021 are hereinafter referred to from step S302. The process of S306 is executed.

  First, in step S302, the edge image EDGE [x] [y] and the gradient direction image DIRC [x] in the determination region 31 (SXG [g], SYG [g], EXG [g], EXG [g]). [Y] is used to calculate the common similarity, which is the similarity of the parts that are common to each other among the plurality of types of objects. Here, the common similarity calculation unit 1030 performs a process of obtaining the similarity between the pedestrian's lower body pattern, which is a common part between the pedestrian not wearing the umbrella and the pedestrian holding the umbrella. MT0 is calculated. Details of the process will be described later.

  Next, in step S303, the edge image EDGE [x] [y] and the gradient direction image DIRC [x] in the determination region 31 (SXG [g], SYG [g], EXG [g], EXG [g]). ] [Y] is used to calculate the first specific similarity that is the similarity to the pattern of the characteristic part unique to the object of the predetermined type of object. Here, in the first unique similarity calculation unit 1031, a process for obtaining the similarity with the pattern of the upper body of the pedestrian, which is a characteristic part unique to the pedestrian who does not wear the umbrella, A first specific similarity MT1 that is the similarity of the specific part is calculated. Details of the process will be described later.

  Next, in step S304, the edge image EDGE [x] [y] and the gradient direction image DIRC [x] in the determination region 31 (SXG [g], SYG [g], EXG [g], EXG [g]). ] [Y] is used to calculate the second specific similarity that is the similarity to the pattern of the characteristic part unique to the object of other types of objects. Here, the second unique similarity calculation unit 1032 performs a process for obtaining the similarity with the pattern of the shape of the upper body of the pedestrian, which is a characteristic part unique to the pedestrian wearing the umbrella, and walks holding the umbrella The second unique similarity MT2 that is the similarity of the unique part unique to the person is calculated. Details of the process will be described later.

  Next, in step S305, the processing of the first object determination unit 1041 is performed, and the coordinates (SXG [g], SYG [g]) of the first determination result storage array DTC1 [x] [y] are set. If it is determined that an object exists, “1” is stored, and if not, “0” is stored.

  Further, in step S306 as well, the processing of the second object determination unit 1042 is performed, and the coordinates (SXG [g], SYG [g]) of the second determination result storage array DTC2 [x] [y] are set. If it is determined that an object exists, “1” is stored, and if not, “0” is stored.

  The above processing is executed for all the determination areas 31 (SXG [g], SYG [g], EXG [g], EXG [g]), and finally, in step S307, the first determination result storage is performed. The result of the array DTC1 [x] [y] and the second determination result storage array DTC2 [x] [y] are integrated to calculate the object position. Details of the process will be described later.

[Similarity calculation unit]
Next, a processing flow in the common similarity calculation unit 1030 will be described with reference to FIGS. FIG. 5 is a schematic diagram of a local edge determiner used in the common similarity calculation unit of FIG. 1, and FIG. 6 is a block diagram showing a similarity calculation method by the common similarity calculation unit of FIG.

  In step S302 in which the processing of the common similarity calculation unit 1030 is performed, the discriminator 61 described in detail below for a certain determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]). The output obtained from the processing result is treated as the similarity.

  First, a method for calculating similarity using the classifier 61 will be described.

  As a method for calculating the degree of similarity with a predetermined pattern by image processing, a template matching method is used in which a plurality of templates representing patterns are prepared and the degree of coincidence is obtained by performing a difference accumulation operation or a normalized phase relationship operation. And a pattern recognition method using a classifier such as a neural network.

  Regardless of which method is selected, a source database is required as an index for determining in advance whether the pattern is a predetermined pattern or not. Various patterns are stored as a database, from which a representative template is created or a discriminator is generated. Since various states are expected in the pattern in the real environment, it is necessary to prepare a large amount of databases and reduce erroneous determinations. In this case, the former method based on template matching is not realistic because the number of templates becomes enormous if the omission of determination is prevented.

  Therefore, in this embodiment, a method of calculating the similarity using the latter discriminator is adopted. The size of the discriminator does not depend on the size of the source database. A database for generating a classifier is called teacher data.

  The discriminator 61 used in the present embodiment is based on a plurality of local edge discriminators, and the similarity of the lower body of the pedestrian, which is a common feature of the pedestrian who does not wear the umbrella and the pedestrian who holds the umbrella. Is calculated.

  First, the local edge determiner 51 will be described using the example of FIG. The local edge determiner 51 includes an edge image EDGE [x] [y], a gradient direction image DIRC [x] [y], and a matching determination region 31 (SXG [g], SYG [g], EXG [g], EYG [G]) as an input and outputs a binary value of 0 or 1, and includes a local edge frequency calculation unit 511 and a threshold processing unit 512.

  The local edge frequency calculation unit 511 includes a local edge frequency calculation region 5112 in a window 5111 having the same size as the matching determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]). The edge image EDGE [x] [y] and the gradient direction image DIRC [] from the positional relationship between the matching determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]) and the window 5111. The position for calculating the local edge frequency in x] [y] is set, and the local edge frequency MWC is calculated.

  The local edge frequency MWC is the total number of pixels in which the angle value of the gradient direction image DIRC [x] [y] satisfies the angle condition 5113 and the edge image EDGE [x] [y] at the corresponding position is 1. It is. In the example of FIG. 4, the angle condition 5113 is between 67.5 degrees and 112.5 degrees, or between 267.5 degrees and 292.5 degrees, and the gradient direction image DIRC [x] It is determined whether or not the value of [y] is within a certain range.

  The threshold processing unit 512 has a predetermined threshold THWC #, and outputs 1 if the local edge frequency MWC calculated by the local edge frequency calculation unit 511 is equal to or higher than the threshold THWC #, and outputs 0 otherwise. . The threshold processing unit 512 may output 1 if the local edge frequency MWC calculated by the local edge frequency calculation unit 511 is equal to or less than the threshold THWC #, and may output 0 otherwise.

  Next, the classifier will be described with reference to FIG. The discriminator 61 includes an edge image EDGE [x] [y], a gradient direction image DIRC [x] [y], and a determination area 31 (SXG [g], SYG [g], EXG [g], EYG [g]. ) As an input, a positive value is output if the determination area 31 is similar to the lower body of the pedestrian, and a negative value is output if it is not similar. The discriminator 61 includes 40 local edge frequency determiners 6101 to 6140 and a summation unit 612.

  The local edge frequency determiners 6101 to 6140 are the same as the local edge determiner 51 described above, but the local edge frequency calculation region 5112, the angle condition 5113, and the threshold value THWC # are different. . Summing unit 6120 multiplies the outputs from local edge frequency determiners 6101 to 6140 by corresponding weights WWC1 # to WWC40 #, and outputs the total value SC.

  The local edge frequency calculation region 5112, the angle condition 5113, the threshold value THWC, and the weights WWC1 # to WWC40 #, which are parameters of the local edge frequency determiners 6101 to 6140 of the classifier 61, are input to the classifier 61 by walking. The teacher data is set so that the total value SC output from the total unit 612 is a positive value when the person is the lower body of the pedestrian, and is a negative value when the lower body is not the pedestrian. To adjust. For the adjustment, for example, a machine learning means such as AdaBoost may be used, or manual adjustment may be performed.

  For example, the procedure for determining parameters using AdaBoost from NPD pedestrian lower body teacher data and NBG non-pedestrian teacher data is as follows. Hereinafter, the local edge frequency determiner is represented as cWC [m]. Here, m is the ID number of the local edge frequency determiner.

  First, a plurality of local edge frequency determiners cWC [m] having different local edge frequency calculation areas 5112 and angle conditions 5113 (for example, 1 million) are prepared, and the value of the local edge frequency MWC is obtained from all teacher data in each. The threshold value THWC is determined by calculation. The threshold THWC selects a value that can most classify the teacher data of the lower body of the pedestrian and the teacher data of the non-pedestrian.

  Next, a weight of wPD [nPD] = 1/2 NPD is given to each teacher data of the lower body of the pedestrian. Similarly, a weight of wBG [nBG] = 1 / 2NBG is given to each non-pedestrian teacher data. Here, nPD is the ID number of the teacher data of the lower body of the pedestrian, and nBG is the ID number of the teacher data of the non-pedestrian. Then, it is assumed that k = 1, and the following process is repeated.

First, the weights are normalized so that the sum of the weights of the teacher data of all the lower body and non-pedestrians of the pedestrian is 1. Next, the false detection rate cER [m] of each local edge frequency determiner is calculated. The false detection rate cER [m] is obtained when the local edge frequency determiner cWC [m] outputs 0 when the pedestrian's lower body teacher data is input to the local edge frequency determiner cWC [m]. Alternatively, when the non-pedestrian teacher data is input to the local edge frequency determiner cWC [m], the output is 1, that is, the sum of the weights of the teacher data whose output is incorrect.
After calculating the false detection rate cER [m] of all the local edge frequency determiners, the local edge frequency determiner ID Mmin that minimizes the false detection rate is selected, and the final local edge frequency determiner WC [k] = cWC. [MMin].

  Next, the weight of each teacher data is updated. The update is such that the result of applying the final local edge frequency determiner WC [k] among the teacher data of the lower body of the pedestrian becomes 1, and the final local edge frequency determiner of the non-pedestrian teacher data Multiply the coefficient BT [k] = cER [mMin] / (1-cER [mMin]) by the weight of the teacher data whose result of applying WC [k] is 0, that is, the output is correct.

  k = k + 1 is repeated until k reaches a preset value (for example, 40). The final local edge frequency determiner WC obtained after the end of the iterative process becomes the discriminator 61 that is automatically adjusted by AdaBoost. The weights WWC1 to WWC40 are calculated from 1 / BT [k].

  Next, the flow of processing in the first specific similarity calculation unit 1031 and the second specific similarity calculation unit 1032 will be described with reference to FIG. FIG. 7 is a block diagram illustrating a similarity calculation method in the first specific similarity calculation unit and the second specific similarity calculation unit in FIG. 1.

  In step S303 in which the first specific similarity calculation unit 1031 performs processing, the processing of the discriminator 611 is performed for a certain determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]). The output obtained as a result is treated as the first specific similarity MT1. The discriminator 611 is a discriminator designed in the same manner as the discriminator 61, but uses the upper body of a pedestrian not wearing an umbrella as teacher data.

  Similarly, in step S304 in which the second unique similarity calculation unit 1032 performs processing, a discriminator is determined for a certain determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g]). The output obtained as a result of the processing at 612 is handled as the second specific similarity MT2. The discriminator 612 is a discriminator designed in the same manner as the discriminator 61, but uses the upper body of a pedestrian wearing an umbrella as teacher data.

  Next, processing of the object position calculation unit 1051 will be described. The object position calculation unit 1051 groups locations where “1” is included in the determination result storage array DTC1 [x] [y], and determines that a pedestrian exists at the center position. A known technique such as labeling is used for grouping.

  If the height of the pedestrian is assumed to be 180 cm using the Y coordinate of the grouped area and the camera geometry, the distance to the pedestrian can be calculated. Also, using the X coordinate of the grouped area and the camera geometry, if the pedestrian width is assumed to be 60 cm, calculate how much pedestrians are present laterally (outside in the vehicle width direction) from the center of the vehicle. can do.

  When the object position acquisition unit 1111 is present, information is replaced when object information (PXR [b], PYR [b]) is present around the position calculated by the above method. Alternatively, the object information (PXR [b], PYR [b]) is projected onto the determination result storage array DTC1 [x] [y] using camera geometry, and the center position of each group and the array of the projected object information When the upper distance is less than or equal to the predetermined value, it is determined that the object is an object, and the object information (PXR [b], PYR [b]) is set as the position of the pedestrian. Similar processing is performed for the determination result storage array DTC2 [x] [y].

  As described above, the vehicle external environment recognition apparatus 1000 can simultaneously detect a pedestrian who is not wearing an umbrella and a pedestrian who is holding an umbrella at the same time. Furthermore, since the common similarity calculation unit 1030 is provided, the processing load can be reduced as compared to simply executing two types of pedestrian pattern matches.

  For example, when pattern matching is performed using a template for the whole body of a pedestrian who is not wearing an umbrella and a template for the whole body of a pedestrian who is holding an umbrella, the lower part of the pedestrian, which is a common part, is duplicated. Thus, matching is determined, and processing is wasted.

  On the other hand, the vehicle external environment recognition apparatus 1000 is similar to the pedestrian's lower body, which is a characteristic part common to a pedestrian who is not wearing an umbrella and a pedestrian who is holding an umbrella, using a common lower body template. The degree of similarity is calculated for the upper body of each pedestrian using the template for the upper body with and without an umbrella, thus preventing duplicate matching judgments for the lower body, The processing load of the entire apparatus can be reduced.

  Note that the discriminators 61, 611, and 612 used for detection of pedestrians are not limited to the example of the method described in this embodiment. Template matching using normalized correlation, a neural network classifier, a support vector machine classifier, a Bayes classifier, or the like may be used. Alternatively, the determination may be performed by the classifiers 61, 611, and 612 using the grayscale image or the color image as they are without extracting the edge.

  The discriminators 61, 611, and 612 adjust the image data of various pedestrians and the image data of the area where there is no risk of collision for the own vehicle as teacher data using machine learning means such as AdaBoost. Also good. In particular, when the vehicle external environment recognition apparatus 1000 includes the object position acquisition unit 1111 as an embodiment, the image data of various pedestrians collide with millimeter wave radars or laser radars such as pedestrian crossings, manholes, cat's eyes, etc. The image data of the area that is erroneously detected in spite of no danger may be used as teacher data.

<Second embodiment>
Next, a second embodiment of the vehicle external environment recognition device of the present invention will be described below with reference to the drawings.

  FIG. 8 is a block diagram showing the configuration of the vehicle external environment recognition device according to the second embodiment. In the following description, only portions different from the vehicle external environment recognition device 1000 in the first embodiment described above will be described in detail, and the same portions will be denoted by the same reference numerals and detailed description thereof will be omitted.

  What is characteristic in the present embodiment is that the environment around the host vehicle is estimated, and the first reliability and the second object determination with respect to the determination result of the first object determination means are determined according to the estimated vehicle environment. The second reliability for the determination result of the means is adjusted.

  The vehicle external environment recognition device 2000 is incorporated in a camera device mounted on an automobile, an integrated controller, or the like, and detects two types of preset objects from an image captured by the camera 1010. The present embodiment is configured to detect a pedestrian who is not wearing an umbrella and a pedestrian who is holding an umbrella from an image obtained by imaging the front of the vehicle.

  The vehicle external environment recognition apparatus 2000 is configured by a computer having a CPU, a memory, an I / O, and the like. A predetermined process is programmed and the process is repeatedly executed at a predetermined cycle.

  As shown in FIG. 8, the vehicle external environment recognition device 2000 includes an image acquisition unit 1011, a processing region setting unit 1021, a common similarity calculation unit 1030, a first specific similarity calculation unit 1031, and a second Specific similarity calculation unit 1032, first object determination unit 1041, second object determination unit 1042, environment estimation unit 2060, first reliability 2061, second reliability 2062, and object position A calculation unit 2071, and further includes a wiper operation acquisition unit 2161 and an object position acquisition unit 1111 according to the embodiment.

  The wiper operation acquisition unit 2161 acquires information on the start / end of the wiper operation of the vehicle by the driver. This operation information may be acquired by directly inputting a wiper ON / OFF signal to the vehicle external environment recognition device 2000, or may be acquired by performing communication using a LAN (Local Area Network). .

  The environment estimation unit 2060 uses the information of the wiper operation acquisition unit 2161, the first object determination unit 1041, the second object determination unit 1042, and the object position acquisition unit 1111 to use the first trust The values of the degree 2061 and the second reliability 2062 are changed. Details of the processing will be described later.

  The first reliability 2061 and the second reliability 2062 are numerical values representing that the higher the value, the higher the reliability. These values are changed by the environment estimation unit 2060 and used by the object position calculation unit 2071.

  The object position calculation unit 2071 calculates the object position using the first object determination unit 1041, the second object determination unit 1042, the first reliability 2061, and the second reliability 2062. When the object position acquisition unit 1111 is included, the object position information acquired from the object position acquisition unit 1111 is used as the object position. Details of the processing will be described later.

[Environment estimation section]
Next, contents of processing in the environment estimation unit 2060 will be described with reference to FIGS. 9 and 10 are flowcharts showing the processing flow of the environment estimation unit 2060. Here, two adjustment methods will be described.

  First, a method using information of the wiper operation acquisition unit 2161 will be described with reference to FIG. First, in step S91, an ON / OFF state of the wiper is acquired from the wiper operation acquisition unit 2161. Next, in step S92, the first reliability 2061 and the second reliability 2062 are set with reference to the table 921 in FIG. 9B according to the ON / OFF state of the wiper.

  Thus, by changing the first reliability 2061 and the second reliability 2062 according to the ON / OFF state of the wiper, for example, it is expected that it is raining when the wiper is ON. When the second reliability 2062 is set larger than the first reliability 2061 (first reliability 2061 <second reliability 2062), conversely, when the wiper is OFF, it is expected that it is not raining. Therefore, the second reliability 2062 is set to be smaller than the first reliability 2061 (first reliability 2061> second reliability 2062).

  Second, a method using the first object determination unit 1041 and the second object determination unit 1042 will be described with reference to FIG.

  First, in step S101, the first object determination unit 1041 counts the number N1 of detections of the first object in the past T seconds. Here, T = 1 second. Next, in step S102, the number N2 of detections of the second object in the past T seconds by the second object determination unit 1042 is counted. Here, T = 1 second. Next, in step S103, it is determined whether the following conditions are satisfied. Here, THN is a predetermined value.

  (N1-N2) / (N1 + N2)> THN

  When the above condition is satisfied, the process proceeds to step S105, and the first reliability 2061 and the second reliability 2062 are set as r11 and r21 (r11> r21), respectively. As a result, when there are enough pedestrians with umbrellas than pedestrians without umbrellas, the reliability of pedestrians with umbrellas is higher than the reliability of pedestrians without umbrellas. It is set large.

  If the condition in step S103 is not satisfied, the process proceeds to step S104 to determine whether the following condition is satisfied. Here, THN is a predetermined value.

  (N2-N1) / (N1 + N2)> THN

  When the above condition is satisfied, the process goes to step S106, and the first reliability 2061 and the second reliability 2062 are set as r12 and r22 (r12 <r22), respectively. As a result, when the number of pedestrians not wearing umbrellas is sufficiently greater than the number of pedestrians holding umbrellas, the reliability of pedestrians not holding umbrellas is higher than the reliability of pedestrians holding umbrellas. It is set large.

  If the condition in step S104 is not satisfied, the process proceeds to step S107, and the first reliability 2061 and the second reliability 2062 are set to r13 and r23 (r13 = r23), respectively. As a result, when the number of pedestrians holding umbrellas and pedestrians holding umbrellas is approximately equal, the reliability of pedestrians holding umbrellas and pedestrians holding umbrellas is set to be the same. The In this way, by referring to the past number of detections, when one detection frequency is higher than the other, processing for increasing reliability is performed.

  Next, the contents of processing in the object position calculation unit 2071 will be described.

  The object position calculation unit 2071 groups places where “1” is included in the determination result storage array DTC1 [x] [y], and determines that a pedestrian exists at the center position. A known technique such as labeling is used for grouping.

  Next, referring to the detection result of the past T seconds, the number of times DT each group has existed in the past is counted. And the value JG which multiplied the frequency | count DT and the 1st reliability 2061 is calculated, and if the value JG is more than the predetermined threshold value THJG, it will determine with a pedestrian existing there.

  If the height of the pedestrian is assumed to be 180 cm using the Y coordinate of the grouped area and the camera geometry, the distance to the pedestrian can be calculated. Also, assuming that the pedestrian's width is 60 cm using the X coordinate of the grouped area and the camera geometry, how much the pedestrian is in the lateral position (vehicle width direction) from the center of the vehicle. Can be calculated.

  When the object position acquisition unit 1111 is present, information is replaced when object information (PXR [b], PYR [b]) is present around the position calculated by the above method.

  Alternatively, the object information (PXR [b], PYR [b]) is projected onto the determination result storage array DTC1 [x] [y] using camera geometry, and the center position of each group and the array of the projected object information When the upper distance is less than or equal to the predetermined value, it is determined that the object is an object, and the object information (PXR [b], PYR [b]) is set as the position of the pedestrian. Similar processing is performed for the determination result storage array DTC2 [x] [y].

  As described above, by using the reliability and the number of past detections, the corresponding object can be detected immediately when the first reliability or the second reliability is high, and the detection is performed when the reliability is low. It takes time to wait and wait for a while.

  When the environment estimation unit 2060 is in the first format shown in FIG. 9, the reliability of a pedestrian wearing an umbrella is changed by turning on / off the wiper. Therefore, both pedestrians who are not wearing umbrellas and pedestrians who are holding umbrellas are performing object determination, but pedestrians who are holding umbrellas are not holding umbrellas when wipers are OFF. Each pedestrian is less likely to be detected as the final result. Therefore, erroneous determination can be reduced, and the detection accuracy of pedestrians can be improved.

  Further, when the environment estimation unit 2060 is in the second format shown in FIG. 10, the reliability of the pedestrian not wearing the umbrella and the pedestrian holding the umbrella is changed depending on the frequency of the past determination result. . Therefore, although the pedestrian who is not wearing an umbrella and the pedestrian who is holding an umbrella perform object determination, if the number of detections of one is sufficiently large, the one with a small number of detections is less likely to be detected as a final result. Therefore, erroneous determination can be reduced, and the detection accuracy of pedestrians can be improved.

<Third embodiment>
Next, a third embodiment of the vehicle external environment recognition device of the present invention will be described below with reference to the drawings.

  FIG. 11 is a block diagram showing an embodiment of the vehicle external environment recognition device 3000. In the following description, only parts different from the vehicle external environment recognition device 1000 of the first embodiment and the vehicle external environment recognition device 2000 of the 20th embodiment will be described in detail, The same number is attached and explanation is omitted.

  The vehicle external environment recognition device 3000 is incorporated in a camera device mounted on an automobile, an integrated controller, or the like, and detects two types of objects set in advance from an image captured by the camera 1010. In this embodiment, a lateral pedestrian facing in the vehicle width direction of the own vehicle and a vertical pedestrian facing in the front-rear direction of the own vehicle are detected from within an image obtained by imaging the front of the own vehicle. Is configured to do.

  The vehicle external environment recognition device 3000 is configured by a computer having a CPU, a memory, an I / O, and the like. A predetermined process is programmed, and the process is repeatedly executed at a predetermined cycle. As shown in FIG. 11, the vehicle external environment recognition device 3000 includes an image acquisition unit 1011, a processing region setting unit 1021, a common similarity calculation unit 3030, a first specific similarity calculation unit 3031, and a second Specific similarity calculation unit 3032, first object determination unit 3041, second object determination unit 3042, environment estimation unit 3060, first reliability 2061, second reliability 2062, and object position A calculation unit 3071 and an object position acquisition unit 1111 are included.

  The common similarity calculation unit 3030 calculates the similarity with the outline template from the head to the shoulder of the pedestrian, which is a common pattern for the pedestrians in the horizontal direction and the pedestrians in the vertical direction. The common similarity calculation unit 3030 includes a template TP0 of the pedestrian's head, and the determination region 31 (SXG [g], SYG [g], EXG [g], EYG [g] of the image IMGSRC [x] [y]. ]) And the similarity MT0 between the template TP0. The details of the process are the same as those of the above-described common similarity calculation unit 1030 except that the template TP0 is a contour from the head to the shoulder of the pedestrian, and thus will be omitted.

  The first unique similarity calculation unit 3031 calculates a similarity between the lateral pedestrian's shoulder and the template below, which is a unique pattern of the lateral pedestrian. The first unique similarity calculation unit 3031 has a contour template TP1 from the shoulder of a sideways pedestrian to the determination region (SXG [g], SYG [g], EXG of the image IMGSRC [x] [y]. [G], EYG [g]) and the similarity MT1 between the template TP1 are calculated. The details of the process are the same as those of the first specific similarity calculation unit 1031 described above except that the template TP1 is a contour from the shoulder of the sideways pedestrian, and thus will not be described.

  The second unique similarity calculation unit 3032 calculates the similarity between the vertical pedestrian's shoulder and the lower contour template, which is a specific pattern of the vertical pedestrian. The second unique similarity calculation unit 3032 has a template TP2 below the shoulder of the vertically oriented pedestrian, and the determination region (SXG [g], SYG [g], EXG of the image IMGSRC [x] [y]. [G], EYG [g]) and the similarity MT2 between the template TP2 are calculated. The details of the process are the same as those of the second unique similarity calculation unit 1032 described above except that the template TP2 has a contour extending downward from the shoulder of a vertically oriented pedestrian, and thus will be omitted.

The first object determination unit 3041 uses the common similarity MT0 with the common part template TP0 and the first specific similarity MT1 with the sideways pedestrian-specific template TP1 to determine whether there is a sideways pedestrian. Determine. The determination is made, for example, when the condition that the value obtained by adding the common similarity MT0 and the first specific similarity MT1 is larger than the threshold TH1_MT using the threshold TH1_MT, that is,
(MT0 + MT1)> TH1_MT
If the common similarity MT0 is greater than the threshold TH1_MT0 and the first specific similarity MT1 is determined using the common similarity threshold TH1_MT0 and the first specific similarity threshold TH1_MT1. Is larger than the threshold value TH1_MT1, that is,
MT0> TH1_MT0 and MT1> TH1_MT1
If so, it may be determined that an object exists.

The second object determination unit 3042 uses the common similarity MT0 with the common part template TP0 and the second specific similarity MT2 with the vertical pedestrian-specific template TP2 to determine the pedestrian in the vertical direction. Determine presence or absence. The determination is made, for example, when the condition that the value obtained by adding the common similarity MT0 and the second specific similarity MT2 is larger than the threshold TH2_MT using the threshold TH2_MT, that is,
(MT0 + MT2)> TH2_MT
If the common similarity MT0 is larger than the threshold TH2_MT0 and the second specific similarity MT2 is determined using the common similarity threshold TH2_MT0 and the second specific similarity threshold TH2_MT2. Is larger than the threshold value TH2_MT2, that is,
MT0> TH2_MT0 and MT2> TH2_MT2
If so, it may be determined that an object exists.

  The environment estimation unit 3060 performs processing for adjusting the first reliability 2061 and the second reliability 2062 based on the history of the object position using the information from the object position acquisition unit 1111. This process will be described with reference to FIG.

  First, in step S121, information (relative distances PYR [b], PXR [b]) of an object that is a processing target selected by the processing area setting unit 1021 is acquired. Next, in step S122, the vertical movement speed VY [b] and the horizontal movement speed VX [b] of the object are calculated from the difference in the position information of the past object and the vehicle speed.

  In step S123, it is determined whether the following condition is satisfied. Here, Abs (X) is an operation for calculating an absolute value, and THVX is a predetermined value determined in advance. For example, it is set to 0.5 [m / s].

Abs (VX [b])> Abs (VY [b])
And Abs (VX [b])> THVX

  That is, it is determined whether or not the condition that the horizontal movement speed is higher than the vertical movement speed and the absolute value is sufficiently large is satisfied. If this condition is satisfied, the process proceeds to step S215, and the first reliability 2061 and the second reliability 2062 are set to r11 and r21 (r11> r21), respectively. Thereby, when there are more lateral pedestrians than vertical pedestrians, the reliability of lateral pedestrians is set larger than the reliability of vertical pedestrians.

  If the condition in step S123 is not satisfied, the process goes to step S124 to determine whether the following condition is satisfied. Here, THVY is a predetermined value set in advance, and is set to 0.5 [m / s], for example.

Abs (VY [b])> Abs (VX [b])
And Abs (VY [b])> THVY

  That is, it is determined whether or not the condition that the vertical movement speed is larger than the horizontal movement speed and the absolute value is sufficiently large is satisfied. If this condition is satisfied, the process proceeds to step S126, and the first reliability 2061 and the second reliability 2062 are set to r12 and r22 (r12 <r22), respectively. Thereby, when there are sufficiently more pedestrians in the vertical direction than pedestrians in the horizontal direction, the reliability of the pedestrian in the vertical direction is set larger than the reliability of the pedestrian in the horizontal direction.

  If the condition of step S124 is not satisfied, the process proceeds to step S127, and the first reliability 2061 and the second reliability 2062 are set to r13 and r23 (r13 = r23), respectively. Thereby, when there is no large difference in the number of pedestrians in the horizontal direction and in the vertical direction, the reliability of the pedestrians in the horizontal direction and the reliability of the pedestrians in the vertical direction are set to be the same.

  As described above, in the image processing for an object moving in the horizontal direction, the pedestrian in the horizontal direction is determined by changing the reliability of the vertical pedestrian and the horizontal pedestrian based on the object position information. In the image processing for the object moving in the vertical direction, it is possible to easily determine the pedestrian in the vertical direction.

  In the present embodiment, the case where the environment estimation unit 3060, the first reliability 2061, the second reliability 2062, and the object position calculation unit 3071 are used has been described as an example. Alternatively, the object position calculation unit 1051 (see FIG. 1) in the first embodiment described above may be used. In the present embodiment, the case where the environment estimation unit 3060 is used has been described as an example. Instead, the second format illustrated in FIG. 10 of the environment estimation unit 2060 in the second embodiment described above can be used. .

  Furthermore, when the processing capability of the computer is low, it may not be possible to execute all pattern matching processes when parallel processing such as white line recognition is performed from a predetermined own vehicle speed. Therefore, for example, in the second embodiment of the vehicle external environment recognition device shown in FIG. 8 or the third embodiment of the vehicle external environment recognition device shown in FIG. When the processing of the determination unit with higher reliability and the processing of the common similarity calculation unit and the specific similarity calculation unit referred to by the determination unit is prioritized and there is a margin in processing The remaining object determination unit having a lower reliability and the processing of the unique similarity calculation unit referred to by the determination unit may be executed.

  As described above, by providing reliability, when the processing load is large and not all processes can be performed, the priority of the process is changed based on the reliability, and the object determination process with high reliability is given priority. be able to.

<Fourth embodiment>
Next, a fourth embodiment of the vehicle external environment recognition device of the present invention will be described below with reference to the drawings. FIG. 13 is a block diagram illustrating an embodiment of a vehicle external environment recognition device 4000. In the following description, only the parts different from the above-described vehicle external world recognition devices 1000 to 3000 of the first embodiment to the third embodiment will be described in detail, and the same parts will be denoted by the same reference numerals. Detailed description is omitted.

  The vehicle external environment recognition device 4000 is incorporated in a camera device mounted on an automobile, an integrated controller, or the like, and detects two types of preset objects from an image captured by the camera 1010. In the present embodiment, for example, a pedestrian who can see the whole body and a pedestrian who can see only the lower body are detected from an image obtained by imaging the front of the vehicle at night by a low beam of a headlight. Has been.

  The vehicle external environment recognition device 4000 is configured by a computer having a CPU, a memory, an I / O, and the like. A predetermined process is programmed, and the process is repeatedly executed at a predetermined cycle.

  As shown in FIG. 13, the vehicle external environment recognition device 4000 includes an image acquisition unit 1011, a processing region setting unit 1021, a common similarity calculation unit 4030, a first specific similarity calculation unit 4031, An object determining unit 4041, a second object determining unit 4042, and an object position calculating unit 1051 are included, and an object position acquiring unit 1111 is included in some embodiments.

  The common similarity calculation unit 4030 calculates a similarity between the pedestrian's lower body template TP0, which is a common pattern of a pedestrian who can see the whole body and a pedestrian who can see only the lower body. The common similarity calculation unit 4030 includes a template TP0 for the lower body of the pedestrian, and the determination regions (SXG [g], SYG [g], EXG [g], EYG [g] of the image IMGSRC [x] [y]. ) And the template TP0 is calculated. The template for the lower body of the pedestrian is shown in FIG. 5, and the calculation process of the similarity MT0 is the same as that of the common similarity calculation unit 1030 in the first embodiment described above.

  The first unique similarity calculation unit 4031 calculates the similarity with the template TP1 of the upper body of the pedestrian, which is a unique pattern of the pedestrian who can see the whole body. The first unique similarity calculation unit 4031 has a template TP1 for the upper body of the pedestrian, and the determination regions (SXG [g], SYG [g], EXG [g], EYG of the image IMGSRC [x] [y]. The similarity MT1 between [g]) and the template TP1 is calculated. Note that the template TP1 for the upper body of the pedestrian is shown in FIG. 7A, and the calculation process of the similarity MT1 is the same as that of the first specific similarity calculation unit 1031 in the first embodiment described above. Because there is, explanation is omitted.

The first object determination unit 4041 uses the common similarity MT0 with the template TP0 of the common part and the first specific similarity MT1 with the template TP1 unique to the pedestrian who can see the whole body to see the whole body. Determine the presence or absence of pedestrians. The determination is made, for example, when the condition that the value obtained by adding the common similarity MT0 and the first specific similarity MT1 is larger than the threshold TH1_MT using the threshold TH1_MT, that is,
(MT0 + MT1)> TH1_MT
If the common similarity MT0 is greater than the threshold TH1_MT0 and the first specific similarity MT1 is determined using the common similarity threshold TH1_MT0 and the first specific similarity threshold TH1_MT1. Is larger than the threshold value TH1_MT1, that is,
MT0> TH1_MT0 and MT1> TH1_MT1
If so, it may be determined that an object exists.

The second object determination unit 3042 determines the presence / absence of a pedestrian who can see only the lower body, using the common similarity MT0 with the template of the common part. The determination is performed when, for example, the threshold TH2_MT is used and the condition that the common similarity MT0 is larger than the threshold TH2_MT is satisfied, that is,
MT0> TH2_MT
If it is, it will determine with a pedestrian existing.

  In the present embodiment, as shown in FIG. 13, the case where the object position calculation unit 1051 is used is described as an example, but instead of using this, the environment in the above-described third embodiment is used instead. The estimation unit 3060, the first reliability 2061, the second reliability 2062, and the object position calculation unit 3071 may be used.

  According to the present embodiment, it is possible to detect both a pedestrian who can see the whole body from an image obtained by capturing the front of the host vehicle and a pedestrian who can see only the lower body with a smaller processing load.

<Fifth embodiment>
Next, referring to FIG. 14, taking a pre-crash safety system as an example, an alarm is output according to the pedestrian determined by the vehicle external environment recognition device of each embodiment described above, or the brake is automatically applied. An operation method of the system such as control will be described.

  FIG. 14 is a flowchart showing an operation method of the pre-crash safety system.

  First, in step S141, the position information of the pedestrian detected by any of the above-described vehicle external recognition devices is read as obstacle information (PYO [b], PXO [b], WDO [b]).

  Next, in step S142, the collision prediction time TTC [i] of each detected object is calculated using the following equation (3). Here, the relative velocity VYO [b] is obtained by pseudo-differentiating the relative distance PYO [b] of the object.

  TTC [b] = PY [b] ÷ VY [b] (3)

  Further, in step S143, the risk level DRECI [b] for each obstacle is calculated.

  Hereinafter, an example of a method for calculating the degree of risk DRECI [b] for the object X [b] detected by any one of the above-described vehicle external world recognition devices will be described with reference to FIG.

  First, a method for estimating a predicted course will be described. As shown in FIG. 15, when the vehicle position is the origin O, the predicted course can be approximated by an arc of a turning radius R passing through the origin O. Here, the turning radius R is expressed by the following equation (4) using the steering angle α, the speed Vsp, the stability factor A, the wheel base L, and the steering gear ratio Gs of the host vehicle.

  R = (1 + AV2) × (L · Gs / α) (4)

  Here, the stability factor A is an important value that determines the magnitude of the change depending on the speed of the steady circular turning of the vehicle. As can be seen from the above equation (2), the turning radius R changes in proportion to the square of the speed Vsp of the host vehicle with the stability factor A as a coefficient. Further, the turning radius R can be expressed by Expression (5) using the vehicle speed Vsp and the yaw rate γ.

  R = V / γ (5)

  Next, a perpendicular line is drawn from the object X [b] to the center of the predicted course approximated by the arc of the turning radius R to obtain the distance L [b].

  Further, the distance L [b] is subtracted from the own vehicle width H. When this is a negative value, the danger level DRECI [b] = 0, and when the value is a positive value, the danger level DRECI [b] is given by the following equation (6). ] Is calculated.

  DRECI [b] = (HL [b]) / H (6)

  In addition, the process of step S141-S143 is set as the structure which performs a loop process according to the detected number of objects.

  In step S144, an object satisfying the condition of Expression (7) is selected according to the risk degree DRECI [b] calculated in step S143, and the predicted collision time TTC [b] is the smallest among the selected objects. The object bMin is selected.

  DRECI [b] ≧ cDRECI # (7)

  Here, the predetermined value cDRECI # is a threshold value for determining whether or not the vehicle collides.

  Next, in step S145, it is determined whether or not the brake is automatically controlled in accordance with the predicted collision time TTC [bMin] of the selected object k. If Expression (8) is established, the process proceeds to step S146, brake control is executed, and the process is terminated. On the other hand, if Expression (8) is not established, the process proceeds to step S147.

  TTC [bMin] ≦ cTTCBRK # (8)

  In step S147, it is determined whether or not the alarm is output in accordance with the predicted collision time TTC [bMin] of the selected object bMin. If Expression (9) is established, the process proceeds to step S148, where an alarm is output and the process is terminated. In addition, when Expression (9) is not established, neither the brake control nor the warning is executed and the process is terminated.

  TTC [bMin] ≦ cTTCALM # (9)

  As described above, the alarm and the brake control can be activated for an object determined as a pedestrian in any one of the above vehicle external recognition devices according to the present invention, depending on the degree of risk.

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

1000 Vehicle External Recognizer 1011 Image Acquisition Unit 1021 Processing Area Setting Unit 1030 Common Similarity Calculation Unit 1031 First Specific Similarity Calculation Unit 1032 Second Specific Similarity Calculation Unit 1041 First Object Determination Unit 1042 Second Object determination unit 1051 Object position calculation unit 1111 Object position acquisition unit 2000 Vehicle external environment recognition device 2161 Wiper operation acquisition unit 2060 Environment estimation unit 2061 First reliability 2062 Second reliability 2071 Object position calculation unit 3000 Vehicle external environment recognition Device 3030 Common similarity calculation unit 3031 First specific similarity calculation unit 3032 Second specific similarity calculation unit 3041 First object determination unit 3042 Second object determination unit 3060 Environment estimation unit 4000 Vehicle external environment recognition device 4030 Common similarity calculation unit 4031 First unique similarity calculation unit 4041 First object size Part 4042 second object determination unit

Claims (11)

An external recognition device for a vehicle that detects a predetermined first object and a second object from around the vehicle,
Image acquisition means for acquiring an image of the periphery of the vehicle;
A common similarity calculation means for calculating a common similarity that is a similarity of a common part of the first object and the second object from the image;
A first unique similarity calculating means for calculating a first unique similarity that is a similarity of a specific part of the first object from the image;
A second specific similarity calculating means for calculating a second specific similarity that is a similarity of a specific part of the second object from the image;
First object determination means for determining the presence or absence of the first object using the common similarity and the first specific similarity;
Second object determination means for determining the presence or absence of the second object using the common similarity and the second specific similarity;
A vehicle external environment recognition device characterized by comprising:   The environment around the host vehicle is estimated, and the first reliability with respect to the determination result of the first object determination means and the second reliability with respect to the determination result of the second object determination means according to the estimated environment The environment recognizing device for a vehicle according to claim 1, further comprising environment estimating means for adjusting   The environment estimation unit is configured to determine the first reliability based on a determination result of a determination process of the first object determination unit and the second object determination unit that is repeatedly executed at a predetermined period in a preset period. The external environment recognition device for a vehicle according to claim 2, wherein the second reliability is adjusted.   The environment estimation unit is configured such that, among the first object determination unit and the second object determination unit, the reliability of the one with the larger number of object detections is higher than the reliability with the smaller number of detections. The vehicle external environment recognition device according to claim 3, wherein the first reliability and the second reliability are adjusted. An object position acquisition means for acquiring an object position of an object existing in front of the host vehicle;
The vehicle environment recognition according to claim 2, wherein the environment estimation unit adjusts the first reliability and the second reliability based on a history of the object position acquired by the object position acquisition unit. apparatus. When the first reliability is higher than the second reliability, the processing of the first specific similarity calculation unit and the first object determination unit is performed by calculating the second specific similarity. Priority over the processing of the means and the second object determination means,
When the second reliability is higher than the first reliability, the processing of the second specific similarity calculation means and the second object determination means is performed by calculating the first specific similarity. The vehicle external environment recognition device according to any one of claims 2 to 5, wherein the vehicle has priority over the processing of the first object determination unit and the first object determination unit.   When the first object determining means determines that the first object exists, the position of the first object is calculated, and the second object determining means determines that the second object exists. The vehicle external environment recognition device according to any one of claims 1 to 6, further comprising an object position calculating unit that calculates the position of the second object when the second object is detected. An external recognition device for a vehicle that detects a predetermined first object and a second object from around the vehicle,
Image acquisition means for acquiring an image of the periphery of the vehicle;
A common similarity calculation means for calculating a common similarity that is a similarity of a common part of the first object and the second object from the image;
A first unique similarity calculating means for calculating a first unique similarity that is a similarity of a specific part of the first object from the image;
First object determination means for determining the presence or absence of the first object using the common similarity and the first specific similarity;
Second object determination means for determining the presence or absence of the second object using the common similarity;
An external environment recognition device for a vehicle. A vehicle system comprising the vehicle external environment recognizing device according to any one of claims 2 to 8,
A collision risk calculating means for calculating a risk of collision of the vehicle according to the position of the first object and the position of the second object detected by the object position detecting means;
Control means for performing control to avoid a collision of the own vehicle according to the collision risk calculated by the collision risk calculation means;
A vehicle system comprising:   The control means calculates an expected collision time when the host vehicle collides with the first object or the second object based on the position of the first object and the position of the second object, and the calculation 10. The braking control for the host vehicle is performed when the predicted collision time is compared with a preset braking threshold and the predicted collision time is less than or equal to the braking threshold. The vehicle system described. The control means calculates a predicted collision time when the host vehicle collides with the object based on the position of the first object and the position of the second object, and the calculated predicted collision time is preset. 10. The vehicle system according to claim 9, wherein an alarm control is performed on a driver of the host vehicle when the estimated collision time is equal to or less than the alarm threshold by comparing with an alarm threshold. 10.

Images