JP2003099762A - Precedent vehicle recognition device and recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To combine vehicle recognition using a laser radar and that using an image sensor for carrying out highly precise recognition. SOLUTION: As to a target area in a photographed image during actual travel, an edge histogram is calculated by a camera module for finding a maximum point and extracting a vehicle possible area (S1). By a laser radar module, a reflection point of a laser beam is specified for deriving a received light intensity histogram (S2). By means of a CPU, vectors and laser vectors in the X and Y directions are formed and fused for forming a fusion vector (S3), and the fusion vector of a vehicle possible area and a dictionary fusion vector are collated with each other (S4). From a distance between them, it is determined whether the vehicle possible area is a vehicle or not (S5). If the distance is lower than a threshold value, it is predicted that the vehicle possible area is a vehicle (S7).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a front vehicle recognizing device and a recognizing method for recognizing a vehicle traveling in front of an own vehicle.

[0002]

2. Description of the Related Art In recent years, a vehicle equipped with a more advanced and more comfortable driving support system, such as a follow-up running function and a steering assist function on a highway, has been proposed. There is a recognition device that recognizes the vehicle.

In a conventional recognition device using this kind of scan laser radar, the distance to the reflection point can be detected by measuring the time from the irradiation of the laser beam to the observation of the reflected light, and the scanning mechanism is provided. By providing 10
If the vehicle has a horizontal field of view of several degrees and is a short-distance vehicle or a medium-distance vehicle, as shown in FIGS. 11 (a) and 11 (b), the reflectors (reflectors) mounted on both ends of the rear end surface of the vehicle can be used. In addition, many reflections from the body are observed, and the number of reflection points reaches three or more. On the other hand, in the case of a vehicle at a long distance, as shown in FIG. Only 2 points are observed.

However, in this case, there is no constant reflection from the same part of the vehicle body, and the direction, shape,
The reflection position changes every moment due to the positional relationship and the distribution is not uniform.

Therefore, there has been proposed a method of recognizing a preceding vehicle by clustering the obtained reflection point group by a fuzzy method or the like. As a specific example thereof, Preprints 931 No. 93017
19, pp53-56 (1993-5) ("Development of preceding vehicle recognition algorithm by laser radar"), or Japanese Patent Application Laid-Open No. 6-30
There is a method described in Japanese Patent No. 9600.

For example, the relative speed of the vehicle candidate point group with respect to the own vehicle is derived from the change in the distance between the own vehicle and the vehicle candidate point group,
When the derived relative speed is greater than a predetermined value, a fitness function that is a function of the difference between the own vehicle speed and the relative speed of the preceding vehicle candidate, the amount of positional deviation of the preceding vehicle candidate with respect to the own vehicle in the vehicle width direction, etc. is used. Each time it calculates the certainty factor that represents the certainty that it is a preceding vehicle, and every time the coordinates and relative speed of the vehicle candidate point group in the coordinate system whose coordinate axes are the scanning direction of the laser light and the traveling direction of the own vehicle are calculated. The confidence factor calculation is repeated, and the difference between the confidence factor and the reference value is cumulatively added to determine whether the added total value is greater than another reference value. Is judged to be running, and when it is large, it is judged not to be on the same lane.

[0007]

However, these conventional methods have almost no problem when the distance between vehicles or the distance between vehicles and a roadside structure is wide like an expressway. If you try to expand to, the following problems will occur. That is, as a first problem, the reflection from a traveling vehicle, a parked vehicle, or a roadside object in the vicinity of a vehicle in front,
The second problem is that the distance measurement value of the front vehicle is erroneously processed because it is included in the reflection from the front vehicle, and the second problem is that when there is no vehicle, there is no reflection from roadside structures such as a reflector of a guardrail or a signboard. Reflections are also observed, and these may be mistakenly recognized as parked vehicles or forward vehicles. As a third problem, the horizontal detection field of view of the radar is generally narrow, so the detection of vehicles interrupting at a short distance is delayed. , There is a problem.

Further, in the case of a laser radar, when the front vehicle splashes water on the road surface, the laser light is attenuated, the front vehicle cannot be recognized, or the front vehicle traveling in the adjacent lane is in the same lane as the observation vehicle. Although it is determined that the vehicle is a moving vehicle, these are problems unique to the laser radar and other problems that require a very high level of recognition processing, and are not problems to be solved by the present invention this time.

On the other hand, conventionally, an apparatus for recognizing a vehicle by processing an image picked up by an image sensor has also been proposed. In the case of the image sensor, the pattern recognition ability is superior to that of the laser radar described above. In this type of device, for example, as shown in FIG. 12, a horizontally long gaze area is set in an image picked up by an image sensor, and an edge histogram EHx in the X direction (horizontal direction) from the density of each pixel in the gaze area, Edge histogram E in the Y direction (vertical direction) orthogonal to
Hy and the product EHxy of edge histograms in both directions
Has been proposed to extract the vehicle candidate area.

However, the image sensor has the following problems in a complicated scene. In other words, the distance measurement can be approximated by setting some assumptions such as the width of the preceding vehicle, and the prediction value is difficult to stabilize when applied to a position prediction algorithm such as a Kalman filter. Also,
In a long distance, the pattern recognition power cannot be sufficiently exerted, and in the case of a close-up distance, a large vehicle cannot be photographed entirely, which makes it difficult to recognize the vehicle. Further, in a complicated scene, an edge created by other than a vehicle may be erroneously recognized as a vehicle, and it is often difficult to recognize the vehicle when the apparent image changes due to a temporary shadow or cloud.

Therefore, according to the present invention, the vehicle recognition using the laser radar and the vehicle recognition using the image sensor are combined to eliminate reflections from vehicles other than the vehicle in front of the vehicle, roadside objects, and the like, thereby realizing highly accurate recognition. The purpose is to be able to do.

[0012]

In order to achieve the above-mentioned object, a front vehicle recognition device according to the present invention is a front vehicle recognition device for recognizing a vehicle traveling in front of the own vehicle. An image sensor, an image processing unit that sets a predetermined gaze area for an image captured by the image sensor, and derives an edge histogram that is a density projection value of each pixel in the gaze area in a predetermined direction; A laser radar that irradiates a laser beam while scanning in the horizontal direction and receives reflected light from a reflection point to identify the positions of the plurality of reflection points, and sample images of various sample vehicles prepared in advance The pattern of the edge histogram derived by the image processing unit and a plurality of reflections specified by the laser radar for each of the sample vehicles. A storage unit for storing a plurality of dictionary pattern obtained by dimensional compression by fusing a pattern eigenspace method, a matching pattern based on the edge histogram derived during actual running by the image processing unit,
Of the dictionary patterns stored in the storage unit, a fusion pattern formed by fusing the distribution pattern of the respective reflection points specified during actual traveling by the laser radar and dimensionally compressing by applying the eigenspace method is used. It is characterized in that it is provided with a collating unit for collating whether or not it substantially matches any of the above and judging whether or not the vehicle is a forward vehicle.

With such a configuration, the laser radar is excellent in distance measurement, and the image sensor can accurately distinguish between the vehicle and the others. Therefore, during actual traveling, the matching pattern based on the edge histogram derived by the image processing unit is used. , And the distribution pattern of each reflection point specified by the laser radar are fused at the stage of raw data and dimensionally compressed by the eigenspace method, and the fusion pattern almost matches one of the dictionary patterns in the storage unit. Whether the vehicle is ahead or not is determined and it is determined whether or not the vehicle is ahead.Therefore, it is possible to prevent reflections from vehicles other than the front vehicle or roadside objects from being included in the reflections from the front vehicle as in the case of laser laser alone. In addition, it is possible to reduce the frequency of erroneous recognition of a vehicle in a complicated scene such as in the case of an image sensor, and it is possible to recognize a front vehicle with high accuracy. .

In the forward vehicle recognizing device according to the present invention, the image processing unit sets an edge histogram in the X direction of the screen from the grayscale of each pixel in the gaze area set for the picked-up image, which is orthogonal to the edge histogram. The vehicle candidate area is extracted by calculating the product of the edge histogram in the Y direction of the screen and the edge histogram in both directions and deriving the maximum point thereof, and the collation unit causes the image processing unit to detect the vehicle candidate area. A first creating unit that creates a size-reduced X-direction vector and Y-direction vector as the matching pattern based on each of the edge histograms in the X-direction and the Y-direction that are derived in step 1; and the vehicle candidate by the laser radar. A received light intensity histogram of each of the reflection points specified in the area is formed and based on the received light intensity histogram. A second creation unit for creating a laser vector obtained by reducing the size as the distribution pattern,
And a fusion unit that forms a fusion vector of the vehicle candidate region, which is dimensionally compressed by applying an eigenspace method, as the fusion pattern, based on the X-direction vector, the Y-direction vector, and the laser vector in the vehicle candidate region. It is characterized by being.

According to this structure, the X-direction vector and the Y-direction vector as the matching pattern obtained by the image processing and the laser vector as the distribution pattern obtained from the received light intensity histogram of the reflection points by the laser radar are fused. And dimensionally compressed by the eigenspace method,
Since a fusion vector is formed as a fusion pattern, it is possible to improve the accuracy of pattern recognition by dimensionally compressing the fusion of the data at the sensor raw data stage of the laser and image, and in a complex scene. Even if there is, the vehicle ahead can be recognized with high accuracy.

Further, in the forward vehicle recognizing device according to the present invention, the dictionary pattern is an edge histogram in the X direction and the Y direction derived by the image processing unit with respect to the sample image of each sample vehicle. Based on the dictionary X-direction and Y-direction vectors created by reducing the size by the first creating unit, and the reflection points specified by the laser radar for the sample vehicle, the second The dictionary laser vector created by reducing the size by the creating unit, a dictionary fusion vector obtained by being fused by the fusion unit and dimensionally compressed by the eigenspace method, and the collating unit,
The distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual traveling is derived, and the vehicle candidate area is a forward vehicle depending on whether or not the distance is smaller than a predetermined threshold value. The feature is to judge whether or not.

According to this structure, it is a complicated scene because it is determined whether or not the vehicle candidate area is a front vehicle from the distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual driving. However, it is possible to significantly reduce the frequency of erroneous recognition as in the case of the image sensor alone, and it is possible to prevent the error from expanding as in the case of the laser radar alone.

Further, the forward vehicle recognizing device according to the present invention is characterized in that the fusion vector of the vehicle candidate area determined to be a forward vehicle by the collation unit is registered in the storage unit as the dictionary fusion vector. It has a feature.

According to this structure, the accuracy of the dictionary fusion vector is increased, and thus the recognition accuracy of the front vehicle can be improved.

Further, the forward vehicle recognizing device according to the present invention is characterized in that the collating unit predicts a motion of the vehicle candidate area determined to be a forward vehicle by a Kalman filter and uses it for the next collating process.

According to such a configuration, the movement of the vehicle candidate area determined to be a forward vehicle is predicted and used in the next matching process, so that more accurate vehicle recognition can be realized.

Further, in the forward vehicle recognition method according to the present invention, a predetermined gaze area is set in the picked-up image by the image sensor, and an edge histogram which is a density projection value of each pixel in the gaze area in a predetermined direction is calculated. Derivation step of deriving, and a laser radar, a specific step of irradiating the front side of the vehicle while scanning the laser beam in the horizontal direction and receiving reflected light from a reflection point to specify the positions of the plurality of reflection points. , A fusion pattern in which a collation pattern based on the edge histogram derived during actual traveling and a distribution pattern of each reflection point specified during actual traveling by the laser radar are fused and dimensionally compressed by an eigenspace method is prepared in advance. For each of the various sample vehicles described above, the edge histogram pattern derived from the sample image and the laser Matching with the patterns of multiple reflection points specified by the method and matching with any of the dictionary patterns obtained by dimensional compression by the eigenspace method to determine whether it is a forward vehicle It is characterized by including a process.

According to this structure, at the time of actual traveling, the matching pattern based on the edge histogram derived by the image processing unit and the distribution pattern of each reflection point specified by the laser radar are processed at the raw data stage. The fusion pattern fused and dimensionally compressed by the eigenspace method is checked to see if it almost matches any one of the dictionary patterns, and it is determined whether the vehicle is a forward vehicle. It is possible to prevent reflections from vehicles other than the front vehicle or roadside objects from being included in the reflections from the front vehicle, and reduce the frequency of erroneous recognition of the vehicle in a complicated scene such as in the case of an image sensor. The vehicle can be recognized with high accuracy.

Further, in the forward vehicle recognition method according to the present invention, the derivation step is an edge histogram in the X direction of the screen from the grayscale of each pixel in the gaze area set for the captured image, which is orthogonal to the edge histogram. The method includes a step of calculating a product of an edge histogram in the Y direction of the screen and a product of the edge histograms in both directions and deriving a maximum point thereof to extract a vehicle candidate area, and the matching step is derived in the vehicle candidate area. Based on each of the X-direction and Y-direction edge histograms, the size-reduced X-direction vector and Y-direction vector are created as the matching pattern; and the laser radar identifies each of the vehicle candidate regions. Form a received light intensity histogram of reflection points and reduce the size based on the received light intensity histogram A laser vector as the distribution pattern, and a fusion vector of the vehicle candidate region, which is dimensionally compressed by applying an eigenspace method based on the X-direction vector, the Y-direction vector and the laser vector in the vehicle candidate region. Is formed as the fusion pattern.

According to this structure, the X-direction vector and the Y-direction vector as the matching pattern obtained by the image processing and the laser vector as the distribution pattern obtained from the received light intensity histogram of the reflection points by the laser radar are fused. And dimensionally compressed by the eigenspace method,
Since a fusion vector is formed as a fusion pattern, it is possible to improve the accuracy of pattern recognition by dimensionally compressing the fusion of the data at the sensor raw data stage of the laser and image, and in a complex scene. Even if there is, the vehicle ahead can be recognized with high accuracy.

In the forward vehicle recognition method according to the present invention, the size of the dictionary pattern is determined with respect to the sample image of each sample vehicle based on the derived edge histograms in the X and Y directions. A dictionary X-direction vector and a Y-direction vector created by reduction, and a dictionary laser vector created by reducing the size of each of the reflection points specified by the laser radar with respect to the sample vehicle. It consists of a dictionary fusion vector obtained by fusion and dimensional compression by the eigenspace method,
The collation step derives a distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual traveling, and determines whether the distance is smaller than a predetermined threshold value. Is characterized by including a step of determining whether or not the vehicle is a forward vehicle.

According to such a configuration, whether or not the vehicle candidate area is a forward vehicle is determined from the distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual traveling, which is a complicated scene. However, it is possible to significantly reduce the frequency of erroneous recognition as in the case of the image sensor alone, and it is possible to prevent the error from expanding as in the case of the laser radar alone.

Further, the front vehicle recognition method according to the present invention includes a step of registering the fusion vector of the vehicle candidate area determined to be a front vehicle by the collation step as the dictionary fusion vector. It has a feature.

According to this structure, the accuracy of the dictionary fusion vector can be improved, so that the recognition accuracy of the preceding vehicle can be improved.

Further, in the forward vehicle recognition method according to the present invention, the collation step performs a motion estimation by a Kalman filter for the vehicle candidate area determined to be a forward vehicle,
It is characterized in that it includes a step used for the next matching process.

According to such a configuration, the movement of the vehicle candidate area determined to be a forward vehicle is predicted and used in the next matching process, so that more accurate vehicle recognition can be realized.

[0032]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described with reference to FIGS. However, FIG. 1 is a block diagram, and FIGS. 2 to 10 are operation explanatory diagrams.

As shown in FIG. 1, a laser radar module 1 including a scan laser radar, its scanning mechanism and a light receiver, and a monocular CCD as an image sensor.
Camera module 2 as an image processing unit including a camera
Is mounted on the vehicle and the laser radar module 1
The laser light is emitted in the horizontal direction in front of the vehicle, the reflected light from the reflection point is received, the positions of the plurality of reflection points are specified, and the front of the vehicle is imaged by the camera module 2. In addition, a dictionary fusion vector as a dictionary pattern, which will be described later, is stored in the memory 4 as a storage unit including a RAM and the like.

Further, the reflection point data specified by the laser radar module 1 and the picked-up image obtained by the camera module 2 are saved by the CPU 3 in another memory such as RAM which is not shown in FIG. . At this time, the data from the laser radar module 1 and the data from the camera module 2 may be stored in an area different from the storage area of the dictionary fusion vector in the memory 4.

The relationship between the coordinate systems of the laser radar module 1 and the camera module 2 is set, for example, as shown in FIG. That is, as shown in FIG. 2, the camera coordinate system takes the camera optical axis as the Z axis with the lens center as the origin, and takes the X axis and the Y axis parallel to the x axis and the y axis on the image plane. Here, the focal length f is known.

In the laser radar coordinate system, the light receiver is the origin, the laser optical axis is the ZR axis, the YR axis is set in the vertical direction, and the XR axis is set in the horizontal direction. XR YR ZR X from coordinate system
The YZ coordinate system can be converted by a composite matrix ωT of the rotation vector ω and the translation vector T, and as described later,
In the present invention, the coordinates of the reflection point specified by the laser radar module 1 are converted into camera coordinates for verification. However,
The composite matrix ωT has been calibrated in advance,
It is known.

Then, the CCD camera of the camera module 2 picks up an image of the front of the vehicle, and as shown in FIG. 12, as shown in FIG. 12, the camera module 2 makes a rectangular shape long in the horizontal direction (X direction) with respect to the picked-up image. Gaze area ROI
(Rectangular region of white frame in FIG. 12) is set, and the X direction (horizontal direction) from the shade of each pixel in the gazing region ROI.
Edge histogram EHx, an edge histogram EHy in the Y direction (vertical direction) orthogonal to the edge histogram EHx, and an edge histogram product EHxy in both directions are calculated.

At this time, since the rear glass of the vehicle and the vicinity of the rear bumper include many Y-direction edge components, the edge histogram EHy in the Y direction has a large value on the rear end surface of the vehicle, whereas the poles of the electric pole and the streetlight, For road structures such as guardrails, edge histogram E in the X direction
Hx increases, but edge histogram EH in the Y direction
y has a small value. Therefore, the edge histogram EHx in the X direction and the edge histogram EHy in the Y direction
By integrating and, the maximum points of the edge histogram other than the vehicle rear end surface can be suppressed to some extent, and conversely, the maximum points of the edge histogram of the vehicle rear end surface can be emphasized.

Therefore, the maximum point (xL, x) of the product EHxy of the edge histogram is preferentially calculated from the vehicle predicted position in the current frame calculated based on the vehicle recognized position in the previous frame.
R) is cut out as a vehicle rear end position candidate pair, and an edge histogram EVy is created by horizontally counting Y-direction edges Ey (see FIG. 12) equal to or larger than a predetermined threshold value in the section (xL, xR). A vehicle upper and lower end candidate (yU, yD) is obtained from the degree of proximity of the vehicle upper (lower) end in consideration of the inter-vehicle distance and the estimated vehicle width at the maximum point in EVy. By doing so, it is possible to cope with the vehicle vertical position shift due to the change in the road gradient to some extent. A rectangle defined by these four points is extracted as a vehicle candidate area.

Then, the CPU 3 causes the vehicle candidate area to be in the X direction (horizontal direction) and the Y direction (vertical direction).
An X-direction vector and a Y-direction vector as a collation pattern with a reduced size are created based on each of the edge histograms for the. X by such CPU3,
The process of creating the Y-direction vector corresponds to the first creating unit.

On the other hand, the laser radar module 1 irradiates the front of the vehicle with laser light, and at that time, the scanning mechanism scans the laser light horizontally at a predetermined angle (for example, 0.1 °), and the front of the vehicle is scanned. The distance from the subject vehicle to the preceding vehicle traveling in the same lane is detected from the time from the irradiation of the laser light from the reception of the reflected light from the target object to the reception of the reflected light. . One like this
The time required to detect the vehicle-to-vehicle distance is about 100 ms
This is a short time, and this detection operation is repeated at regular intervals.

Further, when a plurality of reflection points on the object are specified by the laser radar module 1 by such an operation, as shown in FIG. 3, for example, the received light intensity histogram (FIG. 3) of each of the specified reflection points is shown. A white list in 3) is formed, and the CPU 3 creates a laser vector as a distribution pattern with a reduced size based on the received light intensity histogram. At this time, in the case of the reflection point from the middle and long distances, since the number of laser beams received is small, the received light intensity list value is missing.Therefore, the missing position is compensated with the average value of the received light intensity around the left and right positions to some extent. It is desirable to make the shape closer to the received light intensity model at that distance. Such a laser vector creation process by the CPU 3 corresponds to a second creation unit.

By the way, a plurality of dictionary fusion vectors as dictionary patterns are stored in the memory 4 as described above, and the dictionary fusion vectors are obtained as follows. For example, various kinds of sample vehicles as shown in FIG. 4 are prepared in advance, and based on the edge histogram EHx in the X direction and the edge histogram EHy in the Y direction derived by the camera module 2 for the sample image, the CPU 3 Thus, the reduced size X-direction vector hx_ for dictionary and the Y-direction vector hy_ for dictionary are created, and based on the received light intensity histogram created for each reflection point specified by the laser radar module 1 for various sample vehicles, The CPU 3 creates the reduced-size dictionary laser vector lp_, and the dictionary X-direction and Y-direction vectors hx_,
The hy_ and the dictionary laser vector lp_ are fused at the stage of raw data and dimensionally compressed by applying the eigenspace method to obtain the fusion vector v_.

At this time, the X-direction and Y-direction vectors hx_ and hy_ for the dictionary are expressed as follows: hx _ = [hx1, hx2, hx3, ..., Hxm] ... (1) hy _ = [hy1, hy2, hy3, ..., hym] ... (2) and the dictionary laser vector lp_ is defined as lp _ = [lp1, lp2, lp3, ..., Lpn] (3), the fusion vector v_ is v _ = [hx_, hy_, lp_] ^T It can be defined as (4). Here, for the dictionary laser vector lp_, N models of various distances are prepared. That is, the fusion vector v_ is d-dimensional (2 × m +
n), and the number of samples is M × N.

The covariance matrix Q of the fusion vector v_ is expressed as follows: Q = E {(v_-m _) (v_-m_) ^T } (5), where m_ in the equation (5) is v_ It is an average vector, and m_ = E {v _} ... (6). Here, the eigenvalue problem of Qej = λj ej (7) is solved. Assuming that the created dictionary is k-dimensional, the eigenvector e_ corresponding to k large eigenvalues is defined by e _ = {e1_, e2_, e3 _, ..., ek _} ... (8). The value of k is the minimum value at which the cumulative contribution rate α represented by the formula α = Σ ^k _{j = 1} λj / Σ ^d _{j = 1} λj (9) becomes 0.9 or more.

Furthermore, the d-dimensional fusion vector v_
Can be compressed into a k-dimensional principal component vector u_ by the equation u _ = [e1_, e2_, e3 _, ..., ek_] ^T v _... (10) using the eigenvector e_ represented by the equation (8). The main component vector u_ can be calculated in advance for all prepared fusion vectors v_, and registered and stored in the memory 4 as a dictionary fusion vector.

Then, the above-mentioned dictionary X-direction and Y-direction vectors hx_, hy_ and the dictionary laser vector lp_.
In the same way as the above, the CPU 3 creates an X direction vector, a Y direction vector, and a laser vector for the vehicle candidate area at the time of actual traveling, and these are fused at the stage of raw data, and the eigenspace method is applied. Dimensionally compressed to create a fusion vector as a fusion pattern. Here, in order to fuse the image data and the laser data, it is premised that the detection range of the CCD camera and the detection range of the laser radar overlap each other as shown in FIG. The fusion vector creation process by the CPU 3 corresponds to the fusion unit.

Further, the CPU 3 collates the fusion vector at the time of actual traveling with the dictionary fusion vector stored in the memory 4, and judges whether the vehicle rear area corresponding to the fusion vector at the actual traveling is a front vehicle. Is done. At this time, the CPU 3 projects the fusion vector of the vehicle candidate area during actual traveling onto the vehicle eigenspace by the above equation (10), derives the distance from the dictionary fusion vector, and Euclidean of the distance in the feature space. If the distance is smaller than a predetermined threshold value, it is determined that the vehicle candidate area is a forward vehicle, and if the distance is large, it is determined that the vehicle candidate area is a non-vehicle. In this way, the series of matching processing by the CPU 3 including the creation of the fusion vector corresponds to the matching unit.

Further, the CPU 3 registers the fusion vector of the vehicle candidate area determined to be a vehicle ahead in the memory 4 as a dictionary fusion vector.
By doing so, the accuracy of the dictionary fusion vector can be increased, and the recognition accuracy of the front vehicle can be improved.

Further, with respect to the vehicle candidate area determined to be the front vehicle, the CPU 3 performs the motion prediction by the Kalman filter, which is used for the next collation processing, thereby realizing more accurate vehicle recognition. You can do it.

Next, the vehicle recognition processing procedure by the CPU 2 will be described with reference to the flowchart shown in FIG. As shown in FIG. 6, regarding the gazing area in the image in front of the vehicle captured by the CCD camera while the vehicle is traveling,
The edge histograms EHx, EHy, EHxy are calculated by the camera module 2 as described above (S1),
The maximum point of the edge histogram EHxy is obtained,
The vehicle candidate area is extracted based on the maximum point and the previous recognition result.

Further, as shown in FIG. 6, the laser radar module 1 identifies the reflection point of the laser light and derives the received light intensity histogram (S2).
An X, Y direction vector and a laser vector for the vehicle candidate area are created, and a fusion vector is created by fusing these vectors (S3) and stored in the memory 4 and the fusion vector of the vehicle candidate area. The dictionary fusion vectors are collated (S4).

Then, it is judged whether or not the vehicle candidate area is a vehicle depending on whether or not there is a dictionary fusion vector whose distance from the fusion vector of the vehicle candidate area is equal to or less than a predetermined threshold value (S5). If the determination result is NO, the vehicle candidate area is rejected as a non-vehicle (S6), and if the determination result is YES,
The vehicle candidate area is predicted to be a vehicle (S7), and is used, for example, for control when following the vehicle while keeping the inter-vehicle distance with the preceding vehicle at a predetermined value. Note that after step S7, the process returns to step S3 described above, and from the next frame, the rectangular area is preferentially tracked by the Kalman filter.

In this way, if images such as those shown in FIGS. 7 to 9 are taken as the continuous scenes during actual traveling, the laser reflection points from the vehicle and the wall which is the roadside structure are shown in FIG. In a close scene, clustering is difficult with the laser radar alone as in the conventional case, and it is difficult to cut out the vehicle part with the image sensor alone, which causes deterioration of the distance accuracy. According to FIG. 7, it can be seen that the vehicle recognition is successful, as indicated by the white rectangle in FIGS. Further, as shown in FIG. 9, even when the guardrail exists within the laser detection range, the laser radar alone erroneously recognized the guardrail as a vehicle as in the conventional case. According to the method, it can be seen that the vehicle recognition is successful.

Further, FIG. 10 shows a comparison of vehicle reliability R for the same vehicle. The solid line shows the case of fusion in the present invention, the dotted line shows the case of the conventional image only, and the method of the present invention is more reliable. It can be seen that the degree R is high.

Therefore, according to the above-described embodiment, at the time of actual traveling, it is based on the X and Y direction vectors based on the edge histograms derived by the image processing unit and the received light intensity histograms of the respective reflection points specified by the laser radar. The fusion vector obtained by fusing the laser vector is collated with each dictionary fusion vector stored in the memory 4 to determine whether or not the vehicle is a forward vehicle. It is possible to prevent the reflection from a vehicle or a roadside object from being included in the reflection from a front vehicle, and further reduce the frequency of erroneous recognition of the vehicle in a complicated scene such as in the case of an image sensor. It can be recognized with accuracy.

Further, by learning, the accuracy of the dictionary fusion vector can be increased, and the recognition accuracy of the front vehicle can be improved.

In the above embodiment, the case where the monocular CCD camera is used as the image sensor has been described, but it goes without saying that the image sensor is not limited to the monocular CCD camera described above. Further, the storage unit is not limited to the memory 4 including the RAM described above.

The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention.

[0060]

As described above, according to the first and sixth aspects of the invention, the laser radar excels in distance measurement, and the image sensor can accurately distinguish between the vehicle and the other parts. The collation pattern based on the edge histogram derived by the processing unit and the distribution pattern of each reflection point specified by the laser radar are fused at the stage of raw data and dimensionally compressed by the eigenspace method. It is checked whether or not it substantially matches any one of the dictionary patterns of, and it is judged whether or not it is a front vehicle.Therefore, as in the case of laser laser alone, reflection from vehicles other than the front vehicle or roadside objects etc. It is possible to prevent it from being included in the reflection from the vehicle in front, and reduce the frequency of erroneous recognition of the vehicle in complicated scenes such as in the case of an image sensor. It is possible to recognize the accuracy, it is possible to provide a more comfortable driving assistance system is more sophisticated.

According to the second and seventh aspects of the invention, the distribution pattern obtained from the X-direction vector and the Y-direction vector as the collation pattern obtained by the image processing and the received light intensity histogram of the reflection point by the laser radar. Since the fusion vector is combined with the laser vector as dimensional compression by the eigenspace method to form a fusion vector as a fusion pattern, it is possible to dimensionally compress the fusion of the data at the sensor raw data stage of each laser and image. Thus, the accuracy of pattern recognition can be improved, and the front vehicle can be recognized with high accuracy even in a complicated scene.

Further, according to the invention described in claims 3 and 8, it is determined whether or not the vehicle candidate area is a front vehicle from the distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual traveling. Therefore, even in complicated scenes, it is possible to significantly reduce the frequency of false recognition as in the case of the image sensor alone, and
It is possible to prevent the error from expanding as in the case of the laser radar alone.

Further, according to the inventions of claims 4 and 9, it is possible to improve the recognition accuracy of the preceding vehicle by increasing the accuracy as the dictionary fusion vector.

Further, according to the fifth and tenth aspects of the present invention, the movement of the vehicle candidate area judged to be a forward vehicle is predicted and used for the next matching process, so that more accurate vehicle recognition is realized. It will be possible.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 3 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 4 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 5 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 6 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 7 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 8 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 9 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 10 is an operation explanatory diagram of the embodiment of the present invention.

FIG. 11 is an explanatory diagram of an operation of detecting a reflection point by the scan laser radar which is the background of the present invention.

FIG. 12 is an explanatory diagram of a vehicle recognition operation by the image sensor which is the background of the present invention.

[Explanation of symbols]

1 Laser Radar Module 2 Camera Module (Image Processing Unit) 3 CPU (Collision Unit, First Creating Unit, Second Creating Unit,
Fusion part) 4 memory (storage part)

Front page continuation (51) Int.Cl. ⁷ identification code FI theme code (reference) G06T 7/00 300 G06T 7/00 300G H04N 7/18 H04N 7/18 J // G08G 1/16 G08G 1/16 E F-term (reference) 5B057 AA16 BA02 DA08 DB02 DB09 DC23 DC36 5C054 FC12 FC14 FC16 HA30 5H180 AA01 CC03 CC04 CC14 5L096 AA06 BA04 CA02 EA27 FA06 FA36 FA64 FA66 GA51 HA07 JA11

Claims (10)

[Claims] 1. A front vehicle recognition device for recognizing a vehicle traveling in front of the host vehicle, comprising: an image sensor for capturing an image of the front of the host vehicle; An image processing unit that derives an edge histogram, which is the density projection value of each pixel in a predetermined direction in the area, and irradiates the front of the vehicle while scanning the laser light horizontally while receiving the reflected light from the reflection point. A laser radar for specifying the positions of a plurality of the reflection points, a pattern of the edge histogram derived by the image processing unit for sample images of various sample vehicles prepared in advance, and the laser for each sample vehicle Multiple dictionary patterns obtained by fusing multiple reflection point patterns specified by radar and dimensionally compressing them by the eigenspace method An eigenspace method combining the stored storage unit, the matching pattern based on the edge histogram derived during actual traveling by the image processing unit, and the distribution pattern of each reflection point specified during actual traveling by the laser radar. And a collating unit for collating whether or not the fusion pattern formed by dimensionally compressing by applying the above-mentioned substantially matches any one of the dictionary patterns stored in the storage unit to determine whether the vehicle is a forward vehicle. A front vehicle recognition device characterized in that. 2. The image processing unit, an edge histogram in the X direction of the screen from the density of each pixel in the gaze area set for the captured image, an edge histogram in the Y direction of the screen orthogonal thereto, A vehicle candidate region is extracted by calculating a product of edge histograms in both directions and deriving a maximum point thereof, and the collating unit calculates the X direction and the Y direction in the vehicle candidate region by the image processing unit. A first creation unit that creates a reduced-size X-direction vector and a Y-direction vector as the matching patterns based on the respective edge histograms, and each of the reflection points specified in the vehicle candidate area by the laser radar. A received light intensity histogram is formed, and based on the received light intensity histogram, the laser vector with the reduced size is forwarded. A second creation unit that creates a distribution pattern, and a fusion vector of the vehicle candidate region that is dimensionally compressed by applying an eigenspace method based on the X-direction vector, the Y-direction vector, and the laser vector in the vehicle candidate region. The front vehicle recognition device according to claim 1, further comprising a fusion portion formed as the fusion pattern. 3. The size of the dictionary pattern is calculated by the first creating unit based on the edge histograms in the X direction and the Y direction derived by the image processing unit with respect to the sample image of each sample vehicle. The X-direction and Y-direction vectors for the dictionary and the reflection points specified by the laser radar for the sample vehicle are created by reducing the size by the second creating unit. The dictionary laser vector is composed of a dictionary fusion vector obtained by being fused by the fusion unit and dimensionally compressed by the eigenspace method, and the collating unit, the fusion vector of the vehicle candidate region during actual traveling, and the Derive the distance to the dictionary fusion vector and see if the distance is less than a predetermined threshold. The front vehicle recognition device according to claim 2, wherein whether or not the vehicle candidate area is a front vehicle is determined based on the above. 4. The front according to claim 3, wherein the fusion vector of the vehicle candidate area determined to be a forward vehicle by the matching unit is registered in the storage unit as the dictionary fusion vector. Vehicle recognition device. 5. The collation unit performs motion estimation by a Kalman filter for the vehicle candidate area determined to be a forward vehicle, and uses it in the next collation processing. Forward vehicle recognition device. 6. A front vehicle recognition method for recognizing a vehicle traveling in front of the host vehicle, wherein a predetermined gaze area is set for a captured image by an image sensor, and the density of each pixel in the gaze area in a predetermined direction is set. Derivation step of deriving the edge histogram which is the projection value, and by the laser radar, irradiates the front of the vehicle while scanning the laser light in the horizontal direction and receives the reflected light from the reflection point to receive a plurality of the reflection points. The identification step of identifying the position, the matching pattern based on the edge histogram derived at the time of actual traveling, and the distribution pattern of each reflection point identified at the time of actual traveling by the laser radar are fused, and the dimension is compressed by the eigenspace method. The fusion pattern is a pattern of the edge histogram derived from the sample image of various sample vehicles prepared in advance. And a pattern of a plurality of reflection points specified by the laser radar are combined and compressed by the eigenspace method. A method of recognizing a forward vehicle, including a collation step of determining whether the vehicle is a vehicle. 7. The edge histogram in the X direction of the screen from the density of each pixel in the gaze area set for the captured image in the deriving step, the edge histogram in the Y direction of the screen orthogonal to this, and both directions. To a vehicle candidate area by deriving a maximum point of the edge histogram of the vehicle candidate area and deriving a maximum point of the edge histogram in the X-direction and the Y-direction edge derived in the vehicle candidate area. Creating a reduced-size X-direction vector and a Y-direction vector as the matching pattern based on each histogram, and forming a received light intensity histogram of each reflection point specified in the vehicle candidate region by the laser radar, Based on the received light intensity histogram, the reduced size laser vector and the distribution pattern A step of creating Te, the X-direction vector in the vehicle candidate region, Y
7. A front vehicle according to claim 6, further comprising the step of forming a fusion vector of the vehicle candidate region, which is dimensionally compressed by applying an eigenspace method, as the fusion pattern, based on a direction vector and a laser vector. Recognition method. 8. The dictionary X-direction is created by reducing the size of the dictionary pattern based on the derived edge histograms in the X-direction and the Y-direction with respect to the sample image of each sample vehicle. And the Y direction vector,
For the sample vehicle, from the dictionary fusion vector obtained by fusion with a dictionary laser vector created by reducing the size of each reflection point specified by the laser radar and dimensionally compressed by the eigenspace method The matching step derives the distance between the fusion vector of the vehicle candidate area and the dictionary fusion vector during actual traveling, and determines whether the distance is smaller than a predetermined threshold value. The front vehicle recognition method according to claim 7, further comprising a step of determining whether the candidate area is a front vehicle. 9. The front side according to claim 8, further comprising a step of registering the fusion vector of the vehicle candidate area determined to be a front vehicle as the dictionary fusion vector by the matching step. Vehicle recognition method. 10. The collation step includes a step of performing a motion estimation by a Kalman filter for the vehicle candidate area determined to be a forward vehicle, and using it in the next collation processing. The method for recognizing a forward vehicle according to item 1.