DE102013203300A1 - Image processing apparatus for use in vehicle control system for detecting pedestrian captured by camera, has object region estimating unit estimating object region in which object is present on basis of partial regions - Google Patents

Abstract

The apparatus (10) has a sub-model storing unit (61) for storing multiple models, each provided with corresponding parts of an object. A partial region extracting unit (51) extracts sub-regions for corresponding parts of an object from an image using corresponding sub-models, which are stored in the sub-model storing unit. An object region estimating unit estimates an object region in which the object is present on the basis of the partial regions extracted by the partial region extracting unit. An independent claim is also included for a vehicle control system.

Description

BACKGROUND

(Technical field)

The present invention relates to methods of detecting a pedestrian in an image captured by a vehicle camera.

(State of the art)

In order to avoid a collision with an object interfering with a running of a vehicle, methods for detecting a pedestrian in an image taken by a vehicle camera have been proposed. The methods include a method of detecting a pedestrian in a captured image by applying an object discrimination method based on neural networks using an pedestrian outline pattern to an object captured in the image.

However, in such a discrimination method, an object having an outline pattern similar to the pedestrian outline, such as a tree or the like, may be incorrectly detected as a pedestrian, which may result in lower accuracy in detecting a pedestrian.

To overcome such a disadvantage, a known method, as described in the Japanese Patent Application Laid-Open Publication No. 2008-21034 discloses pattern matching for each of specific parts constituting a pedestrian to output a degree of similarity, and determines whether or not a sum of output values for the respective specific parts is greater than a predetermined limit, thereby to make a pedestrian to capture. The specific parts of the pedestrian may include, but are not limited to, a head, a body, arms and legs.

However, there is the disadvantage of the method used in the Japanese Patent Application Laid-Open Publication No. 2008-21034 is disclosed in that a pedestrian behind another pedestrian or a pedestrian behind an object (other than a pedestrian) may not be detected accurately. In other words, when some of the specific parts such as a head, a body, hands, legs or the like of the pedestrian are invisible in a captured image, output values for the invisible parts decrease, resulting in a decrease in a sum of output values for the specific ones Parts results. As a result, detection of a pedestrian or pedestrians may fail based on the sum of output values for the specific parts. A similar situation may also occur when capturing an object (other than a pedestrian) in a captured image.

In view of the foregoing, therefore, it would be desirable to have an image processing apparatus and vehicle control system using the same which can reliably detect an object.

OVERVIEW OF THE INVENTION

According to an embodiment of the present invention, there is provided an image processing apparatus for detecting an object on the basis of an image captured by an image pickup unit mounted in a vehicle.

In the apparatus, a sub-model storing means has a plurality of sub-models stored therein and corresponding to parts of the object, respectively. If the object is a pedestrian, the submodels include, but are not limited to, a head head model, a left / right arm model, a hip hip model, and a right / left leg model. If the object is a vehicle, the submodels include, but are not limited to, a tire part model for a tire, a tail lamp sub model for a tail light, and a window sub model for a window.

A partial region extracting means extracts partial regions for the corresponding parts of the object from the image using the respective respective partial models stored in the partial storage means. Moreover, an object region estimating means estimates an object region in which the object exists on the basis of the partial regions extracted by the partial region extracting means.

With this configuration, for each of the parts of the object, the partial regions are extracted from the image using the corresponding partial model, and the object region is estimated on the basis of the extracted partial regions. For example, even if only the pedestrian right / left leg partial region is extracted from the image using the leg part model, the pedestrian region is estimated based on the right / left leg partial region. This leads to a correct capture of the object in the image.

BRIEF DESCRIPTION OF THE DRAWINGS

Show it:

1 a schematic block diagram of a vehicle control system according to an embodiment of the present invention;

2 a flowchart of a pedestrian detection processing;

3A an example of extracting high-valued regions by using a partial model;

3B an example of extracting high-valued regions using a reduced-size submodel;

3C highly valued areas of different sizes;

4A relative positions of specific parts of several persons;

4B an example for setting a threshold for ratings in a rating record for each of specific parts of a pedestrian;

5A an example of grouping high score areas into subgroup regions by distances therebetween and sizes thereof;

5B an example of grouping high scoring areas in subgroup regions by distances between them;

5C an example of grouping high score areas into subgroup regions by distances between them and calculating a density of high score areas of each subgroup region;

6 an example of extracting a sub-region based on sub-group regions and calculating an evaluation of the sub-region;

7A an example of extracting a pedestrian region based on a partial region;

7B an example for linking different subregions together; and

7C an example of estimating a pedestrian region based on subregions that are linked together.

DESCRIPTION OF SPECIFIC EMBODIMENTS

The present inventions will be explained more fully hereinafter with reference to the drawings. Like reference numerals refer to like elements.

A vehicle control system 1 The present embodiment is provided with an image processing apparatus 10 and a vehicle control device 20 configured. The image processing device 10 is with a distance measuring unit 31 and an image capture unit 32 connected. The vehicle control device 20 is with a speaker 41 , a brake 42 and a steering 43 connected.

The image processing device 10 and the vehicle control device 20 are each configured as a microcomputer. The image processing device 10 captures a pedestrian in an image taken by the image capture unit 3 is received, and determines a collision risk based on distance information provided by the ranging unit 31 Will be received. A vehicle control device 20 represents by means of the loudspeaker 41 in response to the risk of collision, that of the image processing device 10 a notification is provided and / or causes in the presence of a large degree of urgency, an avoidance control using the brake 42 and the steering 43 ,

The distance measuring unit 31 may be formed by a laser radar unit, a millimeter-wave radar unit or the like. The distance measuring unit 31 detects a distance between a vehicle in which the vehicle control system 1 is mounted (hereinafter referred to as the own vehicle), and an object in front of the own vehicle.

The image capture unit 32 may be formed by a molecular camera or the like. The image capture unit 32 takes a picture in front of the own vehicle. The image processing device 10 captures a pedestrian in the captured image.

The speaker 41 issues an audio warning message such as "pedestrian in front of the vehicle" or "attention!" or the like when a risk of collision with a pedestrian is increased.

The brake 42 includes a brake driver for decelerating the own vehicle in response to a signal from the vehicle control device 20 , The steering 43 provides an electric power steering (EPS) system for assisting a driver in steering in response to a signal from the vehicle control device 20 , The vehicle control using the brake 42 and the steering 43 is impaired when the risk of a Collision with the pedestrian is further increased, that is, in the presence of a high degree of urgency.

The image processing device 10 includes a controller 50 and a memory 60 , The controller 50 may be formed by a microcomputer. The controller 50 includes as functional blocks a pedestrian detection section 51 and a collision risk determination section 52 (Pedestrian region estimating means). The memory 60 may be formed by a random access memory (RAM), a flash memory, a hard disk drive (HDD), or the like. The memory 60 includes a partial model storage section 61 (Partial model storage means) and a collision risk decision table storage section 62 ,

The pedestrian detection section 51 (Partial region extracting means and object region estimating means) detects a pedestrian in an image taken by the image pickup unit 32 Will be received. As explained in more detail below, the pedestrian detection section detects 51 specific parts of a pedestrian by scanning the image and appreciates the presence of the pedestrian and his position.

The collision risk determination section 52 determines a collision risk as a function of a distance between the subject vehicle and the pedestrian by the pedestrian detection section 51 is detected. The distance between the own vehicle and the pedestrian is determined by the distance measuring unit 31 measured.

The submodule storage section 61 stores submodels, each of which stores a corresponding one of the specific parts of the pedestrian. Each submodel is used to detect a distribution of the corresponding part of the pedestrian in the image. In the present embodiment, the specific parts of the pedestrian include, but are not limited to, a head, a right arm and a left arm, a hip, and right and left legs. A partial model modeling the head is hereinafter referred to as a "head part model", a partial model modeling the right / left arm is hereinafter referred to as an "arm part model", a partial model modeling the hip is hereinafter referred to as a "hip part model And a sub model modeling the right / left leg is hereinafter referred to as a "leg part model". In such a partial model approach for detecting a pedestrian, for example, a feature vector represents features (outline information) for the corresponding specific parts of the pedestrian.

The collision risk decision table storage section 62 stores a collision risk decision table, which is a table defining a correspondence relationship between a distance between the own vehicle and the detected pedestrian and a collision risk. The collision risk determination section 52 determines a collision risk with respect to the collision risk decision table included in the collision risk decision table storage section 62 is stored.

The vehicle control device 20 includes a lesson controller 70 and a vehicle controller 80 each of which may be formed by a microcomputer. The information controller 70 provides a notification by means of the loudspeaker 41 ready when the risk of collision increases. The vehicle controller 80 causes collision avoidance control or collision avoidance control using the brake 42 and the steering 43 if the risk of collision continues to rise.

Hereinafter, a pedestrian detection processing performed in the pedestrian detection section 51 is executed, explained in detail. The pedestrian detection processing is repeatedly executed every predetermined time interval.

As in 2 is shown, a captured image from the image pickup unit at step S110 32 receive.

Subsequently, at step S120, an evaluation record is generated, and high-ranking areas are extracted from the image with respect to the evaluation record. As described above, the submodels that model the corresponding specific parts of the pedestrian are in the sub-model storage section 61 saved. The evaluation record is generated by scanning the captured image of the submodel. For each of the submodels, regions having a rating greater than a predetermined threshold (high score regions) are extracted from the image.

In particular as in 3A That is, the header model TP1 is successively shifted (or moved) at predetermined intervals along a right direction from an upper left corner to an upper right corner of the image. When the header model TP1 reaches the upper right corner of the image, the header model TP1 is returned to the upper left corner of the image and then shifted downward by a predetermined distance. The header model TP1 is again successively at predetermined intervals along the right direction from the left edge to the right edge of the image. When the header model TP1 reaches the right edge of the image, the header model TP1 is returned to the left edge of the image again and then shifted downward by the predetermined distance. Such successive shifts are continued until the header model TP1 reaches a lower right corner of the image. A similarity degree of an area overlapping the header model TP1 at a position after each shift is stored as an evaluation of the area, resulting in an evaluation record (not shown) for the header model TP1.

In general, a pedestrian appears relatively small in the image at a relatively spaced distance. To detect such a pedestrian, which appears relatively small in the image, as in 3B Further, the image is scanned using a reduced-size header model TP2 in a similar manner as described above, wherein a similarity degree of a region overlapping the header model TP2 of a position after each shift is stored as an evaluation of the region an evaluation record (not shown) for the header model TP2 leads.

In the present embodiment, the image is subjected to the two-level image scanning, wherein in the first stage the image is scanned using the headboard model TP1 for detecting a pedestrian at a relatively close distance to the own vehicle, and in the second stage the image is scanned using the headboard model TP2, which is smaller than the header model TP1 is scanned to detect a pedestrian at a relatively large distance to the own vehicle. In practice, image scans with three or more levels can be used to detect pedestrians at different distances to the subject vehicle. Moreover, in such a multi-level image scan, it is possible to resize only detail models while keeping the size of the image unchanged, or only to resize the image while leaving the size of the submodel unchanged.

After generating such an evaluation record, high-valued regions having a rating greater than a predetermined threshold are extracted from the image with respect to the evaluation record. For example, as in 3C is shown, the high-valued regions TC and the high-valued regions TT are extracted from the image, each high-valued region TC is a high-valued region extracted by scanning the image with the header model TP1, and each high-valued region TT is a high-valued region, which is extracted by scanning the image with the header model TP2.

In the present embodiment, the image is further scanned using the other part models, that is, the arm part model, the hip part model, and the leg part model. Highly valued ranges for each of these submodels are used in a similar way as in 3C also extracted from the image with respect to the evaluation record.

Now, an example of setting the limits for extracting the high-valued ranges for the sub-models will be explained. As in 4A 2, two people J1 and J2 appear in the image, with one of the arms and one of the legs of the background person J1 partially hidden behind one of the arms and one of the legs of the foreground person J2. In the light of the fact that in the present embodiment, some of the specific parts are less visible than the other parts, as in FIG 4B 3, predetermined limits for extracting less visible portions such as arms L1 and L2 and legs N1 and N2 are set smaller than predetermined limits for extracting more visible portions such as the head K and the hip M.

According to 2 At step S130, partial group regions are extracted from the image, and the high-valued regions extracted at step S120 are grouped into the divided-group regions.

In particular, as in 5A is shown, the high-scoring areas TC and TT are grouped into the sub-group regions by distances between the high-scoring areas TC and TT and sizes of the high-scoring areas TC and TT. A more distant pedestrian (smaller distance) and a closer pedestrian (larger distance) can be uniquely extracted from the image using the sizes of the high-valued areas TT and TC, resulting in the improved extraction accuracy. 5A Fig. 15 shows a subgroup region G1 grouping together the high-ranking regions TC and a subgroup region G2 grouping the high-valued regions TT together. The term "distance" refers to a two-dimensional (2D) distance in the image or a three-dimensional (3D) distance in a real space.

As in 5B 1, the subgroup regions may be extracted based only on distances between the high-valued regions TC and TT, which may advantageously result in relatively simpler extraction processing. Alternatively, the subgroup regions may be based are only extracted to sizes of the high-valued regions TC and TT, which can also advantageously lead to a relatively simpler extraction processing.

In addition, the subgroup regions can be extracted from the image based on their densities from highly valued regions. This can cause a subgroup region that has a relatively low density to be detected incorrectly. In particular, as in 5C is shown, the high-valued areas TC extracted in step S120 are grouped into two subgroup regions G4 and G5 by distances between the high-valued areas TT. For each of the sub-group regions G4 and G5, a density of high-valued regions of the sub-group region is calculated by dividing the number of included high-valued regions TT by an area of the sub-group region. When the density of high-scoring regions of the subgroup region is greater than a predetermined threshold, the subgroup region is qualified as a subgroup region. When the density of high-scoring regions of the subgroup region is equal to or smaller than the predetermined threshold, the subgroup region is disqualified as a subgroup region.

The subgroup regions may be extracted from the image based on their overall scores, in addition to or instead of the densities of high scoring regions of the subgroup regions. This can prevent a subgroup region with a relatively low overall score from being detected incorrectly. In particular, as in 5C is shown, the high-valued areas TT extracted in step S120 are grouped into the subgroup regions G4 and G5 by distances between the high-valued areas TT. For each of the subgroup regions G4 and G5, an overall score of the subgroup region is calculated by summing up the scores of the high scoring areas TT included therein. If the overall score of the subgroup region is greater than a predetermined threshold, the subgroup region is qualified as a subgroup region. If the overall score of the subset region is equal to or less than the predetermined limit, the subset region is disqualified as a subset region.

According to 2 In step S140, subgroup regions are extracted from the image, wherein each subgroup region in the present embodiment is a rectangular circumscribed region that includes the subgroup regions extracted in step S130 that are close to each other. A distance between a pair of sub-group regions may be a minimum distance between boundaries of the respective sub-group regions, a distance between centers of the respective sub-group regions, or a distance between evaluation centers of the respective sub-group regions. A pair of subgroup regions may be determined to be close to each other if the distance between them is below a predetermined threshold. 6 shows an example of an extraction of a partial region P, which includes two subgroup regions G6 and G7.

Subsequently, at step S150, a score of each partial region extracted in step S140 is calculated. Specifically, a score of each subregion is calculated by summing up scores of the respective subgroup regions included in the subregion, wherein an score of each subgroup regions is calculated by summing up scores of the respective high scoring ranges included in the subgroup region. Example becomes in 6 A sum of scores of the respective high-scoring regions included in each of the sub-group regions G1 and G2 is calculated, and then a sum of scores of the respective sub-group regions G1 and G2 is calculated, resulting in an evaluation of the sub-region P. The evaluation of the subregion P can thus be calculated in such a relatively simple manner.

For each of the subgroup regions G1 and G2, the score of the subgroup region may be calculated not only based on the scores of the corresponding high scoring areas included in the subgroup region, but also on the proximity (proximity) between the high scoring areas that are included in the subgroup region. In particular, as in a lower part of 6 3, the spatial proximity between the high-scoring regions of each subgroup region may be calculated by an inverse of a sum of absolute positional differences between the high scoring regions, where the position of each high scoring region may be a middle or upper left position thereof. The score of each of the subgroup regions G1 and G2 may be given by a product or a sum of the sum of scores of the respective high scoring areas in the subgroup region and the spatial proximity between the high scoring areas in the subgroup region.

For each of the subgroup regions G1 and G2, the score of the subgroup region may be calculated not only based on the scores of the respective high scoring regions included in the subgroup region, but also based on a similarity in size between the high scoring scopes described in of the Subgroup region is included, in addition to or instead of the spatial proximity between the high-valued areas that are included in the subgroup region. In particular, as in a lower part of 6 2, the similarity in size between the high-valued regions of each subgroup region may be defined by an inverse of a sum of absolute differences in sizes between the high-valued regions, where the size S of each high-valued region may be a length or a width thereof (assuming that each highly valued area is rectangular or square). The score of each of the subgroup regions G1 and G2 may be given by a product or a sum of the sum of scores of the respective high scoring areas in the subgroup region and the similarity in size between the high scoring areas in the subgroup region (and in some embodiments the spatial proximity between the highly valued areas and the subgroup region).

For each of the subgroup regions G1 and G2, the evaluation of the subgroup region may be based not only on the scores of the respective high scoring regions included in the subgroup region, but also based on the density of high scoring regions of the subgroup region as described above instead of the similarity in size between the high-valued regions included in the subgroup region and / or the spatial proximity between the high-valued regions included in the subgroup region, are calculated. The score of each of the subgroup regions G1 and G2 may be represented by a product or a sum of the sum of scores of the respective high scoring areas in the subgroup region and the density of high scoring areas (and in some embodiments, the spatial proximity between the high scoring areas of the subgroup region and / or the Similarity in size between the highly valued areas in the subgroup region).

Subsequently, at S160, pedestrians are detected by extracting pedestrian regions each of which has at least one pedestrian present based on the partial regions extracted at S140. Knowledge of types of submodels used to extract the subregions and sizes of the high scoring areas included in each subregion allows the pedestrian regions to be extracted from the image.

In particular, according to 7A It can be assumed that a pedestrian body section exists below a partial region P which is extracted in step S140. A size of the pedestrian body portion can be estimated from the sizes of the respective high-valued areas included in the partial region P, allowing the pedestrian region S to be estimated. When the score of the partial region P is equal to or smaller than a predetermined threshold, the pedestrian region S is disqualified as a pedestrian region.

In particular, as in 7B 12, the pedestrian regions are detected on the basis of relative positions of two partial regions P1 and P3 extracted using the header model and a partial region P2 extracted using the leg partial model. For example, when the high-valued regions included in the partial region P1 and the high-valued regions included in the partial region P2 have the same size, it is determined that the partial regions P1 and P2 are related to each other. That is, it is assumed that the sums P1 and P2 respectively include heads and legs of pedestrians belonging to the same pedestrian group. However, the high-valued regions included in the partial region P3 differ in size from the high-valued regions in the partial regions P1 and P2. In the absence of a partial region associated with the partial region P3, it can be determined that the partial region P3 is erroneously detected.

As in 7C is illustrated, it is assumed that the two subregions P1 and P2 are linked together. A rectangular circumscribed region including two pedestrian regions S1 and S2 estimated from the partial regions P1 and P2, respectively, is estimated as a pedestrian region S3. When a sum of scores of the respective partial regions P1 and P2 is above a predetermined threshold, the pedestrian region S3 is qualified as a pedestrian region. When the sum of ratings of the respective partial regions P1 and P2 is equal to or smaller than the predetermined threshold, the pedestrian region is disqualified as a pedestrian region.

As described above, the partial model storage section (as partial model storage means) of the image processing apparatus 10 several sub-models stored therein and each of which models a corresponding one of the specific parts (features) of the pedestrian. The specific parts in the present embodiment include a head K, a hip M, right and left arms L1 and L2, and left and right legs N1 and N2, respectively, as shown in FIG 4B is shown. The pedestrian detection section 51 (when Partial region extracting means and object region estimating means) of the controller 50 receives a captured image from the image capture unit 32 (in step S110 of FIG 2 ). Subsequently, the pedestrian detection section generates 51 an evaluation record using the partial models stored in the memory section 61 are stored, and for each partial model, extract the high-valued regions from the image with respect to the evaluation record (in step S120). Afterwards, the pedestrian detection section extracts 51 the subgroup regions (in step S130) and extracts the subregions on the basis of the subgroup regions (in step S140). The pedestrian detecting section calculates the scores of the partial regions (at step S150), and estimates the pedestrian regions of which at least one pedestrian (pedestrian group) is present in each one (at step S160).

The pedestrian detection section 51 extracts the subregions using the submodels for the respective parts of the pedestrian and estimates the pedestrian regions based on the subregions. For example, if there is only a partial region for legs of the pedestrian, a pedestrian region from the leg partial region is estimated. This allows a pedestrian to be properly captured in the image.

In the present embodiment, as in 3A is shown, moves the pedestrian detection section 51 the header model TP1 successively at predetermined intervals in the image and calculates a degree of similarity (as a rating) of an area overlapping the header model TP1 after each conversion resulting in a score record (not shown) resulting in the header model TP1. The pedestrian detection section 51 extracts the high-scoring areas having a rating greater than the predetermined threshold (in step S120). Subsequently, the pedestrian detection section extracts 51 the sub-regions based on the highly valued areas.

In light of the fact that in the present embodiment, as in 4B 1, some of the specific parts are less visible than the other parts, the predetermined limits for extracting the arms L1 and L2 and the legs N1 and N2, which are less visible parts, are set lower than the predetermined limits for extracting the head and hip M, which are more visible parts. This allows each part to be detected correctly.

In the present embodiment, the high-valued regions are divided into the divided-group regions (at step S130 of FIG 2 ) and the subgroup regions are extracted from the image based on the subgroup regions (at step S140). This allows each part to be reliably recorded.

In particular, as in 5A is shown, the high-scoring regions TC and TT are grouped into the two sub-group regions by distances between the high-scoring regions TC and TT and sizes of the high-scoring regions TC and TT. The subgroup regions are extracted from the image such that each subgroup region includes the high-valued regions with distances therebetween equal to or smaller than the predetermined threshold and having similar magnitudes. This allows a pedestrian who is farther away and a pedestrian closer to be clearly extracted from the image by using the sizes of the high-valued areas TT and TC, resulting in improved extraction accuracy.

Further, in the present embodiment, partial regions, each in the form of a rectangular circumscribed region including at least partial-group regions, are extracted from the image (at step S140 of FIG 2 ), which can advantageously lead to relatively simpler extraction processing.

Each sub-region in the present embodiment is a rectangular circumscribed region that includes one or more subgroup regions that are close to each other. 6 shows an example of an extraction of a partial region P, which includes two subgroup regions G6 and G7. The partial region P, which is a rectangular circumscribed region including two subgroup regions having a distance between them equal to or smaller than the predetermined threshold, is extracted. This allows the subgroup regions, of which there is a pedestrian group in each, to be reliably extracted.

In the present embodiment, a score of each sub-region is calculated by summing up scores of the respective sub-group regions included in the sub-region (in step S150 of FIG 2 ), wherein a score of each subgroup region is calculated by summing up scores of the respective high scoring areas included in the subgroup region. For example, in 6 A sum of scores of the respective high-scoring regions included in each of the sub-group regions G1, G2 is calculated, and then a sum of scores of the respective sub-group regions G1 and G2 is calculated, resulting in an evaluation of the sub-group region P. This allows a score of each subgroup region to be calculated in a relatively simple manner.

Further, in the present embodiment, as in FIG 7A 4, assume that a pedestrian body portion exists below a partial region P extracted using the header model. A size of the pedestrian body portion can be estimated from the sizes of the respective high-valued areas included in the partial region P, allowing the pedestrian region S to be estimated. When the score of the partial region P is equal to or smaller than the predetermined threshold, the pedestrian region S is disqualified as a pedestrian region.

As in 7B 4, pedestrian regions are detected on the basis of relative positions of two partial regions P1 and P3 extracted using the header model and a partial region P2 extracted using the leg partial model. For example, when the high-valued regions included in the partial region P1 and the high-valued regions included in the partial region P2 have the same size, it is determined that the partial regions P1 and P2 are linked with each other. That is, it is assumed that the partial regions P1 and P2 include heads or legs of pedestrians belonging to the same pedestrian group. However, the high-valued regions included in the partial region P3 differ in size from the high-valued regions in the partial regions P1 and P2. Since there is no subregion associated with the subregion P3, it is determined that the subregion P3 was detected erroneously.

According to 7C In the present embodiment, it is assumed that the two partial regions P1 and P2 are linked together. A pedestrian region S3 in the form of a rectangular circumscribed region including pedestrian regions S1 and S2 estimated from the partial regions P1 and P2, respectively, is estimated. The pedestrian area in the form of a rectangular region, including the subregions that are linked together, allows the pedestrian area to be properly appreciated.

When a sum of scores of the respective subregions included in the pedestrian region is equal to or smaller than the predetermined threshold, the pedestrian region is disqualified as a pedestrian region. That is, the pedestrian region score is calculated based on the partial regions therein, and when the pedestrian region score is equal to or smaller than the predetermined threshold, the pedestrian region is disqualified as a pedestrian region.

Further, in the present embodiment, when the pedestrian region is estimated and qualified as a pedestrian region, the collision risk determination section determines 22 a collision risk based on a distance between the own vehicle and the pedestrian, by the distance measuring unit 31 Will be received. In response to the risk of collision, the information controller provides 70 the vehicle control device 20 the driver by means of the loudspeaker 41 a notification ready and the vehicle controller causes collision avoidance control using the brake 42 and the steering wheel 43 , which provides safe driving support.

(Modifications)

• (i) In der vorstehend erläuterten Ausführungsform sind, wie in 5A dargestellt ist, werden die hochbewerteten Bereiche TC und TT in Teilgruppenregionen durch Distanzen dazwischen und Größen davon gruppiert.
• (i-1) Alternativ, wie in 5B dargestellt ist, können die hochbewerteten Bereiche TC, TT in Teilgruppenregionen durch Distanzen, zwischen diesen gruppiert werden. Das heißt, die Teilgruppenregionen können von dem Bild so extrahiert werden, dass sie die hochbewerteten Bereiche beinhalten, die Distanzen zwischen denselben aufweisen, die gleich oder kleiner als der vorbestimmte Grenzwert sind. Dies kann vorteilhaft zu einer relativ einfacheren Extrahierungsverarbeitung führen.
• (i-2) Darüber hinaus, wie in 5C dargestellt ist, wird für jede der Teilgruppenregionen G4, G5 eine Dichte von hochbewerteten Bereichen der Teilgruppenregion durch Teilen der Anzahl der hochbewerteten Bereiche TT, die darin beinhaltet sind, durch den Bereich bzw. die Fläche der Teilgruppenregion berechnet. Wenn die Dichte von hochbewerteten Bereichen der Teilgruppenregion größer als der vorbestimmte Grenzwert ist, kann die Teilgruppenregion als eine Teilgruppenregion qualifiziert werden. Wenn die Dichte von hochbewerteten Bereichen der Teilgruppenregion gleich oder kleiner als der vorbestimmte Grenzwert ist, kann die Teilgruppenregion als eine Teilgruppenregion disqualifiziert werden. Dies kann ebenso ein Risiko fehlerhafter Erfassung reduzieren, was ermöglicht, dass die Teilgruppenregionen verlässlich erfasst werden.
• (i-3) Alternativ, wie in 5C dargestellt ist, wird für jede der Teilgruppenregionen G4, G5 eine Gesamtbewertung der Teilgruppenregion durch Aufsummieren von Bewertungen der jeweiligen hochbewerteten Bereiche TT, die darin beinhaltet sind, berechnet. Wenn die Gesamtbewertung der Teilgruppenregion größer als der vorbestimmte Grenzwert ist, kann die Teilgruppenregion als eine Teilgruppenregion qualifiziert werden. Wenn die Gesamtbewertung der Teilgruppenregion gleich oder kleiner als der vorbestimmte Grenzwert sein kann, wird die Teilgruppenregion als eine Teilgruppenregion disqualifiziert. Dies kann ebenso ein Risiko fehlerhafter Erfassung verringern, was ermöglicht, dass die Teilgruppenregionen verlässlich erfasst werden.
• (ii) In der vorstehend erläuterten Ausführungsform wird eine Bewertung jeder Teilregionen durch Aufsummieren von Bewertungen der jeweiligen Teilgruppenregionen berechnet, die in der Teilregion beinhaltet sind (in bzw. bei Schritt S150), wobei eine Bewertung jeder Teilgruppenregion durch Aufsummieren von Bewertungen der jeweiligen hochbewerteten Bereiche berechnet wird, die in der Teilgruppenregion beinhaltet sind.
• (ii-1) Für jede Teilgruppenregion kann die Bewertung der Teilgruppenregion durch ein Produkt oder eine Summe der Summe von Bewertungen der jeweiligen hochbewerteten Bereiche in der Teilgruppenregion und die räumliche Nähe zwischen den hochbewerteten Bereichen in der Teilgruppenregion gegeben sein. Insbesondere, wie in einem unteren Teil von 6 dargestellt ist, kann die räumliche Nähe zwischen den hochbewerteten Bereichen jeder Teilgruppenregion als eine Inverse einer Summe von absoluten Differenzen der Position zwischen den hochbewerteten Bereichen sein, wobei die Position p jedes hochbewerteten Bereichs eine mittlere oder obere linke Position davon sein kann. Die Bewertung jeder Teilgruppenregion kann durch ein Produkt oder eine Summe der Summe von Bewertungen der jeweiligen hochbewerteten Bereiche in der Teilgruppenregion und der räumlichen Nähe zwischen den hochbewerteten Bereichen in der Teilgruppenregion gegeben sein.
• (ii-2) Für jede Teilgruppenregion kann die Bewertung der Teilgruppenregion nicht nur auf der Basis der Bewertungen der jeweiligen hochbewerteten Bereiche, die in der Teilgruppenregion beinhaltet sind, sondern ebenso auf der Basis einer Ähnlichkeit bezüglich Größe zwischen den hochbewerteten Bereichen, die in der Teilgruppenregion beinhaltet sind, zusätzlich oder anstatt der räumlichen Nähe zwischen den hochbewerteten Bereichen, die in der Teilgruppenregion beinhaltet sind, berechnet werden. Insbesondere, wie in einem unteren Teil von 6 dargestellt ist, kann die Ähnlichkeit bezüglich Größe zwischen den hochbewerteten Bereichen jeder Teilgruppenregion durch ein Inverse einer Summe von absoluten Differenzen der Größe zwischen den hochbewerteten Bereichen definiert werden. Die Bewertung jeder Teilgruppenregion kann durch ein Produkt oder eine Summe der Summe der Bewertungen der jeweiligen hochbewerteten Bereiche in der Teilgruppenregion und der Nähe der Größe zwischen den hochbewerteten Bereichen in der Teilgruppenregion (und der räumlichen Nähe zwischen den hochbewerteten Bereichen in der Teilgruppenregion in einigen Ausführungsformen) gegeben sein.
• (iii) In der vorstehend beschriebenen Ausführungsform, wenn die Summe der Bewertungen der jeweiligen Teilregionen, die in der Fußgängerregion beinhaltet sind, gleich oder kleiner als der vorbestimmte Grenzwert ist, wird die Fußgängerregion als eine Fußgängerregion disqualifiziert. Alternativ kann, vor dem Extrahieren der Fußgängerregion, wenn die Summe von Bewertungen der jeweiligen hochbewerteten Bereiche, die in der Teilregion beinhaltet sind, gleich oder kleiner als ein vorbestimmter Grenzwert ist, die Teilregion als eine Teilregion disqualifiziert werden. Eine relativ unverlässliche Teilregion kann somit verworfen werden, was ermöglicht, dass die Fußgängerregion richtiger aus dem Bild extrahiert wird.
• (iv) In der vorstehend erläuterten Ausführungsform wird ein Kollisionsrisiko als eine Funktion einer Distanz zwischen dem Eigenfahrzeug und dem Fußgänger bestimmt, die von der Entfernungsmesseinheit 31 empfangen wird. Alternativ kann ein Kollisionsrisiko unter Verwendung einer Position und/oder einer Größe und/oder einer Bewertung der Fußgängerregion zusätzlich oder anstatt der Distanz zwischen dem Eigenfahrzeug und dem Fußgänger bestimmt werden.
• (v) In der vorstehend erläuterten Ausführungsform wird ein Abstand zwischen dem Eigenfahrzeug und dem Fußgänger durch die Entfernungsmesseinheit 31 gemessen. Alternativ kann die Distanz zwischen dem Eigenfahrzeug und dem Fußgänger auf der Basis der Bildinformation von der Bildaufnahmeeinheit 32 bestimmt werden, wobei die Entfernungsmesseinheit 31 entfernt werden kann. Die Entfernungsmesseinheit ermöglicht, dass die Distanz zwischen dem Eigenfahrzeug und dem Fußgänger genau gemessen wird.

Hereinafter, modifications of the above-explained embodiment which can be applied without departing from the spirit and scope of the present invention will be explained.

• (i) In the embodiment explained above, as in FIG 5A is shown, the high-scoring areas TC and TT are grouped into subgroup regions by distances therebetween and sizes thereof.
• (i-1) Alternatively, as in 5B is shown, the high-valued areas TC, TT can be grouped into subgroup regions by distances between them. That is, the subgroup regions may be extracted from the image to include the high-valued regions having distances therebetween equal to or less than the predetermined threshold. This can advantageously lead to a relatively simpler extraction processing.
• (i-2) In addition, as in 5C is shown, for each of the sub-group regions G4, G5, a density of high-valued regions of the subgroup region is calculated by dividing the number of the high-valued regions TT included therein by the region of the subgroup region. If the density of high-scoring regions of the subgroup region is greater than the predetermined threshold, the subgroup region may be qualified as a subgroup region. When the density of high-scoring regions of the subgroup region is equal to or smaller than the predetermined threshold, the subgroup region may be disqualified as a subgroup region. This may also reduce a risk of erroneous capture, allowing the subgroup regions to be reliably captured.
• (i-3) Alternatively, as in 5C is shown, for each of the sub-group regions G4, G5, an overall score of the sub-group region is accumulated by adding up scores of the respective high-scoring areas TT included therein are calculated. If the overall score of the subgroup region is greater than the predetermined threshold, then the subgroup region may be qualified as a subgroup region. If the overall score of the subgroup region may be equal to or less than the predetermined threshold, the subgroup region is disqualified as a subgroup region. This can also reduce a risk of erroneous capture, allowing the subgroup regions to be reliably captured.
• (ii) In the embodiment explained above, an evaluation of each subregions is calculated by summing up scores of the respective subgroup regions included in the subregion (in step S150), evaluating each subgroup region by summing up scores of the respective high scoring regions is calculated, which are included in the subgroup region.
• (ii-1) For each subgroup region, the subgroup region score may be given by a product or a sum of the sum of scores of the respective high scoring areas in the subgroup region and the spatial proximity between the high scoring areas in the subgroup region. In particular, as in a lower part of 6 3, the spatial proximity between the high-scoring regions of each subgroup region may be an inverse of a sum of absolute differences of the position between the high scoring regions, where the position p of each high scoring region may be a middle or upper left position thereof. The score of each subgroup region may be given by a product or a sum of the sum of scores of the respective high scoring areas in the subgroup region and the spatial proximity between the high scoring areas in the subgroup region.
• (ii-2) For each subgroup region, the rating of the subgroup region may be based not only on the scores of the respective high scoring regions included in the subgroup region, but also on similarity in size between the high scoring regions included in the subgroup region are included, in addition to or instead of the spatial proximity between the high-scoring areas included in the subgroup region. In particular, as in a lower part of 6 2, the similarity in size between the high-valued regions of each subgroup region may be defined by an inverse of a sum of absolute differences in size between the high-valued regions. The score of each subgroup region may be a product or a sum of the sum of the scores of the respective high scoring areas in the subgroup region and the proximity of the size between the high scoring areas in the subgroup region (and the spatial proximity between the high scoring areas in the subgroup region in some embodiments). be given.
• (iii) In the above-described embodiment, when the sum of the scores of the respective subregions included in the pedestrian region is equal to or smaller than the predetermined threshold, the pedestrian region is disqualified as a pedestrian region. Alternatively, before extracting the pedestrian region, if the sum of scores of the respective high-scoring regions included in the sub-region is equal to or smaller than a predetermined limit, the sub-region may be disqualified as a sub-region. A relatively unreliable subregion can thus be discarded, allowing the pedestrian region to be more properly extracted from the image.
• (iv) In the embodiment discussed above, a collision risk is determined as a function of a distance between the host vehicle and the pedestrian being traveled by the ranging unit 31 Will be received. Alternatively, a collision risk may be determined using a position and / or size and / or rating of the pedestrian region in addition to or instead of the distance between the subject vehicle and the pedestrian.
• (v) In the embodiment explained above, a distance between the own vehicle and the pedestrian by the distance measuring unit 31 measured. Alternatively, the distance between the own vehicle and the pedestrian may be determined based on the image information from the image pickup unit 32 be determined, wherein the distance measuring unit 31 can be removed. The distance measuring unit enables the distance between the own vehicle and the pedestrian to be accurately measured.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• JP 2008-21034 [0004, 0005]

Claims (25)

Image processing device ( 10 ) for capturing an object in an image that is captured by an image capture unit ( 32 ) mounted in a vehicle, said device ( 10 ) comprises: a partial model storage means ( 61 ) for storing a plurality of sub-models each modeling corresponding parts of the object; a partial region extracting agent ( 51 ) for extracting partial regions for the corresponding parts of the object from the image using the respectively corresponding partial models which are present in the partial model storage means ( 61 ) are stored; and an object region estimation means ( 51 ) for estimating an object region in which the object is present on the basis of the partial regions generated by the partial region extraction means ( 51 ) are extracted. Contraption ( 10 ) according to claim 1, characterized in that the object is a pedestrian. Contraption ( 10 ) according to claim 1 or 2, characterized in that the partial region extracting agent ( 51 ) is adapted to shift each sub-model successively at predetermined intervals in the image, to calculate a degree of similarity of an area which overlaps the sub-model at a position in the image after each shift as a score, and to allocate the sub-regions on the basis of the high-ranked areas which are areas each having a rating greater than a predetermined threshold. Contraption ( 10 ) according to claim 3, characterized in that the predetermined limit value is set for each of the parts of the object. Contraption ( 10 ) according to claim 3 or 4, characterized in that the partial region extracting agent ( 51 ) is designed to extract subgroup regions, each of which includes high scoring regions extracted using a corresponding submodel, and subsequently to extract the subregions based on the subgroup regions. Contraption ( 10 ) according to claim 5, characterized in that the partial region extracting agent ( 51 ) is adapted to extract the sub-group regions each of which includes the high-valued regions with intervals therebetween equal to or smaller than a predetermined limit. Contraption ( 10 ) according to claim 6, characterized in that the partial region extracting agent ( 51 ) is designed to extract the subgroup regions, each of which has the high score areas of similar sizes. Contraption ( 10 ) according to claim 6 or 7, characterized in that the partial region extracting agent ( 51 ) is configured to determine, for each subgroup region, whether or not a density of high scoring regions of the subgroup region is equal to or less than a predetermined threshold, and if it is determined that the density of high scoring regions of the subgroup region is equal to or less than the predetermined threshold to disqualify the subgroup region as a subgroup region. Contraption ( 10 ) according to one of claims 6 to 8, characterized in that the partial region extracting agent ( 51 ) is configured to determine, for each subgroup region, whether or not an overall score of the subgroup region that is a sum of scores of the corresponding high scoring areas included in the subgroup region is equal to or smaller than a predetermined threshold, and if it is determined the overall score of the subgroup region is equal to or less than the predetermined threshold to disqualify the subgroup region as a subgroup region. Contraption ( 10 ) according to one of claims 5 to 9, characterized in that the partial region extracting agent ( 51 ) is adapted to extract each of the sub-regions in the form of a right-angle region including the sub-group regions extracted using a corresponding sub-model. Contraption ( 10 ) according to claim 10, characterized in that the partial region extracting agent ( 51 ) is adapted to extract each of said sub-regions in the form of a right-angle region including sub-group regions extracted using a corresponding sub-model with intervals therebetween equal to or less than a predetermined limit. Contraption ( 10 ) according to claim 10 or 11, characterized in that the Partial region extracting agent ( 51 ) is configured to calculate, for each of the subregions, a score of each of the subgroup regions included in the subregion by summing up scores of the corresponding high scoring areas included in the subgroup region, and evaluating the subregion by summing ratings of the subregion corresponding subgroup regions included in the subregion. Contraption ( 10 ) according to claim 12, characterized in that the partial region extracting agent ( 51 ) is designed to compute the rating of each of the subgroup regions included in the subregion for each of the subregions not only based on the sum of scores of the respective high scoring regions in the subgroup region, but also based on spatial proximity between the subgroups highly valued areas of the subgroup region. Contraption ( 10 ) according to claim 13, characterized in that the spatial proximity between the highly scored areas of each subgroup region is an inverse of a sum of absolute positional differences between the high scoring areas of the subgroup region. Contraption ( 10 ) according to one of claims 12 to 14, characterized in that the partial region extracting agent ( 51 ) is designed to calculate, for each of the sub-regions, the score of each of the sub-group regions included in the sub-region, based not only on the sum of scores of the respective high-scoring regions in the sub-group region, but also on a similarity in size between the highly valued areas of the subgroup region. Contraption ( 10 ) according to claim 15, characterized in that the similarity in size between the high-valued regions of each subgroup region is an inverse of a sum of absolute size differences between the high-valued regions of the subgroup region. Contraption ( 10 ) according to one of claims 12 to 16, characterized in that the partial region extracting agent ( 51 ) is designed to calculate, for each of the sub-regions, the score of each of the sub-group regions included in the sub-region, based not only on the sum of scores of the respective high-scoring regions in the sub-group region but also on a density of high-scoring regions the subgroup region. Contraption ( 10 ) according to one of claims 12 to 17, characterized in that the partial region extracting agent ( 51 ) is arranged to determine, for each of the partial regions, whether or not the score of the partial region is equal to or smaller than a predetermined threshold, and if it is determined that the score of the partial region is equal to or smaller than the predetermined threshold, the partial region as one Disqualify subregion. Contraption ( 10 ) according to one of claims 3 to 18, characterized in that the partial region extracting agent ( 51 ) is adapted to estimate an object region in which the object is present on the basis of magnitudes of the corresponding high-valued regions included in the sub-region. Contraption ( 10 ) according to one of claims 3 to 19, characterized in that the partial region extracting agent ( 51 ) is adapted to determine an object region in which the object exists on the basis of relative positions of the subregions. Contraption ( 10 ) according to claim 20, characterized in that the partial region extracting agent ( 51 ) is designed to estimate the object region in the form of a rectangular circumscribed region that includes the subregions. Contraption ( 10 ) according to one of claims 1 to 21, characterized in that the partial region extracting agent ( 51 ) is configured to calculate for each object region a score of the object region estimated from the sub-regions, to determine whether or not the score of the object region is greater than a predetermined threshold, and to determine that the score of the object region is the same or less than the predetermined threshold is to disqualify the object region as an object region. Contraption ( 10 ) according to one of claims 1 to 22, further characterized by an object region estimation means ( 52 ) for determining a collision risk on the basis of a position and / or a size and / or an evaluation of the object region which is detected by the object region estimation means ( 51 ) and the distance from the subject vehicle to the object. Vehicle control system ( 1 ), comprising: image processing apparatus according to claim 23; and a vehicle control device ( 20 ), which is an information control 70 ) to provide a driver with information in response to the risk of collision caused by the collision risk determination means ( 52 ) is determined. Vehicle control system ( 1 ), comprising: image processing apparatus according to claim 23; and a vehicle control device ( 20 ), the vehicle control means ( 80 ) to effect collision avoidance control in response to the risk of collision caused by the collision risk determination means (10). 52 ) is determined.

Images