DE102008041679A1 - Method for environment recognition for navigation system in car, involves storing data of object or feature in storage, and classifying object or feature by comparison of data after visual inspection of object or feature - Google Patents

Abstract

The method involves storing data of an object or a feature in a storage. The object or the feature is classified by comparison of the data stored in the storage after visual inspection of the object or the feature. The object or the feature is recorded in a digital card of a navigation device, and the data is derived from two time points from a sensor (11) e.g. light detection and ranging sensor, and/or is computed from sensor data of the sensor. The classification is associated with probability or reliability for a right classification. An independent claim is also included for a device for environment recognition for a system to be moved.

Description

State of the art

The The invention is based on a device or a method for memory-based environment recognition by type of independent Claims.

From the DE 10 2006 058 304 A1 a method for location-based processing of data is known, which stores occurring events in connection with a vehicle, wherein the location at which this event takes place, is stored. The aim of this invention is that the repeated approach to such places a memory is made, which can be spent, for example, a timely warning against previously encountered hazards. Furthermore, other driver assistance systems can be activated or informed, as well as using data from them, in particular a navigation device. Specifically, this invention is the treatment of events, so an event must occur at least once before it can be counteracted preventively. Furthermore, the document does not disclose a concrete method of how to obtain the information or what data is used.

Furthermore, from the DE 10 335 601 A1 a method for object classification using a 3D model database known in which stored in a database reference data are stored as a 3D data model, and the real measured data are compared to different variations of the 3D position of the model and classified. The use of reference data recorded by a different path (sensor) than the data to be tested leads to the problem that inaccuracies in both acquisition paths (eg component tolerances) are fully included in the classification result. An attempt is made to classify a known object via the faulty image of the sensor system.

Disclosure of the invention

The Inventive device and method with have the features of the independent claims In contrast, the advantage that a self-sufficient environment detection, analogous to that of a human is possible in the objects perceived in an unknown area and on a return to be recognized. In technology, depending on the sensor equipment variants sensor-specific characteristics of the environment objects, such as position, relative speed, extension, Contour, size, shape or reflectance recorded and it creates an environment model, which is adapted when it returns can be. From the change of parameters of this Objects between at least two perceptions can draw conclusions pulled, or classifications are made, in particular to clarify the question of whether an object is static, or dynamic (movable). The former will be the case when the object is at repeated recurrence every time, a dynamic object, such as a pedestrian is not every Time to show up. This makes an adaptive environment model more static Objects are created. Unlike other methods, you can differentiate dynamic and static objects from each other faster be there at the sudden appearance of a dynamic Object in a known location where normally no object exists, can be classified immediately.

Limited according to the invention the classification does not apply to dynamic and static objects, but can also be used for other classification if the inventive concept of Distinction between multiple perceptions applies. Conceivable is also a more diversified classification, such as the recognition of regularly recurring dynamic Objects, such as parked cars during business hours, Schoolchildren at the time the lesson starts / ends or one Days site.

By those listed in the dependent claims Measures are advantageous developments and improvements possible.

advantageously, The creation of a map takes place in which the detected Objects are registered. This allows a self-sufficient map to be created which will usually be more up-to-date and more accurate as such externally supplied digital cards, such as the a navigation system. The prerequisite for this, however, is that a track has been ridden several times, resp. By the entries of objects in a digital map, it is possible Capture and assign objects, even if you do not use the same infrastructure at the return drives. Such are objects, for example on the way there, on the way back already known unless special circumstances exist, such as For example, the fact that an object might be obscured. For example, if an object is only visible on the return trip because it is obscured depending on the direction of view, it is advantageous for the next way the object is already known, although it is not visible for this direction of travel.

Furthermore, by the digital map, the problem of aberrations can be corrected by the so-called triangulation, which at the Po For determining the distance of objects with the help of a stereo video sensor system, imaging errors are caused, which can be made plausible by plausibility checking with the already existing data in the digital map.

Advantageous is the possibility of the invention Always perform procedures with the same sensors, whereby the comparison of data always on the same sensor sources based. Unlike known methods, this approach is used not intended, about the faulty figure of the Sensors a known static object, which for example via other sensors were recorded and stored in a database, to classify, but the objects from the reduced information recognize and store the same sensor signals. this has the advantage that the "sensor view" of a vehicle used to create the adaptive environment model, which effects of component variations on the classification process omitted, because these errors cut out, so to speak. On the other hand, in known methods would be from a certain Degree of error-prone mapping of a classification no longer possible. By using the adaptive environment model less accurate sensors can now be used, or sensors that perform conflicting tasks, such as distance and contour sensing and usually need to be optimized differently. In addition to the reduction of technical errors, such as Sensor noise, manufacturing process variations, aberrations By triangulation, the use of the same sensors is also suitable for environmental influences, such as aberrations Weather conditions, wet, cold or Compensate for contamination.

advantageously, can one from the information, which is characterized by the invention System yield and information from other sources a sensor data fusion which offers some advantages: On the one hand ambiguous environment sensor data can be secured with the invention and by the presence of supplementary information an at least second system improves the scene interpretation become safer, d. H. more accurate and faster results or functional decisions to get. Furthermore, by means of the comparison of the sensor data, or evaluation results, or intermediate steps of at least two Systems that perform a plausibility check are the accuracy increases, as well as additional information provides, what a greater foreknowledge for the scene interpretation causes. The adaptive environment model allows the storage of geographic road information that themselves were explored, and the comparison of this information with current digital maps, such as navigation devices or supplementary or alternative sources of information like GPS or information about a human-machine interface. For example, with short-term changes the course of the road (construction sites, traffic calming, road recession) this information is perceived from the vehicle's point of view and thus a more realistic and up-to-date image of the environment produce. Through this additional knowledge about the driving condition and the road course can be more stable Driver assistance functions are enabled. So, for example a lane-keeping-support system that uses data from the adaptive environment model, to the Fahrtrajektorie, or Fahrschlauch to known, or with a certain probability of passing around obstacles. Another example is the recognition of pedestrians as dynamic objects by means of the a priori knowledge of the adaptive Environment model can be done quickly, causing warnings or interference can take place in the driving dynamics.

Advantageous is the possibility the adaptive environment model for Create routes that frequent the vehicle frequently be what z. B. can be the daily way to work. The more frequently a particular route is traveled, the more the significance of the adaptive environment model becomes more reliable the classification of static objects. For example, you can these consequently as landmarks for the scene interpretation be used. Which route as frequently driven Track applies, can be defined either by the driver or else be determined independently by the system. The advantage of selective creation of the adaptive environment model only for Certain routes manifest themselves in that anyway limited and expected large amount of data thus storage space can be saved.

A then another optimization task would be the information in the adaptive environment model to the characteristics reduce, which with the built-in sensors in the vehicle, a high relevance for the considered driver assistance function.

It is advantageous to store a probability together with the classification result, which states with what certainty the classification is correct. This increases the meaningfulness of the classification and allows a more accurate evaluation or reaction. For example, the unnecessary warning of the driver can be avoided or a braking intervention can only be carried out if an obstacle with a high probability was detected. Ideally, a probability is determined for each recognized object, either for the classification result, or for the Evaluation of the individual parameters, from which a reliability of the hypothesis to static objects can be made. This hypothesis formation or probability determination will depend on the prior knowledge of the adaptive environment model, ie, for example, how often the vehicle has already passed the location and has detected an object or feature of an object, and the quality of the sensor system that provides the features. Thus, even static objects that could not be clearly statically classified by the sensor system due to the unique sensation due to various levels of concealment (regardless of whether they are obscured by static or dynamic objects) can be stabilized.

In a development of the system could also be dynamic Objects are classified according to whether they are, for example regularly show up, like children nearby a school at break time, or parked cars in temporary parking zones, or closed level crossings with the transit times of trains. For the realization of such Systems can provide information about the temporary occurrence of objects either intrinsically about the history and certain correlations are obtained, or extrinsic with others Data stocks, such. Eg timetables or data in navigation systems.

Advantageous is that the invention additionally the type and number of Information extended for a scene interpretation be used. Based on this supplementary information the development of new driver assistance functions is possible, not implemented with sensors of current driver assistance systems can be. For example, this is conceivable in this way a pedestrian warning system, or with special evaluation even to realize a child warning system. Furthermore, as Reaction automatically the horn or flasher be activated.

Advantageous is that the invention additionally a movement profile dynamic objects, such as pedestrians, schoolchildren or cyclists, considered in the map of the adaptive environment model and this for future driver assistance functions holds.

Thereby that the perception of the system according to the invention is similar as in humans, especially with respect to the visual or cognitive abilities, another opens up Object of the invention, namely autonomous driving. requirement this is the information about a route opened, which learns the vehicle itself. Further The condition is that the reliability of the hypotheses for the information in the adaptive environment model in increase to the extent that they are the hypothesis by the driver, or to come straight away.

Irrespective of this from the previous it is possible to use the contents of the adaptive environment model not only with other driver assistance systems, but also with other systems in their own, as well as in the foreign vehicle (for example Ad-hoc communication). So the special information, about which has a single vehicle, such as the Existence of a new construction site, other road users to Will be provided. This may be about peer-to-peer (ad hoc) communication or via a centrally managed data collection system respectively. Functions built on digital maps (eg Lane Keeping Support Systems), can do so with the help of Adaptive environment model independently to a changed Adapt infrastructure.

Advantageous is that about the prior knowledge of the adaptive environment model Outriding routes decisions for the timetable can be taken, even if at the time of the current trip A recognition of objects is not possible if technical or environmental influences prevent a current sensing. Thus, appropriate weather or lighting effects cause imaging errors in the respective sensors used; For example, snow, spray and rain can be faulty Lidar / radar signals, or video recordings Too strong a backlit noise. In such circumstances therefore not reliable for a particular scene, current environment image available. Here you can the landmarks / objects / features perceived from previous journeys be inserted into the current scenery. simultaneously is a prediction of the driving trajectory, or of the driving tube, for the unreasonable road course possible. simultaneously In the trajectory planning the foresight span can be decisive increase.

Definitions

A Hypothesis is the assumption of a classification result. If For example, it is distinguished as classification whether an object static (landmark) or dynamic, that means strong Hypothesis of one or the other that the classification is likely correct is. The same applies to other classification variants, such as the classification of the type of object, or the behavior of the object or its temporary recurrence behavior.

The adaptive environment model is created, in which an environment identifier takes place by means of known sensor signals and a scene interpretation (these terms can also be synonymous be used). The system is adaptive by updating of data or the classification when driving past again Scenery.

When Imaging procedures are called aparative procedures with those physical phenomena are visualized. In the simplest Case is taken via a video camera directly a picture, which can be evaluated. With the help of physical conditions Other sensors, such as infrared cameras, radar, Ultrasound, Lidargeräte or laser scanner information deliver, which can be visualized by means of an image. all imaging is common that measurements one of a Object outgoing physical effect converted into an image becomes.

in the Generally, environment sensors, such as radar, allow ultrasound, Laser scanners or video only a limited illustration the properties of environment objects in the technical system. So that driver assistance functions, the objects that are important to them by means of environmental sensors nevertheless recognize, hypotheses are created on the mapped properties, which make it possible to draw conclusions about an object. The relevant and pictured for an object hypothesis Properties can also be referred to as "features" become. Because features z. B. from the shape, size, Color, surface texture, reflection and material properties or just the knowledge of existing edges and corners an object must exist for a driver assistance function not necessarily the complete knowledge about an "object" is entered in the adaptive environment model become. It is also sufficient for many functions, isolated, but to store relevant features. This results also bsp. positive effects on the required storage requirements.

figure description

embodiments The invention is illustrated in the drawings and in the following Description explained in more detail.

It demonstrate,

1 a block diagram of the system according to the invention,

2 the signal flow of a system according to the invention,

3 the creation of an adaptive environment model during a first trip,

4 the adaptation of the environment model on a return journey,

5 the fast detection of dynamic objects in a known environment,

6 the prediction of a section of road in the case of disturbed sensors,

7 the solution of a triangulation problem.

In 1 a block diagram is shown with the components of the system involved wherein at least one sensor 11 For example, a lidar sensor with optional other sensors 12 For example, radar or video sensors whose signals a basic signal processing 13 are supplied, the results of which are processed by any system in the vehicle, but in particular by the invention described adaptive environment model 14 , This connects the data with the data of a position sensor 15 , for example, a GPS signal. Furthermore, a sensor data fusion is optionally carried out, wherein the data from the system according to the invention 14 be merged with the data of foreign systems to thereby in particular to recognize and classify objects. The results, ie the data or also classification results of the adaptive environment model 14 and the optional external data fusion provided by external systems will enter into a scene interpretation 17 , which describes the current environment and makes it possible, if necessary, a function triggering 18 perform. This can include a variety of things, from the simple warning of a driver in any way on the message to other driver assistant systems of an imminent danger situation, up to directly in order a function triggering, such as a braking operation. For example, airbag systems can also adjust their trigger thresholds depending on the scene being interpreted. The sensors on the vehicle 11 . 12 Provide a limited picture of the environment, since the figure is faulty against environmental influences.

Alternatively, in addition, a feedback of the scene interpretation 17 to the adaptive environment model 14 be realized (dashed line), this results of the scene interpretation to improve the predictive accuracy of the adaptive environment model are considered.

In 2 the signal flow of the inventive method is described in which at least one sensor 21 Provides data from which to extract features 22 and together with the position data 23 in a store 24 be filed. The features are, for example, recognized objects, be they dynamic or static, but can also take into account partial information of these objects, such as surface information, textures, contours, edges or corners. This current information pair is used simultaneously to form a hypothesis 25 with help from previous information from the store 24 to build. These hypotheses then yield the adaptive environment model 26 from which a scene interpretation 27 can be derived, optionally with the help of external data 29 and a possible function triggering 28 as already in 1 described.

In an expanded signal flow, additional information of the adaptive environment model is added 26 in the store 24 drops (dashed line).

In the following 3 to 7 FIG. A describes a real picture of the surrounding situation in which the vehicle is located, in FIG. B the adaptive environment model that results from the method according to the invention.

The scenario in 3 consists of a vehicle having a system according to the invention, two trees B1 and B2, two traffic signs S1 and S2, two buildings H1 and H2, and a pedestrian F1. If the vehicle FZG passes through the scenario, hypotheses can be made about static objects that are stored in the map of the adaptive environment model with a certain degree of reliability. In the figure B this map is shown, wherein the reliability of a hypothesis on the strength of the gray value is indicated. With more frequent occurrence of an object, in particular by driving past several times, the probability for the classification of a static object is always higher and the gray value thus always darker. With a single pass as here in 3 By evaluating several images, a classification can already be carried out and, for example, very dynamic objects such as oncoming vehicles can even be classified. Furthermore, even very long visible static objects, such as the building H1 can be classified as relatively static as static. Particularly critical is an object such as the pedestrian F1, which, if it does not move, could be misclassified as a static object. Furthermore, hidden targets, such as the traffic sign S2, which is covered in contrast to the traffic sign S1 during passing by mostly the building H1, recognized with only a slight accuracy, which is represented by the gray value accordingly.

In particular in 4 the advantage according to the invention now appears in which a more accurate classification becomes possible by multiple sensing of a frequently traveled route, also from different directions (eg on the way back from the work). In this figure, the vehicle FZG finds the same static scenario, whereby a further stabilization of the hypotheses of the adaptive environment model can take place. As the figure can be seen, the previously in 3 as an unstable static object, pedestrian F1 has now been correctly classified as a dynamic object, since it can no longer be detected because it has disappeared. Thus, the probability of its hypothesis as a static object in the digital map of the adaptive environment model can be reduced and deleted in principle. Alternatively, the knowledge of classified dynamic objects can be used to create motion profiles. At the same time, static objects that could not be unambiguously classified as static by the one - time sensing due to the previously described occlusion can now be more accurately assigned, which is reflected in the 4B is shown as a darker gray value of the traffic sign S2. This conducive to the fact that the viewing can take place from another perspective (return).

In 5 the vehicle FZG is confronted with pedestrians F1 to F3, which are suddenly in an - already known - environment, or emerge. Due to the fact that the environment model is already established by repeated passing (see depth of gray values), a significant deviation of the current sensor information from the adapted environment model (here the pedestrian) is sufficient to say with a high degree of certainty that it is a dynamic object is. Thus, a very fast reaction can take place. Thus, for example, a braking maneuver can be initiated when pedestrians are on the road, a construction site has been re-established, the end of a traffic jam is reached or parked cars impede the onward journey. The same applies to warnings or evasive maneuvers or other reactions.

In 6 a dashed bordered area is shown in which the vehicle FZG due to, for example, weather conditions no signal, or no suitable image receives to perceive objects. In such circumstances, therefore, no reliable and up-to-date image is available for a particular street. The static landmarks, objects and features perceived in past pass-bys, which are stored in the adaptive environment model, allow a prediction of the drivable area or a travel tube also in the part of the unreasonable road course. As a result, the foresight range can be significantly increased in comparison to today's systems by the prior knowledge of already worn tracks.

In 7 is shown in the figure A, a situation that can lead to aberrations in today's environment sensors and are difficult to control. It is a faulty figure E due to the triangulation with a stereo video represented sensor system, which generates a dummy object due to the constellation of the objects H1 and H2. By plausibility checking of the obtained sensor information by means of the adaptive environment model, it can be concluded that the determined object E represents an aberration and thus no erroneous entry of a new object into the adaptive environment model takes place or a misclassification takes place.

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• - DE 102006058304 A1 [0002]
• - DE 10335601 A1 [0003]

Claims (10)

Method for environment detection - for a moving system - wherein at least one sensor attached to the moving system is used, - in which by means of an imaging process - at least one object or feature is detected in the environment of the system - at a first time - Data of at least one of these objects or features are stored in a memory, characterized in that - after a further possible screening of this at least one object or feature - at least a second time - a classification of this at least one object or feature takes place - using a comparison the data stored in memory. Method according to claim 1, characterized in that that The at least one object or feature in a digital map is registered. Method according to one of the preceding claims, characterized marked that - the data to be compared from the at least two times, from the at least one same sensor originates and / or from sensor data of the same at least one sensor were calculated. Method according to one of the preceding claims, characterized marked that - for a more precise classification additional information from other sources used can be Method according to one of the preceding claims, characterized marked that - only the data of the objects or Characteristics are stored that are spotted most frequently be and / or - at their usual location the system comes over often and / or - the through the downstream scene interpretation as particularly relevant for a driving task to be displayed, such. B. the autonomous Driving, to be classified. Method according to one of the preceding claims, characterized marked that - the at least one classification with a probability or reliability for the right classification is linked and / or - the Probability and reliability in the form of an iterative Procedure to be adapted over time. Method according to one of the preceding claims, characterized marked that - the data and / or the classification and / or resulting reaction commands to other systems be carried and / or - be stored and / or - For the creation of motion profiles more classified, more dynamic Objects are used. Method according to one of the preceding claims, characterized marked that - at an impossibility the detection of objects or features at the second time to the stored data and / or classification becomes. Device for environment detection - For a moving system, - with at least one sensor associated with the system, - suitable to carry out an imaging process - to Detection of at least one object or feature in the environment - With a memory characterized in that - one Unit for classification is so pronounced - the Compare data from different times from memory Device according to claim 9, characterized in that that - Interfaces are provided in the System import more data for classification, - and or Data and / or classification and / or resulting reaction commands to export to other systems.