DE102020202905A1 - Method and a system for improved environment recognition - Google Patents

Abstract

The invention relates to a method for improved environment recognition based on pose recognition with the following steps:
- Recording (S1) an image of the surroundings or a sequence of images of the surroundings by means of at least one surroundings detection sensor (2) of a vehicle;
- Detecting (S2) one or more person (s) in the image of the surroundings or the sequence of images of the surroundings,
- Determination (S3) of one or more pose (s) of the one or more person (s),
- Classifying (S4) the pose (s) of the one or more person (s),
- Determination (S5) of features in the vehicle environment based on the classification of the pose (s).

Description

The invention relates to a method and a system for improved environment recognition based on pose recognition.

Methods for detecting and registering pedestrians are already sufficiently known from the prior art. For example, the pamphlet shows EP2580739A2 a monocular 3D pose assessment and tracking by detection. The publication also shows US2009175540A1 a system, a method and a computer program product for assessing a pose of a human upper body.

In the previously known methods, the pose is represented by a skeleton representation. To define a relevant area in which the pedestrian is located, a rectangular bounding box is placed around the recognized pedestrian, since it is difficult to capture the exact contour. This is also the case in the figures of FIG EP2580739A2 shown. This approach is also used for other objects such as vehicles in order to be able to quickly estimate the maximum dimensions of the respective object.

It is an object of the invention to provide a method for improved environment recognition.

This object is achieved by the subject matter of independent claims 1 and 7. Further advantageous refinements and embodiments emerge from the subclaims.

The first considerations were that in the methods known from the prior art, only the poses or the objects are considered. However, certain features in the vehicle environment can be deduced from the behavior or the pose of one or more people.

• - Aufnehmen eines Umgebungsbildes oder einer Folge an Umgebungsbildern mittels zumindest eines Umfelderfassungssensors eines Fahrzeugs;
• - Detektieren einer oder mehrerer Person(en) in dem Umgebungsbild oder der Folge an Umgebungsbildern,
• - Ermitteln einer oder mehrerer Pose(n) der einen oder mehreren Person(en),
• - Klassifizieren der Pose(n) der einen oder mehreren Personen,
• - Ermitteln von Merkmalen in der Fahrzeugumgebung basierend auf der Klassifikation der Pose.

Therefore, according to the invention, a method for improved environment recognition based on a pose recognition with the following steps is proposed:

• - Recording an image of the surroundings or a sequence of images of the surroundings by means of at least one surroundings detection sensor of a vehicle;
• - Detecting one or more person (s) in the image of the surroundings or the sequence of images of the surroundings,
• - Determination of one or more pose (s) of the one or more person (s),
• - Classifying the pose (s) of one or more people,
• - Determination of features in the vehicle environment based on the classification of the pose.

The surroundings detection sensor is preferably a mono camera, a stereo camera or a surround view system.

In a particularly preferred embodiment, the determination of the one or more poses is carried out by means of contour finding (e.g. by means of a keypoint detector). After the contour has been found, semantic segmentation can be carried out, for example, in order to be able to better recognize the individual body parts and their alignment. A major advantage of pose recognition compared to the classic bounding box approach is that several people who are standing close together can be recognized in one image. In the case of a bounding box approach, people would possibly be covered by the bounding box of the first person recognized.

Furthermore, when classifying the pose (s) it is preferred to classify according to special poses over time, pose accumulations in the image and / or special pose positions.

In a further preferred embodiment, the carrying along of an object is determined based on special poses over time. The object carried by a person is particularly preferably determined based on special poses over time. Special postures can be deduced from the special poses over time, which are typical when a person is carrying an object. For example, based on the posture, it can be recognized that a person is pushing a stroller or a walker. The recognition of a walking stick, the pulling of a suitcase or the like or the recognition of a wheelchair would also be conceivable. This information can in turn be used to adapt the driving behavior accordingly at a zebra crossing, for example. This information is important in order to avoid a collision of the vehicle with the object carried by adapting the driving strategy. If, for example, it is recognized that a person who is moving towards a zebra crossing is pushing a stroller, the vehicle must come to a standstill earlier accordingly. The assignment of the poses to the respective objects can be carried out, for example, by means of a recognition system (e.g. neural network, look-up table). The object does not have to be recognized directly.

Topographical properties of the vehicle environment are preferably determined based on a special pose position. In the light of this invention, topographical properties can mean / recognize elevations in the vicinity such as bridges or houses. In principle, the information that pedestrians usually walk on the ground can be used to estimate the course of the road. If there are deviations, this allows conclusions to be drawn about peculiarities in the course of the road. For example, when a pose is recognized above the vanishing point, it can be concluded that there are bridges or high adjoining houses, when people are recognized at windows. This information about the presence of vertical structures can in turn be helpful in order to start corresponding algorithms, for example extrinsic / intrinsic auto-calibration.

In a further preferred refinement, certain landmarks in the vehicle environment are determined based on the accumulation of poses. For example, based on accumulation patterns over time, for example, a traffic light or a pedestrian crossing can be inferred. For this purpose, the posture, line of sight or, in general, the orientation of the people can be recognized by means of the pose representation. If several people are recognized at the roadside looking towards the street in this way, it can be concluded that there is a traffic light. This information can be used, for example, as a landmark for localization. A cluster a little further away from the street and facing away from the street could indicate a sight. Accumulations next to a stopping point can be an indication of stops, e.g. a bus stop.

Furthermore, an accumulation of pedestrians or cyclists who were recognized over time on the basis of the special movement pattern of the pose could indicate paths next to the street (e.g. pedestrian paths or cycle paths). If these accumulations are recognized directly on the road, this can be an indication of a marathon / bike race, a demonstration or a move. This information can then be used, for example, to optimize route planning.

• - das aufgenommene Umgebungsbild zu analysieren und eine oder mehrere Person(en) zu detektieren,
• - eine Posen-Erkennung der Person(en) durchzuführen,
• - die Pose zu klassifizieren und
• - Merkmale in der Fahrzeugumgebung basierend auf der Klassifikation der Pose zu ermitteln.

Furthermore, according to the invention, a system for improved environment recognition based on pose recognition is proposed, comprising at least one environment detection sensor for recording an image of the surroundings or a sequence of images of the surroundings, as well as a data processing device which is designed for this purpose

• - to analyze the recorded image of the surroundings and to detect one or more persons,
• - perform a pose recognition of the person (s),
• - Classify the pose and
• - Determine features in the vehicle environment based on the classification of the pose.

• 1: ein schematisches Ablaufdiagram einer Ausführungsform der Erfindung;
• 2: eine schematische Darstellung des Systems nach einer Ausführungsform der Erfindung.

Further advantageous configurations emerge from the figures. Show in it:

• 1 : a schematic flow diagram of an embodiment of the invention;
• 2 : a schematic representation of the system according to an embodiment of the invention.

1 shows a schematic flow diagram of an embodiment of the invention. In step S1 an image of the surroundings or a sequence of images of the surroundings is recorded by means of at least one surroundings detection sensor of a vehicle. In step S2 one or more people are detected in the image of the surroundings or the sequence of images of the surroundings. In a subsequent step S3 one or more pose (s) of the one or more person (s) is determined. In step S4 the pose or poses of the one or more person (s) are classified. In step S5 are determined based on the classification of the pose (s) features in the vehicle environment.

2 shows a schematic representation of the system according to an embodiment of the invention. Here is a system 1 for the detection of road users, in particular people in road traffic shown in a schematic representation. The system 1 comprises at least one environment detection sensor 2 as well as a data processing device 3 . The at least one environment detection sensor 2 as well as the data processing device 3 are thereby via a data connection D. connected, by means of which image data from the optical sensor 2 can be transmitted to the data processing device. It would be conceivable that the data processing device is connected to one or more actuators which, based on the results of the data processing device, control the vehicle accordingly. The data connection D. is preferably designed to be wired. However, wireless connections such as WLAN, Bluetooth or the like would also be conceivable. The at least one environment detection sensor 2 is a mono camera, a stereo camera or a surround view system.

List of reference symbols

1

system

2

Environment detection sensor

3

Data processing device

D.

Data Connection

S1-S5

Procedural steps

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2580739 A2 [0002, 0003]
• US 2009175540 A1 [0002]

Claims (8)

Method for improved environment recognition based on pose recognition with the following steps: - Recording (S1) an image of the surroundings or a sequence of images of the surroundings by means of at least one surroundings detection sensor (2) of a vehicle; - Detecting (S2) one or more person (s) in the environment image or the sequence of environment images; - determining (S3) one or more pose (s) of the one or more person (s); - Classifying (S4) the pose (s) of the one or more person (s); - Determination (S5) of features in the vehicle environment based on the classification of the pose (s). Procedure according to Claim 1 , characterized in that the determination of the one or more poses is carried out by means of contour finding. Procedure according to Claim 1 , characterized in that when classifying the pose (s) according to special poses over time, pose accumulations in the image and / or special pose positions are classified. Procedure according to Claim 3 , characterized in that the carrying along of an object is determined based on specific poses over time. Procedure according to Claim 4 characterized in that the object carried by a person is determined based on the poses over time. Procedure according to Claim 3 , characterized in that topographical properties of the vehicle environment are determined based on special pose positions. Procedure according to Claim 3 , characterized in that certain landmarks in the vehicle environment are determined based on poses accumulations. System (1) for improved environment recognition based on pose recognition, comprising at least one environment detection sensor (2) for recording an image of the surroundings or a sequence of images of the surroundings, as well as a data processing device (3) which is designed for this purpose - to analyze the recorded image of the surroundings and to detect one or more persons, - perform a pose recognition of the person (s), - Classify the pose and - Determine features in the vehicle environment based on the classification of the pose.

Images