GB2564897A - Method and process for motion planning in (un-)structured environments with pedestrians and use of probabilistic manifolds - Google Patents

Abstract

A method for predicting pedestrian movements in structured and unstructured environments to be used for motion planning and control of an autonomous vehicle comprises the steps of: Reading map data where the vehicle is planned to drive. Reading the structure of static (eg buildings)/quasi-static/dynamic (eg pedestrian) obstacles. Computing specific spaces, such as tangency/orthogonal space, and optimal grids. Computing vector fields. Configuring a pedestrian model. Predicting movement based on last captured pedestrian position. Generating a motion plan for the vehicle. Controlling the vehicle according to the motion plan. The method may further comprise the use of uncertainty quantification in motion planning. The method may additionally predict pedestrian trajectories and the environment model by incorporating probabilistic manifold descriptors and topological spaces. The method may include visual measurements (eg of pedestrian gestures) and contextual information about the environment. The method may incorporate network descriptors to capture topological spaces of humans and learn their dynamics

Description

Method and process for motion planning in (un-J structured environments with pedestrians and use of probabilistic manifolds

BACKGROUND OF INVENTION

In everyday life, people often participate in the traffic without being aware of it. Usually we take it as understandable to reach a target safely. Either as a driver, cyclist or pedestrian we change our role in the nature of participation. But in reality there are still too many fatalities on the roads of Europe and the world. Technical innovations in the field of automated driving functions have steadily reduced the number of fatalities. Nevertheless there are still many problems and open questions for automated driving. Especially for situations in complex environments (e.g. cities) with many different road users (e.g. pedestrians, bicycles, animals ...) situations are complex and challenging for the motion planning algorithm. For this, new virtual testing procedures are presented.

Autonomous vehicles should...

• React and drive like a human • Make correct decisions • React appropriately in various situations (e.g. reactive) • Drive safely and efficient also in uncertain and dynamic environments

The technical effect of the invention is the increase of safety for (autonomous) vehicles (Level 3 and Level 4) in complex, uncertain, dynamic virtual environments and real world behavior of pedestrians.

IMPORTANT DEFINITIONS

Following definitions can be interpreted as a help for the understanding.

City Graph (compare: Block W: World): A mathematical description of the road network with nodes and edges (e.g. streets). 1

Open Street Map: Geodata with open access. Development by huge web-community.

Motion planning: Search of future trajectories for the ego-vehicle depending on the believed (future) time state space.

(Future) time state space: Mathematical description of future state space depending on the predictions. Necessary for collision avoidance. Depending on the uncertainty there are several deterministic state space, belief state space, plausible state space.

Uncertainty Quantification (compare: Block M: Uncertainty Quantification/Predictive Time-State-Space with confidence levels): In safety related applications there are some methods to quantify the uncertainty. The uncertainty representation and propagation can be changed.

Autonomous Mode (compare: Block T A: Period A and Block T E: Period E): Autonomous mode means in this document that the ego-vehicle is equipped with on-board sensors and processing units to enable a self-driving mode without external sensors from the infrastructure. Information from external resources offers the possibility to drive less conservative driving trajectories.

Situation prediction (compare: Block 11...In: Machine Perception Units (e.g. different configurations)): Besides the prediction of the (human-) movement (e.g. positions), there are more aspects which can be incorporated in the prediction. Semantic information, personal internal stance or environment aspects can be incorporated.

Cloud service (compare: Block E: Server (e.g. Cloud service)): A cloud service, which assists the egovehicle in following aspects: traffic flow coordination, navigation, motion planning, situation recognition and -prediction. It is assumed there are many sensor networks. For safety reasons several servers to achieve redundancy are presumed. Therefore it is also possible that the ego-vehicle can communicate to multiple sources.

Ego-vehicle: The ego-vehicle, which can drive in an autonomous mode, consists of Block A: (Autonomous) Vehicle(s), Block B: On-Car communication units for communication with the cloud service, Block C: Processing and Navigation Unit and Block D: On-Board sensors and perception units

Predicted Time-State Space: The predicted time-state space is necessary for motion planning of a robotic system. Therefore predictions where the obstacles will move in the future will lead to the predicted time-state space.

STATE OF THE ART

Map data and databases

There are different (online) map data services available. Online-map actualization (e.g. Waze [2]), open source projects. Meanwhile, there are some companies that are specialized in spatial data, for example HERE [3], [4] and TomTom [5]. Also on the format OpenStreetMap maps are available for research purposes [6]. There are also new approaches, databases and technologies for the human movement detection (e.g. Mapillary [7], Placemeter [8]). There is also some current 3D virtual environments available (e.g. 3D-0SM [9]) and new web services (e.g. bostonography.com [10], geOps [11], [12]). With [13] google infrastructure or via other commercial APIs (e.g. [4]) it is possible to use map APIs (e.g. APIs for geocoding, places, map) with several information ([14]). An example is shown in [15], how it is possible to use Google API for tracking applications with smartphones. It is shown, that is possible to communicate between autonomous vehicles and pedestrians via a communication network [16]. Current survey from human movement detection [17] and technology [18]. A current pedestrian detection system for driver assistance with off board and onboard sensing units is presented in [19]. A pedestrian system with onboard systems is presented in [20]

Human Movement Prediction

In [21] a study about the state of the art for movement prediction algorithms is presented. In [22] the growing hidden Markov models are presented, which incremental learn new behaviors. The [23] offers some new principles from a statistical inference perspective, where causal dependencies are incorporated in the movement prediction of pedestrians. In [24] Gaussian Processes are used, where spatial dependencies can be analyzed.

Motion Planning

Surveys for motion planning can be found in [25], [26], [27] [28] to get an overview of the state of the art. There are two current approaches which are promising for motion planning. Optimization based and sampling-based motion planning algorithms.

Sampling-based motion planning

For sampling approaches rapidly exploring random trees are the most famous approaches and they build a graph with different variants of exploration of the state space. For non-holonomic systems kinodynamic versions are used [29] [30] [31]. Rapidly- exploring random trees (RRTs) and variants can be found in automotive path planning [32] [33]. These can be used for real-time applications, but don't have redundant pathways. Redundant pathways could be advantageous for dynamic environments with moving objects, but costly for the computation. In this document a compromise in sense of optimality is presented. Motion planning problems in high-dimensional state spaces is known to be PSPACE-hard [33]. Probabilistic roadmaps (PRM) and RRTs are incremental sampling-based planners. Motion planning problems in highdimensional state spaces is known to be PSPACE-hard [33]. Probabilistic roadmaps (PRM) and RRTs are incremental sampling-based planners.

Optimization-based motion planning

In [34], [35], [36], [37] and [38] mixed integer linear programming algorithms are used for motion planning algorithms. Mixed4nteger Linear Programming can be used as a MPC formulation [38] and are promising because they incorporate binary variables for logical expressions.

Inventions in ADAS and autonomous vehicles

In [39] an automated movement of a vehicle is described, especially in a fixed environment (e.g. park, factory). The surrounding road users are detected with external sensors. In [39] semi-autonomous movements of the ego-vehicle with detection of environments of the vehicle by outdoor-sensor and application for park assistant or robots in industry is presented. In [40] the prediction of preceding vehicles is done with an adaption ofthe perception module (region of interest) with data fusion. Effect on the adaption of the velocity and steering angle, to assist the driver is the result. In [41] the prediction of traffic participant is done, consisting of a system with a localization unit for movable objects. The collision avoidance: prediction of collisions and warnings is done with cooperative sensors (active or passive RFID transponder) for pedestrian detection (not detection of hidden objects with cameras) and classification of the object. In [42] a determination of a driving strategy is done with prediction of movements and evaluation of environment data and modelling the virtual driver with artificial intelligence. In [43] the prediction of the region of movement, situation classification of normality of movement and selection of movement models for prediction. In [44] a collision avoidance system is introduced to bring the vehicle to a safe state with adequate and automated steering and acceleration. Modules with prediction of trajectories of moving objects, warning of the driver, estimation of the risk of collision and building of a Collision-State-map, trying of different acceleration/steering combinations to bring the vehicle to safe driving state and use of hypothetical trajectories. In [45] an digital map of a parking area is used with a Car2X- communication network, so that the position data of mobile objects are detected. This information is used for navigation to a target position with collision avoidance. In [46] a process for collision avoidance and automated configuration of working area of a robot discussed. In [47] a classification of type of object (e.g. bicycle, pedestrian) and a classification and prediction of behavior is presented. Features are adaption and correction of characteristic values and motion planning depending on predictions. In [48] a probabilistic situation analysis is presented for the fusion of Situation Analysis to trigger safety systems. Application is for pre-crash system. In [49] a prediction procedure for trajectories for collision avoidance and the control of velocity is presented. [50] A visual pedestrian detection is described with extraction of a partial image and processing unit with prediction of human behavior. In [51] a communication based vehicle-pedestrian collision warning system with pedestrian detection, prediction of moving objects and ego-vehicle and path collision circuit for detection of collisions is presented. In [52] a communication based vehicle-pedestrian collision warning system is presented. The system includes a base and a mast and a plurality of sensors. The prediction of moving objects and ego-vehicle and a path collision circuit for detection of collisions is described. In [53] a crowd movement prediction using optical flow algorithms is presented with a predictive map of a distribution of objects of interest (OOls). In [54] a computer vision approach for collision avoidance for pedestrians and analysis of the optical flow is presented. In [55] a computer vision approach for estimation of Time to collision (TTC) is presented with use of a plurality of images. In [56], [57] systems for object detection are presented for the usage in autonomous vehicles.

Interacting vehicles

In [58] a new research program is initiated by the German research program for cooperative interacting vehicles. In [59] many aspects about cooperative and interaction based are analyzed for safety reasons.

In [60] a cloud based system for autonomous vehicles is described to assist the internal navigation and motion planning with information from the cloud. In [61] a start-up for optimization of a fleet of autonomous vehicles via a cloud.

REFERENCES [1] Iteam, https://iteam-project.net/, accessed: 2017-03-21.

[2] Waze, waze.com, accessed: 2017-03-14.

[3] wego.here.com, wego.here.com, accessed: 2017-03-14.

[4] here.com, here.com, accessed: 2017-03-14.

[5] tomtom.com, tomtom.com, accessed: 2017-03-14.

[6] H. Winner, S. Hakuli, and G. Wolf, Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme fur aktive Sicherheit und Komfort.

Springer-Verlag, 2011.

[7] Mapillary, mapillary.com, accessed: 2017-03-14.

[8] Placemeter, placemeter.com/, accessed: 2017-03-14.

[9] osm3d, osm-3d.org, accessed: 2017-03-14.

[10] Bostonography, bostonography.com/, accessed: 2017-03-14.

[11] GeOps, geops.de/, accessed: 2017-03-14.

[12] GeOps, tracker.geops.ch, accessed: 2017-03-14.

[13] Devgoogle, developers.google.com/, accessed: 2017-03-14.

[14] Socialapis, https://www.programmableweb.com/news/top-10-social-apis-facebook-twitter-andgoogle-plus/analysis/2015/02/17, accessed: 2017-03-21.

[15] T. Jeske, Sicherheit und Datenschutz in nicht-interaktiven crowdsourcing Szenarien, Ph.D. dissertation, 2015.

[16] C. P. Urmson, I. J. Mahon, D. A. Dolgov, and J. Zhu, Pedestrian notifications, Nov. 24 2015, US Patent 9,196,164.

[17] N. A. Ogale, A survey of techniques for human detection from video, Survey, University of Maryland, vol. 125, no. 133, p. 19, 2006.

[18] xens, xsens.com, accessed: 2017-03-14.

[19] P. Borges, R. Zlot, and A. Tews, Pedestrian detection for driver assist and autonomous vehicle operation using offboard and onboard sensing, in Australian Conference on Robotics and Automation (ACRA), 2010, pp. 1-6.

[20] M. Enzweiler and D. M. Gavrila, Monocular pedestrian detection: Survey and experiments, IEEE transactions on pattern analysis and machine intelligence, vol. 31, no. 12, pp. 2179-2195, 2009.

[21] S. Lefevre, D. Vasquez, and C. Laugier, A survey on motion prediction and risk assessment for intelligent vehicles, Robomech Journal, vol. Ι,ηο. 1, p. 1, 2014.

[22] A. D. V. Govea, Incremental learning for motion prediction of pedestrians and vehicles, Ph.D. dissertation, 2010.

[23] B. D. Ziebart, Modeling purposeful adaptive behavior with the principle of maximum causal entropy, Ph.D. dissertation, 2010.

[24] D. Ellis, E. Sommerlade, and I. Reid, Modelling pedestrian trajectory patterns with Gaussian processes, in Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on. IEEE, 2009, pp. 1229-1234.

[25] C. Goerzen, Z. Kong, and B. Mettler, A survey of motion planning algorithms from the perspective of autonomous uav guidance, in Selected papers from the 2nd International Symposium on UAVs, Reno, Nevada, USA June 8-10, 2009. Springer, 2009, pp. 65-100.

[26] B. Paden, M. “C'ap, S. Z. Yong, D. Yershov, and E. Frazzoli, A survey of motion planning and control techniques for self-driving urban vehicles, IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 3355, 2016.

[27] C. Katrakazas, M. Quddus, W.-H. Chen, and L. Deka, Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions, Transportation Research Part C: Emerging Technologies, vol. 60, pp. 416-442, 2015.

[28] D. Gonz'alez, J. P'erez, V. Milan'es, and F. Nashashibi, A review of motion planning techniques for automated vehicles, IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 4, pp. 11351145, 2016.

[29] J. Choi, Kinodynamic motion planning for autonomous vehicles, International Journal of Advanced Robotic Systems, vol. 11, no. 6, p. 90, 2014.

[30] A. Perez, R. Platt, G. Konidaris, L. Kaelbling, and T. Lozano-Perez, Lqr-rrt*: Optimal sampling-based motion planning with automatically derived extension heuristics, in Robotics and Automation (ICRA), 2012 IEEE International Conference on. IEEE, 2012, pp. 2537-2542.

[31] S. Karaman and E. Frazzoli, Optimal kinodynamic motion planning using incremental samplingbased methods, in Decision and Control (CDC), 2010 49th IEEE Conference on. IEEE, 2010, pp. 76817687.

[32] U. Schwesinger, M. Rufli, P. Furgale, and R. Siegwart, A sampling-based partial motion planning framework for system-compliant navigation along a reference path, in Intelligent Vehicles Symposium (IV), 2013 IEEE. IEEE, 2013, pp. 391-396.

[33] D. J. Webb and J. v. d. Berg, Kinodynamic rrt*: Optimal motion planning for systems with linear differential constraints, arXiv preprint arXiv:1205.5088, 2012.

[34] T. Schouwenaars, Έ. F'eron, and J. How, Safe receding horizon path planning for autonomous vehicles, in PROCEEDINGS OF THE ANNUAL ALLERTON CONFERENCE ON COMMUNICATION CONTROL AND COMPUTING, vol. 40, no. 1. The University; 1998, 2002, pp. 295-304.

[35] A. Richards, T. Schouwenaars, J. P. How, and E. Feron, Spacecraft trajectory planning with avoidance constraints using mixed-integer linear programming, Journal of Guidance, Control, and Dynamics, vol. 25, no. 4, pp. 755-764, 2002.

[36] T. Schouwenaars, B. De Moor, E. Feron, and J. How, Mixed integer programming for multi-vehicle path planning, in Control Conference (ECC), 2001 European. IEEE, 2001, pp. 2603-2608.

[37] T. Schouwenaars, Safe trajectory planning of autonomous vehicles, Ph.D. dissertation, Massachusetts Institute of Technology, 2005.

[38] J. Eilbrecht and 0. Stursberg, Auction-based cooperation of autonomous vehicles using mixed integer planning, AAET-Automatisiertes und vernetztes Fahren 2017, pp. 266-285, 2017.

[39] A. Augst and C. Patron, Verfahren zur Ausfiihrung einer zumindest teilweise automatisierten bewegung eines Fahrzeugs innerhalb eines raumlich begrenzten Bereichs, Patent DE 10 2014 218 429 Al, 03 17, 2016. [Online], Available:

https://register.dpma. de/DPMAregister/pat/register?AKZ=1020142184290 [40] R. Kastner, M. Kleinehagenbrock, M. Nishigaki, H. Kamiya, N. Mori, S. Wako-shi, and Kusuhara, Driver assist system with cut-in prediction, Patent DE 10 2015 200 215 Al, 07 28, 2016.

[41] S. Zecha and R. R. Helmar, Verfahren und vorrichtung zur prdiktion der position und/oder bewegung eines objekts relativ zu einem Fahrzeug, Patent DE 10 2009 035 072 Al, 07 28, 2009.

[42] . W. D. F. M. D.. N. D. G. R. . L. D. H. C. . L. D. H. S. . W. D. I. K. . N. D. K. S. D. . L. D. Fechner, Thomas, Verfahren zum Bestimmen einer Fahrstrategie, Patent DE 10 2014 216 257 Al, 02 18, 2016.

[43] K. Sakai, T. Kindo, and M. Harada, Vorrichtung zum Vorhersagen der Bewegung eines mobilen Korpers, Patent DE 11 2010 000 802 T5, 02 12, 2010.

[44] J. Chassot, G. Ottmar, H. Frederic, S. Paasche, A. Schwarzhaupt, G. Speigelberg, and A. Sulzmann, Verfahren und system zur Vermeidungeiner Kollision eines Kraftfahrzeuges mit einem Objekt, Patent DE 10 2005 023 832 Al, 11 30, 2006.

[45] S. Nordbruch, Verfahren und Vorrichtung zum Betreiben eines Fahrzeugs respektive eines Parkplatzes, Patent DE 10 2014 224 104 Al, 11

26, 2016.

[46] E.-H. Waled, Verfahren und Vorrichtung zum Vermeiden von Kollisionen zwischen Industrierobotern und deren objekten, Patent DE 10226 140 Al, 06 13, 2004.

[47] K. Taguchi, Vorrichtung zur Vorhersage eines Verhaltens, Patent DE 11 2008 002 268 T5, 07 15, 2010.

[48] R. Η. M. S. R. H. Meinecke, Marc-Michael, Probabilistische auslsestrategie, Patent DE 10 2008 046488 Al, 03 11,2010.

[49] n.d., Verfahren und vorrichtung zum prdizieren einer bewegungstrajektorie, Patent DE 10 2006036 363 Al, 04 05, 2007.

[50] T. Kindo, Pedestrian action prediction device and pedestrian action prediction method, Patent EP2 759 998 Al, 09 20, 2011.

[51] L. Caminiti, J. C. Lovell, J. J. Richardson, and C. T. Higgins, Communication based vehicle-pedestrian collision warning system, Dec. 2

2014, US Patent 8,903,640.

[52] L. Caminiti, J. C. Lovell, and J. J. Richardson, Communication based vehicle-pedestrian collision warning system, Mar. 12 2009, US Patent App. 12/403,067.

[53] T. N. Dos Santos, R. C. Folco, and B. H. Leitao, Crowd movement prediction using optical flow algorithm, May 15 2013, uS Patent App. 13/894,458.

[54] D. Rosenbaum, A. Gurman, Y. Samet, G. P. Stein, and D. Aloni, Pedestrian collision warning system, Dec. 29 2015, US Patent App. 14/982,198.

[55] G. Stein, E. Dagan, O. Mano, and A. Shashua, Collision warning system, Jun. 29 2015, US Patent App. 14/753,762.

[56] J. Zhu, M. S. Montemerlo, C. P. Urmson, and A. Chatham, Object detection and classification for autonomous vehicles, Jun. 5 2012, US Patent 8,195,394.

[57] --, Object detection and classification for autonomous vehicles, Oct. 28 2014, US Patent

8,874,372.

[58] C. Stiller, W. Burgard, B. Deml, L. Eckstein, F. Flemisch, F. K'oster, M. Maurer, and G. Wanielik, Kooperativ interagierende Automobile.

[59] M. Naumann, P. F. Orzechowski, C. Burger, 0. S^. Tas_j7 and C. Stiller, Herausforderungen fur die Verhaltensplanung kooperativer automatischer Fahrzeuge.

[60] S. Kumar, S. Gollakota, and D. Katabi, A cloud-assisted design for autonomous driving, in Proceedings of the first edition of the MCC workshop on Mobile cloud computing. ACM, 2012, pp. 41-46.

[61] bestmile, https://bestmile.com/, accessed: 2017-03-21.

[62] Itenach, http://www.lte-anbieter.info/5g/, accessed: 2017-03-22.

DESCRIPTION OF INVENTION

The invention presents a method to predict pedestrian movements in (un-)structured environments. In uncertain and complex urban environments it is a core challenge to find safe trajectories for the egovehicle. Especially in cases in which pedestrians are detected, it is often necessary to predict their movement. In state of the art motion planning procedures it is often assumed to have historical data ora pedestrian model to predict the movement for the motion planning. It is often necessary to have a prediction for the movement of a pedestrian to generate a motion plan. It is proposed to use a trajectory generator, which incorporates the static environment of the city. The static environment constrains the behavior of the pedestrian. Benefit of the invention is that the behavior of pedestrians can be approximated and predicted without any observations. After the movement prediction, a motion plan can be generated and leads to subsequent control.

The biomechanics of the human body and the environment constrain the behavior of pedestrians physically. In labyrinths the movement is constrained by walls. Athletic persons have other capabilities in their dynamics than non-athletic persons. Humans have also some mental constrains. In public places the movement is also constrained by social and civilly regularities. The invention addresses a new concept of topological spaces in urban environments.

In a first step, the structure of the environment is read with use of maps. In a second step optimal grids are generated and the tangency space of the structured environment is learned with algorithms (e.g. manifold learning). In a third step the tangency space and their orthogonal space is used for generating a vector field, especially in pedestrian movable zones. In a fourth step the synthetic movements of the person are generated, depending on the position of the pedestrian. In a fifth step the generated dataset of synthetic pedestrian movements are used for learning and prediction of the pedestrian movements. In a sixth step the motion plan for the ego-vehicle is generated with use of the movement prediction and uncertainty quantification. In a seventh step, the motion planning depending on computed information is carried out, in an eighth step the vehicle is controlled according to the reference trajectory of the motion planning.

Figure 1 shows the flow chart covering the six steps of the invention.

Figure 2 shows the concept of spatial probabilistic manifolds. Some newer approaches for pedestrian movement prediction follow the concept of adapting graphs [22] or vector fields [24], [53]. They can be used on observed movements and can be adapted for complex environments. In this invention it is assumed that pedestrians follow in probabilistic topological spaces. This is a new result based on a combination of concepts followed by differential geometry, the concept of manifolds and machine learning. Advantage of this approach is that the movement prediction is more accurate, environmental conditions can be incorporated. Uncertainty propagation is combined with environmental structures. In daily live there exist many (urban) environments, where it is obvious where a pedestrian might walk (static obstacles like buildings or tunnels). Nevertheless there are some uncertain aspects (intention, biomechanics), which makes movement prediction uncertain. In Figure 2 an example illustrates the concept. The filled geometric structures represent buildings and around there are probabilistic descriptors. The descriptors are connected through a topological space (differential geometry). From each descriptor a movement prediction is possible. The uncertainty propagation is constrained by the environment. This concept leads to more accurate movement predictions and advantage for the usage in the mentioned motion planning. Each descriptor can be an independent machine learning unit. The mentioned concept can be seen as a generalization of adapting graphs and vector fields. Step 4 of Figure 1 can be replaced by probabilistic manifolds. Further (semantic) information can be incorporated (time, intention of pedestrian) in more advanced descriptors.

It is also possible to get synthetic trajectories of pedestrians without observations. It is necessary to have an environment model only with probabilistic manifold. If some trajectories are generated with crossing of obstacles the probabilistic distribution of the probabilistic descriptors have to be adapted.

Claims (8)

1. Method for predicting pedestrian movements in structured and unstructured environments to be used for motion planning and control, compromising, in a first step reading of map data where it is planned to drive, in a second step reading the structure of static obstacles (e.g. buildings) or quasi-static obstacles and dynamic obstacles (e.g. pedestrians), in a third step computation of specific spaces like tangency space and orthogonal space and optimal grids, in a fourth step computation of vector fields, in a fifth step configuration of a pedestrian model, in a sixth step computation movement prediction on last captured pedestrian position, in a seventh step motion planning, in an eighth step control of the vehicle.

2. Method according to claim 1 characterized by the adaption by measurements of new observations of dynamic obstacles.

3. Method according to claim 1 characterized by the adaption with usage of uncertainty quantification methods in motion planning.

4. Method according to claim 1 characterized by the adaption with additional measurements like vision based (i.e. gesture of pedestrian) and context based information about the urban environment.

5. Method according to claim 1 characterized by the adaption with probabilistic manifold descriptors and topological spaces for movement prediction.

6. Method according to claim 1-5 characterized by the generation of pedestrian trajectories and environment model without observations based on statistical assumptions and with probabilistic manifold descriptors.

7. Method according to claim 1-6 characterized by the incorporation of network descriptors to capture topological spaces of humans and learn their dynamics.

8. Method according to claim 1-6 characterized by hierarchically concepts of network descriptors.