DE102022132812A1 - METHOD FOR DISPLAYING A VISUAL REPRESENTATION OF A DESTINATION - Google Patents

Abstract

Examples relate to a method (10) for displaying a visual representation (20b, 30b) of a destination (21, 31) of a route planner in a vehicle. The method (10) comprises the steps of: providing (11) a general visual representation (20a, 30a) of the destination (21, 31); providing (12) current information (24a, 34a) about the destination (21, 31); manipulating (13) the general visual representation (20a, 30a) using the current information (24a, 34a); and displaying (14) the manipulated visual representation (20b, 30b) on a display in the vehicle.

Description

Technical area

Embodiments relate to a method for displaying a visual representation of a destination of a route planner in a vehicle. According to the method, it can be possible to display current visual representations of the destination.

background

In modern vehicles, route planners or navigation devices are state of the art. In addition to driving routes, navigation systems are increasingly displaying contextual information, such as buildings, sights or other information. This information is now displayed as simplified graphics or as a 3D model. The display of webcam images, for example at destinations or selected locations, is also known. Maps that integrate image data, such as 360-degree photos, are also known.

EN 10 2022 103 881 A1 discloses a novel simulator that is controllable to independently adjust a simulation based on an input that identifies space-independent information called a "subject" (e.g., weather, day vs. night, etc.), an input that identifies space-dependent information representing objects to be included in a scene (e.g., adding or removing trees, buildings, cars, pedestrians, etc.), and an input that provides a response to one or more previous frames of a simulation.

EP 3 002 559 A2 discloses a route display method for display on a terminal, comprising: acquiring start point information indicating a start point and destination information indicating a destination; generating route information indicating a route from the start point to the destination, the route information including a plurality of through points; extracting captured images corresponding to the plurality of through points from a database storing the captured images respectively with respect to corresponding image capture locations, each of the image capture locations being within a range of each of the plurality of through points; and sequentially displaying the extracted captured images in conjunction with a map image in an order from a first captured image corresponding to a first through point closest to the start point to a second captured image corresponding to a second through point closest to the destination.

Summary

A disadvantage of other concepts for displaying images of a destination is that such images cannot show the current, real conditions at the destination, but rather show the destination in the situation at the time the image was taken. In some cases, according to other concepts, current images of the destination may not be available for display.

It is an object of the present disclosure to provide improved concepts for the visual representation of a destination of a navigation system.

This object is achieved according to the subject matter of the independent patent claims. Further advantageous embodiments are described in the dependent patent claims, the following description and in conjunction with the figures.

Accordingly, a method for displaying a visual representation of a destination of a route planner in a vehicle is proposed. The method includes providing a general visual representation of the destination and providing current information about the destination. The general visual representation is manipulated using the current information. Finally, the manipulated visual representation is displayed on a display in the vehicle.

The general visual representation can, for example, include an image or photo of the destination. The destination can be an area surrounding a destination address in the route planning, in particular a building or a landmark positioned at the destination address. A neutral general visual representation can advantageously be used in the method. Some representations of locations can, for example, show severe weather conditions such as rain or snow or large crowds of people. The neutral general visual representation, on the other hand, can show a slight influence of weather conditions (e.g. blue sky) and few or no people can be shown (e.g. a pure shot of an object at the destination with or without its surroundings). One advantage of using such a neutral general visual representation can be that the subsequent manipulation can be carried out better or more easily.

Providing current information about the destination may include providing a time of day, weather conditions or current visitor numbers at the destination. These can be recorded in a current manner, e.g. by means of information retrieval from an online service. Publicly available data can advantageously be used to carry out the procedure, which can provide precise information about the current situation at the destination.

The manipulation of the general visual representation can be done in particular using image processing methods. For example, based on the current information about the destination, areas or content of the general visual representation that do not match the current information can be removed (e.g. a blue sky can be removed and replaced with clouds or rain clouds if the current information shows clouds or rain clouds). For example, content can be added to the general visual representation using image processing methods if it is not included in the general visual representation (e.g., depending on the current information, a few or many people can be inserted into the general visual representation if the current information shows a high volume of visitors). To adapt the general visual representation in a particularly realistic and situation-appropriate manner, an artificial intelligence AI, e.g. a text-to-image AI, can advantageously be used during manipulation.

Advantageously, the generated manipulated visual representation can be shown on a vehicle display that also shows the navigation route. Alternatively, the manipulated visual representation can also be shown on a mobile user device in the vehicle, for example if navigation is carried out using this user device.

One advantage of the proposed method is that the current situation at the destination can always be visually presented to the vehicle user. While other methods show general images of the destination, the method gives the user a current picture of the situation at the destination (e.g. the destination is shown with a high number of visitors if this corresponds to the current situation there, instead of a general, e.g. deserted, picture of the destination). This allows the user to better estimate what to actually expect at the destination.

According to one aspect of the disclosure, the manipulation of the general visual representation is carried out by means of artificial intelligence (AI) based on the current information. In this case, individual image areas can be selected (e.g. segments) that correspond in content to the current information. For example, first segments can be selected for the representation of weather conditions and (e.g. other) second segments can be selected for the representation of people at the destination. The manipulation can be carried out by the AI using in-paint areas to match the respective information content.

According to one aspect of the disclosure, it is accordingly provided that the general visual representation is divided into several segments for manipulation, wherein a segment matching the current information in terms of content is selected for manipulation and the selected segment (e.g. exclusively the selected segment) is manipulated based on the current information.

According to one aspect of the disclosure, the current information comprises at least two different content aspects, wherein the different aspects of the current information are used to manipulate different, separate segments.

According to one aspect of the disclosure, it is provided that a first segment of the general visual representation is determined for the manipulation, which is manipulated based on the current information, and that a second segment of the general visual representation is determined for the manipulation, which is not manipulated based on the current information. For example, the first segment may be suitable for inserting and representing people therein (e.g., floor area) while the second segment may be unsuitable for representing people (e.g., sky or treetops).

By segmenting the general visual representation into several segments depending on the content of the current information, it can be achieved that only those image areas are manipulated based on the current information which are also suitable for reproducing the current information.

According to one aspect of the disclosure, the general visual representation of the destination includes a captured photo of the destination or an artificially generated representation of the destination. Providing the general visual representation may, for example, include selecting a photo of the destination from a database, e.g. from an online database. For example, neutral photographic images of the destination may be inserted or provided in the database, which may be particularly well suited for visual manipulation.

According to one aspect of the disclosure, providing the general visual representation includes selecting such a Select a photo that best matches the current information (e.g. time or weather).

According to one aspect of the disclosure, the current information is determined using an Internet source (e.g., weather forecast or current weather information from an online weather service; e.g., crowds at the destination). Such information may be publicly available and easy to use in the method.

Alternatively or additionally, according to one aspect of the disclosure, the current information is determined by means of a vehicle sensor. For example, the current weather in the vicinity of the vehicle can be determined by means of a vehicle camera if the vehicle is in a predetermined environment to the destination (e.g. sufficiently similar weather in the vicinity of the destination). Similar weather conditions can be assumed, for example, at a distance of less than 10 km (or less than 5 km) to the destination (e.g. in generally stable weather conditions). For example, the expected time of arrival can be determined (e.g. based on the speed sensor and GPS position) so that the time of day when the destination is reached can be taken into account as current information about the destination.

According to one aspect of the disclosure, the general visual representation is manipulated again when the current information about the destination changes. For this purpose, the current information can be made available again (e.g. the current information can be determined again, e.g. retrieved online again). For example, updates can be made at predetermined time intervals (e.g. hourly, half-hourly, every 10 minutes or every 5 minutes) in order to always be able to display up-to-date manipulated visual representations (e.g. synthetic images) about the destination. For example, an online service can send a push message when the current information changes and the update can take place after receiving the push message. Since the human eye is sensitive to even minor visual changes, the user can better assess when the current situation at the destination changes and in what way the current situation changes.

A further aspect relates to a computer program product or non-volatile storage medium with a program code to carry out the method described above or below when the computer program is executed on a processor, a computer, or programmable hardware. Such a program can advantageously be executed on an on-board computer, a mobile user device, or a backend (e.g. a server or cloud service coupled to the vehicle and/or a mobile device of the vehicle user) to carry out the proposed method and thus provide the manipulated visual representation for the vehicle user. An advantage of creating the manipulated visual representation using an online service can be that the manipulated visual representation can be used multiple times, e.g. if multiple vehicle users travel to the same destination at the same or sufficiently similar time.

Character description

• 1 ein Flussdiagramm eines Verfahrens zum Anzeigen einer visuellen Darstellung eines Zielortes eines Routenplaners;
• 2a, 2b ein schematisches Beispiel einer Manipulation einer allgemeinen visuellen Darstellung eines Zielortes; und
• 3a-3d schematische Beispiele von auf unterschiedliche Weise manipulierten visuellen Darstellungen basierend auf einer allgemeinen visuellen Darstellung und einer aktuellen Information.

Examples of embodiments are explained in more detail below with reference to the accompanying figures. They show:

• 1 a flowchart of a method for displaying a visual representation of a destination of a route planner;
• 2a , 2 B a schematic example of a manipulation of a general visual representation of a destination; and
• 3a-3d schematic examples of visual representations manipulated in different ways based on a general visual representation and current information.

Description

Various embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are illustrated. In the figures, the thickness dimensions of lines, layers and/or regions may be exaggerated for the sake of clarity. In the following description of the accompanying figures, which only show some exemplary embodiments, like reference numerals may designate like or comparable components.

An element that is referred to as being "connected" or "coupled" to another element may be directly connected or coupled to the other element, or there may be intervening elements. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning that one of ordinary skill in the art to which the embodiments belong.

Navigation systems are known that can display image information about destinations. However, the image information displayed according to the current state of the art, e.g. photos of sights or images of destinations, is not up-to-date according to other concepts. For example, they do not contain a daily/up-to-date representation of the actual situation at the destination. For example, sights may have been photographed on days when there are no people, but the destination is very busy on the day of arrival. Or images may be taken in summer, but the journey takes place in winter. Although systems are known that integrate webcam images into navigation systems, these are only available in very few locations. 3D representations are very complex if they are highly detailed. It is desirable to provide a method that generates (e.g. high-resolution) images with current context knowledge.

1 shows a flow chart of a method 10 for displaying a visual representation of a destination of a route planner. The method 10 comprises the steps of: providing 11 a general visual representation of the destination; 12 providing current information about the destination; 13 manipulating the general visual representation using the current information; and displaying 14 the manipulated visual representation on a display in the vehicle.

The proposed method 10 can advantageously make it possible to provide the vehicle user with a current representation of the destination. Based on a (e.g. neutral) general visual representation of the destination, the intended manipulation can generate a representation that is adapted to the current situation at the destination (e.g. weather, time of day or the number of people or vehicles at the destination). This allows the vehicle user to better estimate what environmental conditions he can expect at the destination. The image manipulation used in the method can generate a current representation of the destination, e.g. without a large selection of images of the destination in different situations being required for the current display of the destination.

2a , 2 B show a schematic example of manipulating a visual representation of a destination. 2a shows a general visual representation 20a of a destination 21, e.g. a famous building 21. 2 B shows a manipulated visual representation 20b of the same destination 21.

For the manipulation 13, 10 different, current information 24a, 24b, 24c about the destination 21 are provided according to the method (purely schematically in 2a shown). The current information can, for example, relate to the weather at the destination 21 (e.g. first weather information 24a about sunshine or second weather information 24b about cloud cover) or relate to a current number of visitors at the destination 21 (e.g. visitor information 24c about the number of people at the destination 21).

The general visual representation 20a can be divided into several segments, which can be used differently for manipulation 13 based on various current information 24, 24b, 24c (eg textual information) about the destination 21. In the example of 2a , 2 B it is shown that the general visual representation 20a is divided into two segments 22, 23 (e.g. guideline segments). Object detection and segmentation can take place on targeted areas. Different image areas can be suitable to different degrees for displaying different information. For example, in-paint areas can be defined for different information (e.g. time, weather, news (e.g. about sporting events etc.), traffic information such as high or low volume). For example, the upper segment 22 (e.g. guideline segment for weather / time; e.g. sunset, sunrise, sunny or cloudy is predominantly shown in this segment) can be used to display information 24a, 24b about the weather, while the lower segment 23 (e.g. guideline segment for people; e.g. people can only appear in this image area) can be used to display current information 24c about the number of visitors. General information such as arrival during the day or at night can, for example, affect all segments of the visual representation. For example, the upper guideline segment 22 can include a representation of the sky, so that information on the weather or time of day can be visually displayed there, but not on the current number of visitors. Information on the current number of visitors 24c, on the other hand, can be displayed in the lower segment 23 (eg image area with ground or green space; eg determined by means of object detection).

For example, at a first point in time (e.g. during the journey), high-resolution graphic images can be calculated based on the guideline segments 22, 23 and context data 24a - 24c (time, people, weather, etc.). For example, at a second point in time, high-resolution graphic images can be calculated again based on the guideline segments 22, 23 and context data 24a-24c (time, people, weather, etc.) if changes occur (e.g. updating). In this case, an update can advantageously only take into account individual information and the segments corresponding to it. that have changed, which means that, for example, the entire manipulated visual representation does not have to be recalculated. For possible route points, for example, at the start of the journey (e.g. before the first point in time mentioned above), guideline segments are determined that can be dynamically adjusted during the journey (for example at the first and/or second point in time). This enables efficient image calculation. Using artificial intelligence AI, only the image material needs to be regenerated at which the current information 24a - 24c changes. For example, dividing into segments 22, 23 is only necessary once.

According to the available current information 24a - 24c, 2 B a manipulated visual representation 20b is shown. In the upper segment 22, visual representations of the weather corresponding to the corresponding information 24a, 24b (eg sun and clouds; eg weather at the destination) have been inserted. In the lower segment 23, a visual representation of the number of visitors corresponding to the corresponding current information 24c (eg number of people at the destination) has been inserted.

For example, an image of a sight 21 and a current weather situation 24a at the destination and/or a photo of the sight 21 and a current number of visitors 24c at the location can be used as input for the process.

It is clear to the expert that the 2a , 2 B shown visual representations are to be viewed purely schematically to illustrate the technical principle of the disclosure. In the practical implementation of the method, for example, photorealistic manipulated visual representations 20b can be generated and displayed by means of image recordings as a general visual representation 20a and in particular by using AI for manipulation 13. The same applies to the following schematic examples of the 3a-3d .

Further details and aspects are mentioned in connection with the embodiments described above or below. 2a , 2 B The embodiment shown may comprise one or more optional additional features corresponding to one or more aspects described in connection with the proposed concept or with one or more of the above (eg 1 ) or below (e.g. 3a-3d ) described embodiments.

3a-3d show schematic examples of differently manipulated visual representations starting from a general visual representation and current information. 3a shows a general visual representation 30a of a schematically represented building 31 (eg a sight at the destination). Schematically represented current information 34a can, for example, relate to the weather at the destination, for example the current information 34a can indicate that there is currently snow at the destination.

The current information 34a can be used according to the method to manipulate the general visual representation 30a so that the output representation of the building 31 at the destination is current; in this example, snow on the building 31 can be inserted into the representation. For this purpose, an area in which snow can be seen can be determined by segmentation (e.g. by analyzing physical conditions), here, for example, the roof of the building. For a more realistic representation, different forms of the existing snow can be inserted into the general visual representation 30a, so that, for example, when repeatedly driving to the same destination, a different manipulated visual representation can be output each time.

Various forms of manipulated visual representations 30b, 30c, 30d are shown in the 3b - 3d shown. The basic information 34a used, that there is snow, is always the same. However, in the first manipulated visual representation 30b, a snow cover 34a_1 shown on the building roof is distributed over a larger area than, for example, in the third manipulated visual representation 30d, which shows a more compact snow distribution 34a_3. In contrast, the snow distribution 34a_2 shown in the second manipulated visual representation 30c has an asymmetry. The actual manifestation of the manipulated visual representation used for display in the navigation system can, for example, be random.

Further details and aspects are mentioned in connection with the embodiments described above or below. 3a-3d The embodiment shown may comprise one or more optional additional features corresponding to one or more aspects described in connection with the proposed concept or with one or more of the above (eg 1 - 2b) or embodiments described below.

The present disclosure relates to context-dependent, in particular AI-based image generation for use in navigation systems. Examples relate to a method for generating synthetic images based on real photos and context knowledge from the vehicle (e.g. weather such as snow or rain near the destination, time of arrival; e.g. based on vehicle sensor data) and environmental systems (e.g. weather conditions at the location, number of visitors at the location; e.g. based on data from online services). The system uses, for example, a real high-resolution photo, e.g. of a sight at the destination, and segments it into areas. Based on information from the vehicle and environmental systems, a high-resolution image is then generated, e.g. via a text-to-image AI system, and displayed in the navigation device. The process or system can also be applied to complete maps or landscapes. In addition, if, for example, no image information is available, it is possible to generate fully synthetic photos (e.g. synthetic generation of the general visual representation, e.g. using AI).

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 102022103881 A1 [0003]
• EP 3002559 A2 [0004]

Claims (10)

Method (10) for displaying a visual representation (20b, 30b) of a destination (21, 31) of a route planner in a vehicle, the method (10) comprising: - providing (11) a general visual representation (20a, 30a) of the destination (21, 31); - providing (12) current information (24a, 34a) about the destination (21, 31); - manipulating (13) the general visual representation (20a, 30a) using the current information (24a, 34a); and - displaying (14) the manipulated visual representation (20b, 30b) on a display in the vehicle. Procedure (10) according to Claim 1 , wherein the manipulation (13) of the general visual representation (20a, 30a) is carried out by means of an artificial intelligence based on the current information (24a, 34a). Procedure (10) according to Claim 1 or 2 , wherein the general visual representation (20a) is divided into several segments (22, 23) for the manipulation (13), wherein a segment (22) matching the current information (24a) in terms of content is selected for the manipulation (13) and the selected segment (22) is manipulated (13) based on the current information (24a). Procedure (10) according to Claim 3 , wherein the current information (24a, 24c) comprises at least two different content aspects, wherein the different aspects of the current information (24a, 24c) are used to manipulate (13) different, separate segments (22, 23). Procedure (10) according to Claim 3 or 4 , wherein for the manipulation (13) a first segment (22) of the general visual representation (20a) is determined which is manipulated (13) based on the current information (24a), and wherein for the manipulation (13) a second segment (23) of the general visual representation (20a) is determined which is not manipulated (13) based on the current information (24a). Method (10) according to one of the preceding claims, wherein the general visual representation (20a, 30a) of the destination (21, 31) comprises a captured photo of the destination (21, 31) or an artificially generated representation of the destination (21, 31). Method (10) according to one of the preceding claims, wherein the provision (11) of the general visual representation (20a, 30a) comprises selecting from a selection of taken photos of the destination (21, 31) a photo that most closely corresponds in content to the current information (24a, 34a). Method (10) according to one of the preceding claims, wherein the current information (24a, 34a) for providing (12) the current information (24a, 34a) is determined using an Internet source. Method (10) according to one of the preceding claims, wherein the current information (24a, 34a) for providing (12) the current information (24a, 34a) is determined by means of a vehicle sensor. Method (10) according to one of the preceding claims, wherein a renewed manipulation (13) of the general visual representation (20a, 30a) takes place when the current information (24a, 34a) about the destination (21, 31) changes.