CN116136408A - Indoor navigation method, server, device and terminal - Google Patents

Abstract

The present application proposes an indoor navigation method, server, device and terminal, which relate to the field of augmented reality technology. The method includes: obtaining real-time indoor actual location information of an augmented reality AR device, where the actual location information is used to represent the location information of the AR device in a world coordinate system; matching the actual location information with a preset navigation map to determine real-time positioning information ; According to the real-time positioning information and the obtained guidance route, determine the virtual guidance information, the guidance route is the route determined based on the acquired initial position information and target position information of the AR device; send the virtual guidance information to the AR device, so that the AR device Generate and dynamically display virtual navigation images based on virtual guidance information. Improve the accuracy of navigation through virtual guidance information; send virtual guidance information to AR devices, so that AR devices can generate and dynamically display high-precision virtual navigation images based on virtual guidance information, so as to intuitively guide users to quickly go to the target location.

Description

Indoor navigation method, server, device and terminal

technical field

The present application relates to the field of augmented reality technology, in particular to an indoor navigation method, server, device and terminal.

Background technique

With the continuous development of cities, large buildings (for example, airports, high-speed rail stations, shopping malls and high-rise office buildings, etc.) continue to emerge. When people move in various large buildings, it is easy to have the problem of being unable to locate. With the increase of spatial data acquisition methods, portable panoramic cameras can be used to collect visual data, and then these visual data can be processed to achieve indoor positioning.

However, due to the complexity of the indoor environment, people's precise requirements for indoor navigation are also increasing. The existing indoor two-dimensional navigation method cannot intuitively guide the user to quickly go to the target location; moreover, the indoor two-dimensional navigation method also has the problem of low navigation accuracy, which cannot meet the user's high-precision navigation needs.

Contents of the invention

The present application provides an indoor navigation method, server, device and terminal.

An indoor navigation method provided by an embodiment of the present application, the method includes: obtaining real-time indoor actual position information of an augmented reality AR device, the actual position information is used to represent the position information of the AR device in the world coordinate system; the actual position information Match the preset navigation map to determine real-time positioning information; determine virtual guidance information based on the real-time positioning information and the obtained guidance route, and the guidance route is a route determined based on the obtained initial position information and target position information of the AR device; Send the virtual guide information to the AR device, so that the AR device generates and dynamically displays the virtual navigation image according to the virtual guide information.

Another embodiment of the present application provides an indoor navigation method, the method includes: sending the actual location information of the augmented reality AR device indoors to the server, so that the server matches the actual location information with the preset navigation map, and determines the real-time positioning information , and according to the guidance route and real-time positioning information, generate and send virtual guidance information to the AR device, wherein the guidance route is a route determined based on the acquired initial position information and target position information of the AR device, and the actual position information is used to represent the AR The location information of the device in the world coordinate system; in response to the virtual guidance information sent by the server, generate a virtual navigation image; dynamically display the virtual navigation image.

A server provided in an embodiment of the present application includes: an acquisition module configured to acquire real-time indoor actual location information of an augmented reality AR device, where the actual location information is used to represent the location information of the AR device in a world coordinate system; a matching module , configured to match the actual location information with a preset navigation map to determine real-time positioning information; the determination module is configured to determine virtual guidance information based on the real-time positioning information and the obtained guidance route, and the guidance route is based on the obtained The route determined by the initial location information and target location information of the AR device; the first sending module is configured to send virtual guidance information to the AR device, so that the AR device generates and dynamically displays a virtual navigation image according to the virtual guidance information.

An augmented reality AR device provided in an embodiment of the present application includes: a second sending module configured to send the actual location information of the augmented reality AR device indoors to the server, so that the server compares the actual location information with the preset navigation map Matching, determining real-time positioning information, and generating and sending virtual guiding information to the AR device according to the guiding route and real-time positioning information, wherein the guiding route is a route determined based on the acquired initial position information and target position information of the AR device; generating The module is configured to generate a virtual navigation image in response to the virtual guidance information sent by the server; the display module is configured to dynamically display the virtual navigation image.

A terminal provided in an embodiment of the present application includes: at least one augmented reality AR device, and the augmented reality AR device is configured to implement any indoor navigation method in the embodiments of the present application.

An embodiment of the present application provides an electronic device, including: one or more processors; memory, on which one or more programs are stored, and when one or more programs are executed by one or more processors, one or more processors to implement any indoor navigation method in the embodiments of the present application.

An embodiment of the present application provides a readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program is executed by a processor, any indoor navigation method in the embodiments of the present application is implemented.

According to the indoor navigation method, server, device, and terminal of the embodiments of the present application, the actual position of the AR device in the world coordinate system can be determined by obtaining the actual indoor position information of the augmented reality AR device in real time; Match the actual location information in the preset navigation map, and can map the actual location information to the preset navigation map in two-dimensional space, so as to determine the real-time positioning information corresponding to the AR device in the preset navigation map, which is convenient for follow-up Processing; according to the real-time positioning information and the obtained guidance route, determine the virtual guidance information, wherein the guidance route is a route determined based on the acquired initial position information and target position information of the AR device, and match the real-time positioning information with the guidance route , determine the virtual guidance information that needs to be provided to the AR device to improve the accuracy of navigation; send the virtual guidance information to the AR device, so that the AR device can generate and dynamically display high-precision virtual navigation images based on the virtual guidance information, and provide intuitive guidance Users quickly go to the target location, improving navigation accuracy.

Regarding the above embodiments and other aspects of the present application and their implementation, more descriptions are provided in the description of the drawings, the detailed description and the claims.

Description of drawings

Fig. 1 shows a schematic flowchart of an indoor navigation method provided by an embodiment of the present application.

Fig. 2 shows a schematic flowchart of an indoor navigation method provided by another embodiment of the present application.

Fig. 3 shows a schematic flowchart of an indoor navigation method provided by yet another embodiment of the present application.

FIG. 4 shows a block diagram of a server provided by an embodiment of the present application.

FIG. 5 shows a block diagram of an augmented reality AR device provided by an embodiment of the present application.

FIG. 6 shows a block diagram of a terminal provided by an embodiment of the present application.

Fig. 7 shows a block diagram of the composition of the indoor navigation system provided by the embodiment of the present application.

Fig. 8 shows a schematic flowchart of a navigation method of an indoor navigation system provided by an embodiment of the present application.

Fig. 9 shows a structural diagram of an exemplary hardware architecture of a computing device capable of implementing the indoor navigation method and apparatus according to the embodiments of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the application clearer, the embodiments of the application will be described in detail below in conjunction with the accompanying drawings. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

With the continuous update of users' needs for navigation technology, the traditional positioning and navigation technology based on the Global Positioning System (GPS) and the positioning and navigation technology based on radio frequency signals cannot meet the user's high-precision navigation needs. .

Moreover, in the indoor environment, due to the time-varying illumination provided by different light sources, the time-varying nature of the indoor deployment environment, and the occlusion of obstacles and changes in the user's observation angle, it is easy to cause the collected positioning information and The navigation information is inaccurate and cannot meet the high-precision navigation needs of users.

Fig. 1 shows a schematic flowchart of an indoor navigation method provided by an embodiment of the present application. The indoor navigation method can be applied to a server. As shown in FIG. 1 , the indoor navigation method in the embodiment of the present application includes at least but not limited to the following steps.

Step S101, acquiring real-time indoor location information of the AR device in real time.

Wherein, the actual location information is used to represent the location information of the augmented reality (Augmented Reality, AR) device in the world coordinate system. The world coordinate system can be defined as: taking the center of a small circle as the origin o, the x-axis is horizontal to the right, the y-axis is vertically downward, and the direction of the z-axis is determined according to the right-hand rule. When doing graphics transformation, the world coordinate system can be used as the starting coordinate space.

By obtaining the actual location information in real time, the indoor location of the AR device can be updated in time, and then the actual location information can be processed to improve the positioning accuracy of the AR device.

Step S102, matching the actual position information with the preset navigation map to determine real-time positioning information.

Wherein, the preset navigation map may include a plane map in the building to be navigated, and the preset navigation map may be a two-dimensional plane map.

The actual location information represents the location information of the AR device in the three-dimensional space. By mapping the three-dimensional actual location information to the two-dimensional preset navigation map, the real-time positioning information of the AR device in the two-dimensional planar map can be determined. Determine the positioning information of the AR device in real time to ensure the accuracy of the real-time information of the AR device. Moreover, the acquired real-time positioning information can facilitate subsequent auxiliary positioning and navigation update of navigation information.

Step S103, determining virtual guidance information according to the real-time positioning information and the obtained guidance route.

Wherein, the guidance route is a route determined based on the acquired initial position information and target position information of the AR device.

For example, the initial location information may include the location of the AR device when it initially enters the navigation system, and the target location information represents the destination location information that the AR device needs to reach. By planning the initial location information and the target location information, multiple suitable navigation paths are found, and the navigation path with the shortest path is found from the multiple navigation paths, and the navigation path is used as the guidance path, so that the AR device can use the navigation path according to the navigation path. Guide the route to reach the end position as quickly as possible.

In the process of providing navigation information for the AR device, the AR device needs to provide real-time positioning information, so that the current server can match the real-time positioning information corresponding to the AR device with the guidance route, dynamically adjust the guidance route in real time, and generate virtual guidance information. The virtual guidance information can avoid the influence of factors such as obstruction of obstacles and changes in observation angles, and send accurate navigation information to the AR device, and use the virtual guidance information to prompt the AR device in time, so as to avoid the AR device from going to the wrong position and prompt the navigation to be accurate To meet the user's high-precision navigation needs.

Step S104, sending virtual guide information to the augmented reality device.

After the AR device obtains the virtual guidance information, it generates and dynamically displays the virtual navigation image according to the virtual guidance information. Since the AR device is a device that can support dynamic and three-dimensional display of image information or video information, it can display navigation information in real time and dynamically through virtual navigation images, which is convenient for users to intuitively view navigation information and the position of the AR device in the actual environment. information to improve navigation accuracy.

In this embodiment, the real position of the AR device in the world coordinate system can be determined by obtaining the actual position information of the augmented reality AR device in the room in real time; Matching, the actual location information can be mapped to the preset navigation map on the two-dimensional space to determine the real-time positioning information corresponding to the AR device in the preset navigation map to facilitate subsequent processing; according to the real-time positioning information and the acquired Guide route, determine the virtual guide information, where the guide route is determined based on the acquired initial position information and target position information of the AR device, match the real-time positioning information with the guide route, and determine the virtual guide that needs to be provided to the AR device Information to improve the accuracy of navigation; send virtual guidance information to the AR device, so that the AR device can generate and dynamically display high-precision virtual navigation images based on the virtual guidance information, so as to intuitively guide users to the target location quickly and improve navigation accuracy sex.

Fig. 2 shows a schematic flowchart of an indoor navigation method provided by another embodiment of the present application. The indoor navigation method can be applied to a server. The difference between this embodiment and the previous embodiment is that: the actual location information includes the real-scene view corresponding to the actual indoor position of the AR device; the real-scene view corresponding to the actual indoor position of the AR device is processed by a deep learning neural network, which can be detailed The positioning features in the real-view view corresponding to the actual position are optimized to improve positioning accuracy.

As shown in FIG. 2 , the indoor navigation method in the embodiment of the present application includes at least but not limited to the following steps.

In step S201, the actual indoor location information of the augmented reality AR device is obtained in real time.

Wherein, the actual location information includes: a real scene view corresponding to an actual indoor location of the AR device. The real scene view corresponding to the actual position of the AR device in the room may include views of multiple levels. For example, the real scene view may be a panoramic view captured by a panoramic camera device and/or a panoramic camera device, or a partial area view captured by an ordinary camera device that can be seen by an observer. The real-view view above is only an example, and the viewing range of the view can be set according to actual needs. Other unexplained real-view views are also within the protection scope of the present application, and will not be repeated here.

Through the real scene view, the actual location information of the AR device in the room can be displayed from multiple angles, avoiding the omission of orientation information, making the obtained actual location information more comprehensive and convenient for subsequent processing.

Step S202, based on the deep learning neural network, process the real-scene view corresponding to the actual indoor location of the AR device to obtain location information to be matched.

Wherein, the real scene view is a view collected by an AR device. Reality views may include: partial area views or panoramic views. The location information to be matched is used to represent the location information of the AR device in the building to be navigated.

In some embodiments, the deep learning neural network can be used to extract the image information in the partial area view and/or the panoramic view, and then extract the features of the image information to obtain the processed image information, so that the processed image information is more accurate. The relative position of the AR device can be reflected, so that the relative position of the AR device can represent the position information to be matched.

For example, it is possible to stratify the views of some areas that the observer can see captured by ordinary camera devices, and divide the views of each level into blocks, and then classify each block of views to further refine each level The category corresponding to the view, and refine the relative position of the AR device, so as to obtain the position information to be matched.

For another example, if the real scene view includes a panoramic view (that is, a 360-degree image), the 360-degree image can be divided into 12 projection surfaces on average; Net VLAD) encoding method, image retrieval and scene recognition are performed on the 12 projection surfaces to obtain the location information to be matched and improve the accuracy of the location information to be matched.

Step S203, searching for a preset navigation map according to the location information to be matched, and determining whether there is location information to be matched in the preset navigation map.

Wherein, the preset navigation map includes a plurality of position information, and the position information to be matched is matched with each position information in the preset navigation map to determine whether the position information to be matched exists in the navigation map.

If it is determined that there is position information to be matched in the navigation map, then the direction of travel of the AR device and the relative position of the AR device are correct; otherwise, if it is determined that there is no position information to be matched in the navigation map, the direction of travel and the relative position of the AR device are represented. The relative position of the AR device is wrong. It is necessary to remind the AR device to adjust the direction of travel or adjust the relative position of the AR device as soon as possible to correct the direction of travel of the AR device and the relative position of the AR device so that the AR device can keep moving on the correct path. Get to the target location as quickly as possible.

Step S204, when it is determined that the location information to be matched exists in the preset navigation map, the location information matching the location information to be matched in the preset navigation map is used as real-time positioning information.

Wherein, it is correct that the real-time positioning information can represent the current traveling direction of the AR device and the relative position of the AR device.

By using the location information matching the location information to be matched in the preset navigation map as the real-time positioning information, the real-time positioning information can reflect the position of the AR device in the preset navigation map, which facilitates subsequent processing.

Step S205, determining virtual guidance information according to the real-time positioning information and the obtained guidance route.

Wherein, the guiding route includes a plurality of guiding position information. Each guide position information represents the direction that the AR device needs to travel and the relative position information of the AR device. Through each guide position information, the travel path of the AR device can be corrected in time, and the AR device can be guided to reach the target position as soon as possible.

For example, determining the virtual guidance information based on the real-time positioning information and the obtained guidance route includes: matching the real-time positioning information with a plurality of guidance position information, and determining the information on the direction to be moved and the distance to be moved corresponding to the AR device; The device corresponds to the information on the direction to be moved, the information on the distance to be moved, and a plurality of guide position information, and updates the guide route; and determines the virtual guide information according to the updated guide route.

Among them, the direction information to be moved can be matched with the direction that the AR device needs to travel in the guided position information, and the direction matching result can be obtained; the distance to be moved information can be matched with the relative position information of the AR device in the guided position information, and the position matching result can be obtained. Result: Through the direction matching result and the position matching result, it can be characterized whether the current traveling state of the AR device matches the guiding route, and when it is determined that there is an error between the current traveling state of the AR device and the guiding route, and the error exceeds the preset threshold , update the guidance route, and determine virtual guidance information according to the updated guidance route, the virtual guidance information is information corresponding to the updated guidance route, and calibrate the AR device in time to improve navigation accuracy.

For example, calibrating the AR device may include: adjusting related shooting parameters (eg, shooting angle, or image resolution, etc.) of the AR device, so as to improve navigation accuracy.

In some specific implementations, the virtual guide information includes: at least one of camera pose estimation information, environment perception information, and light source perception information.

Among them, the environment perception information is used to represent the position information of the AR device in the building to be navigated, the camera pose estimation information is used to represent the direction information corresponding to the AR device, and the light source perception information is used to represent the light source information obtained by the AR device.

Using different dimensions of information to describe the information of the AR device during the navigation process can improve the navigation accuracy of the AR device and avoid the influence of factors such as obstacles and observation angle changes.

Step S206, sending virtual guidance information to the AR device.

It should be noted that step S206 in this embodiment is the same as step S104 in the previous embodiment, and will not be repeated here.

In this embodiment, by processing the real scene view corresponding to the actual indoor position of the AR device based on the deep learning neural network, the location information to be matched is obtained, and the accuracy of the location information to be matched is improved; the preset location information is searched according to the location information to be matched Navigation map, determine whether there is position information to be matched in the preset navigation map, so as to judge whether the traveling direction of the AR device and the relative position of the AR device are correct; Set the location information in the navigation map that matches the location information to be matched as the real-time positioning information, so that the real-time positioning information can reflect the position of the AR device in the preset navigation map, which is convenient for subsequent processing; the real-time positioning information and the obtained guidance route Perform matching, determine virtual guidance information, and send virtual guidance information to the AR device, which can calibrate the travel route of the AR device in time and improve navigation accuracy.

The embodiment of the present application provides another possible implementation, wherein, in step S202, based on the deep learning neural network, the real scene view corresponding to the actual indoor position of the AR device is processed to obtain the position information to be matched, including:

Extract the local features in the real scene view corresponding to the actual position of the AR device in the room; input the local features into the deep learning neural network to obtain the global features corresponding to the actual position of the AR device in the room; based on the global features corresponding to the actual position of the AR device in the room Features, determine the location information to be matched.

Among them, the global feature is used to represent the position information of the AR device in the building to be navigated; the local feature is used to represent the AR device in the partial area view (for example, the image, photo and other view data uploaded by the AR device) relative position information. Through the global feature and the local feature, the positioning of the AR device can be used as an environmental reference object to make the positioning more accurate.

By analyzing the local features through the deep learning neural network (for example, using machine learning to learn and match multiple local positions in the building to be navigated), it can be clarified that the local features corresponding to the AR device are specifically related to the building to be navigated. Which local position of the AR device matches, and based on the actual position of the matched local position in the building to be navigated, determine the position information to be matched. The position information to be matched can reflect the actual position information of the AR device in the building to be navigated, improving the positioning accuracy .

The embodiment of the present application provides yet another possible implementation, wherein, before performing step S101 or step S201 to obtain real-time indoor actual location information of the augmented reality AR device, further include:

Obtain the panoramic data in the building to be navigated; perform point cloud mapping based on the panoramic data and preset algorithms to generate a dense map; determine the preset navigation map based on the dense map and the plane view corresponding to the building to be navigated.

Wherein, the panorama data may include: view data corresponding to all scenes in the entire building to be navigated, or panorama data corresponding to areas in the building to be navigated that need to be navigated.

For example, the panoramic data can also include multi-frame point cloud data collected by the panoramic camera. By rotating and transforming the two frames of point cloud data before and after, the coordinate systems corresponding to the two frames of point cloud data before and after are the same; and based on the above method, they are continuously superimposed Multi-frame point cloud data for point cloud mapping (for example, use the orthographic projection view corresponding to the multi-frame point cloud view to align with the plane view corresponding to the building to be navigated) to generate a dense map. The dense map can fully reflect the location and direction characteristics of the building to be navigated.

Match the plane view corresponding to the building to be navigated with the dense map to obtain a two-dimensional plane view, that is, the preset navigation map, so that the preset navigation map can inherit the location and direction features in the dense map to ensure the comprehensiveness of the map and completeness, to facilitate subsequent positioning and navigation.

In some specific implementations, point cloud mapping is performed based on panoramic data and preset algorithms to generate dense maps, including: processing panoramic data according to the principle of photogrammetry to generate point cloud data, which includes three-dimensional coordinate information and color information ; According to the preset 3D reconstruction algorithm, the point cloud data is processed to generate a dense map.

Among them, the principle of photogrammetry is to collect images through optical cameras, and process the collected images to obtain the shape, size, position, characteristics and their relationship of the object being photographed. For example, acquire multiple images of the subject, measure and analyze each image, obtain the analysis results, and output the analysis results in the form of diagrams or digital data.

Among them, the position of the subject can be represented by three-dimensional coordinate information, and the position information of the subject in the three-dimensional space can be accurately obtained by analyzing and matching the three-dimensional coordinate information in the point cloud data of multiple frames; The color of the subject is collected multiple times, and the color information of the subject is analyzed to accurately know the color characteristics of the subject (for example, based on the red green blue (Red Green Blue, RGB) color space) Color features, or, color features based on YUV color space, etc.)

Among them, "Y" in the YUV color space represents brightness (Luminance or Luma), that is, the grayscale value, and "U" and "V" represent chroma (Chrominance or Chroma), which are used to describe the color and saturation of the image.

Through the preset 3D reconstruction algorithm, the 3D coordinate information and color information in the point cloud data are processed, so that the generated dense map can reflect the position information and color information of each object in the 3D space, and improve the image in the map. The stereo image of each subject makes the dense map more accurate and convenient for navigation and positioning.

In some specific implementations, the preset navigation map is determined according to the dense map and the plane view corresponding to the building to be navigated, including: according to the preset scale factor, the dense map is mapped to the plane view to be processed by using an orthographic projection; The processing plane view is matched with the plane view corresponding to the building to be navigated to determine a preset navigation map.

Wherein, matching the plane view to be processed with the plane view corresponding to the building to be navigated can be achieved in the following manner: comparing the plane view to be processed with the plane view corresponding to the building to be navigated, or comparing the plane view to be processed with the plane view to be navigated The plan view corresponding to the navigation building is aligned to determine the default navigation map.

Wherein, the preset scale factor is a coefficient factor that can be calibrated, and the dense map can be reasonably zoomed through the preset scale factor, so that the zoomed dense map can adapt to display devices of different sizes.

It should be noted that since the dense map is a three-dimensional map, and the plane view corresponding to the building to be navigated is a two-dimensional view, it is necessary to use the orthographic projection method to map the dense map to the plane view to be processed, so as to facilitate the Match the plane view with the plane view corresponding to the building to be navigated, use the plane view corresponding to the building to be navigated to verify the accuracy of the plane view to be processed, and determine that the plane view to be processed matches the plane view corresponding to the building to be navigated In the case of , a preset navigation map can be obtained, which can reflect the accuracy of the dense map, and can also reflect the characteristics of the plane view corresponding to the building to be navigated, and use the preset navigation map to navigate the AR device and positioning, which can further improve the accuracy of navigation and positioning.

In some specific implementations, the plane view corresponding to the building to be navigated includes: a computer aided design (Computer Aided Design, CAD) view, where the CAD view is a vector plane view determined based on a preset scale factor.

Among them, since the preset scale factor can be calibrated, when the preset scale factor remains unchanged, displaying the CAD view on different devices (for example, mobile phones or tablet computers of different sizes, etc.) can ensure the clarity of the view The degree meets the expected requirements, that is, the CAD view has the characteristic of invariant scaling, and the CAD view is a directional vector plane view.

By designing the plane views corresponding to the plane views to be processed and the buildings to be navigated as CAD views, it can be ensured that the plane views to be processed and the plane views corresponding to the buildings to be navigated have the characteristics of invariant scaling and are directional The vector plane view improves the display effect of the terminal and brings users a better experience.

The embodiment of the present application provides another possible implementation, wherein, after performing step S104 or step S206 of sending virtual guide information to the AR device, further include:

Receive the arrival location information fed back by the AR device; compare the arrival location information with the target location information to determine whether the AR device has reached the target location; if it is determined that the AR device has reached the target location, end navigation.

Among them, by comparing the arrival location information fed back by the AR device with the target location information, it is determined whether the arrival location information and the target location information are the same, and if the two location information are determined to be the same, it is determined that the AR device has arrived at the target location, and the end For navigation, it is no longer necessary to obtain real-time location information fed back by AR devices, reducing the amount of information processing and improving information processing efficiency.

Otherwise, when it is determined that the AR device has not reached the target location, it is necessary to continue to obtain the real-time location information fed back by the AR device to assist the AR device in navigation and positioning, and to improve navigation accuracy.

It should be noted that in the process of navigating for the AR device, the navigation beacon can be represented by a cartoon image (for example, a character cartoon image and/or an animal cartoon image, etc.) to increase the fun of AR navigation.

Fig. 3 shows a schematic flowchart of an indoor navigation method provided by yet another embodiment of the present application. The indoor navigation method can be applied to an AR device, and the AR device can be installed on a terminal. As shown in FIG. 3 , the indoor navigation method in the embodiment of the present application includes at least but not limited to the following steps.

Step S301, sending the actual indoor location information of the augmented reality AR device to the server.

Wherein, the actual indoor location information of the AR device may include: three-dimensional space coordinate information corresponding to the location of the AR device. The actual position of the AR device in the building to be navigated can be reflected through the three-dimensional space coordinate information.

When the server obtains the actual location information, it matches the actual location information with the preset navigation map, determines real-time positioning information, and generates and sends virtual guidance information to the AR device according to the guidance route and real-time positioning information, wherein the guidance route is Based on the acquired initial location information of the AR device and the route determined by the target location information, the actual location information is used to represent the location information of the AR device in the world coordinate system.

Through the guidance route, the path information that needs to be navigated is clarified, and the guidance route is matched with the real-time positioning information, the mapping position information of the AR device in the preset navigation map can be clarified, the guidance route can be dynamically adjusted in real time, and the navigation accuracy can be improved.

Step S302, generating a virtual navigation image in response to the virtual guidance information sent by the server.

Wherein, the virtual navigation image may include: an AR image or AR video based on dynamic imaging. Through dynamic AR images or AR videos, the position information of the AR device in the building to be navigated and the target route information can be clearly and three-dimensionally viewed.

Among them, the virtual guidance information includes camera pose estimation information, environment perception information and light source perception information. The environment perception information is used to represent the position information of the AR device in the building to be navigated, and the camera pose estimation information is used to represent the direction information corresponding to the AR device. , the light source perception information is used to represent the light source information acquired by the AR device.

For example, the camera pose estimation information may include: orientation information corresponding to the AR device, for example, the relative position of the AR device (for example, the camera of the AR device faces the direction opposite to the user's face, or the camera of the AR device faces the ground, etc.). The light source perception information may include: light information of multiple angles received by the AR device.

In some specific implementations, generating a virtual navigation image in response to the virtual guidance information sent by the server includes: receiving the virtual guidance information sent by the server; processing environment perception information and light source perception information according to a preset three-dimensional reconstruction algorithm to obtain enhanced The virtual image of real AR; match the camera pose estimation information with the virtual image of AR to determine the virtual navigation image.

Wherein, the preset three-dimensional reconstruction algorithm may include: a multi-view geometry (Open Multiple ViewGeometry, OpenMVG) algorithm and a multi-view stereo reconstruction library (Open Multi-View Stereo reconstruction library, OpenMVS) algorithm.

The OpenMVG algorithm can accurately solve common problems in multi-view geometry. For example, calibration based on scene structure information; self-calibration based on camera active information (eg, pure rotation information); self-calibration independent of scene structure and camera active information, etc. The OpenMVS algorithm is suitable for scenarios such as dense point cloud reconstruction, surface reconstruction, surface refinement and texture mapping, and surface refinement can make images clearer. By optimizing the OpenMVG algorithm and the OpenMVS algorithm, the optimized algorithms obtained can be used together to achieve 3D reconstruction.

Use the OpenMVS algorithm to perform surface reconstruction and surface refinement on the environmental perception information, and perform texture mapping and other processing on the light source perception information to obtain a virtual image of AR, which can reflect the environmental information of the AR device in more detail and the acquired information The projection information of the light source on the AR device, etc. The OpenMVG algorithm is used to process the camera pose estimation information to obtain multiple view information based on the AR device, and match the multiple view information with the AR virtual image to determine the virtual navigation image and improve the accuracy of the virtual navigation image.

Step S303, dynamically displaying the virtual navigation image.

For example, the obtained virtual navigation image can be displayed in real time, and the obtained virtual navigation image can also be played dynamically in the form of frames, which is convenient for users to view, three-dimensionally and intuitively view navigation information, and improve navigation accuracy.

In this embodiment, by sending the actual indoor location information of the AR device to the server, the server can match the actual location information with the preset navigation map, determine real-time positioning information, and generate And send virtual guidance information to the AR device, through the guidance route to clarify the path information that needs to be navigated, and match the guidance route with real-time positioning information, the mapping position information of the AR device in the preset navigation map can be clarified, real-time dynamic Adjust the guidance route to improve navigation accuracy; in response to the virtual guidance information sent by the server, generate a dynamic display of virtual navigation images, which is convenient for users to view navigation information three-dimensionally and intuitively, and improve navigation accuracy.

Each device according to the embodiment of the present application will be described in detail below with reference to the accompanying drawings. FIG. 4 shows a block diagram of a server provided by an embodiment of the present application. As shown in Figure 4. The server 400 includes the following modules.

The acquiring module 401 is configured to acquire the actual position information of the augmented reality AR device indoors in real time, and the actual position information is used to represent the position information of the AR device in the world coordinate system; the matching module 402 is configured to combine the actual position information with the preset Assuming that the navigation map matches, determine the real-time positioning information; the determination module 403 is configured to determine the virtual guidance information based on the real-time positioning information and the obtained guidance route, and the guidance route is based on the obtained initial position information and target position of the AR device The route determined by the information; the first sending module 404 is configured to send virtual guidance information to the AR device, so that the AR device generates and dynamically displays a virtual navigation image according to the virtual guidance information.

In some specific implementations, the actual location information includes: the real scene view corresponding to the actual indoor position of the AR device; the matching module 402 is specifically configured to: process the real scene view corresponding to the actual indoor position of the AR device based on a deep learning neural network , to obtain the location information to be matched; search the preset navigation map according to the location information to be matched, and determine whether there is the location information to be matched in the preset navigation map; The location information in the navigation map that matches the location information to be matched is used as real-time positioning information.

In some specific implementations, based on the deep learning neural network, the real scene view corresponding to the actual indoor position of the AR device is processed to obtain the location information to be matched, including: extracting local features in the real scene view corresponding to the actual indoor position of the AR device ; Input the local features into the deep learning neural network to obtain the global features corresponding to the actual indoor position of the AR device. The global features are used to represent the position information of the AR device in the building to be navigated; based on the actual indoor position of the AR device, the corresponding The global feature determines the location information to be matched.

In some specific implementations, the server 400 also includes: a preset navigation map generation module, which is used to: obtain panoramic data in the building to be navigated; perform point cloud mapping based on the panoramic data and a preset algorithm to generate a dense map; The dense map and the plane view corresponding to the building to be navigated determine the preset navigation map.

In some specific implementations, point cloud mapping is performed based on panoramic data and preset algorithms to generate dense maps, including: processing panoramic data according to the principle of photogrammetry to generate point cloud data, which includes three-dimensional coordinate information and color information ; According to the preset 3D reconstruction algorithm, the point cloud data is processed to generate a dense map.

In some specific implementations, the preset navigation map is determined according to the dense map and the plane view corresponding to the building to be navigated, including: according to the preset scale factor, the dense map is mapped to the plane view to be processed by using an orthographic projection; The processing plane view is matched with the plane view corresponding to the building to be navigated to determine a preset navigation map.

In some specific implementations, the plane view corresponding to the building to be navigated includes: a computer-aided design CAD view, and the CAD view is determined to be a vector plane view based on a preset scale factor.

In some specific implementations, the guidance route includes a plurality of guidance location information; the determination module 403 is specifically configured to: match the real-time positioning information with the plurality of guidance location information, and determine the direction information to be moved and the distance information to be moved corresponding to the AR device ; Update the guidance route according to the direction information to be moved, the distance information to be moved and a plurality of guidance position information corresponding to the AR device; determine the virtual guidance information according to the updated guidance route.

In some specific implementations, the server 400 further includes: a confirmation module, configured to: receive the arrival location information fed back by the AR device; compare the arrival location information with the target location information to determine whether the AR device has reached the target location; In case of location, end navigation.

In some specific implementations, the virtual guidance information includes: at least one of camera pose estimation information, environment perception information, and light source perception information; wherein, the environment perception information is used to represent the position information of the AR device in the building to be navigated, and the camera The attitude estimation information is used to represent the direction information corresponding to the AR device, and the light source perception information is used to represent the light source information acquired by the AR device.

In this embodiment, the actual position information of the augmented reality AR device in the room can be obtained in real time by the acquisition module, and the actual position of the AR device in the world coordinate system can be determined; the actual position information of the AR device in the world coordinate system can be determined by using the matching module Matching with the preset navigation map, the actual location information can be mapped to the preset navigation map on the two-dimensional space, so as to determine the real-time positioning information corresponding to the AR device in the preset navigation map to facilitate subsequent processing; using the determination module Determine the virtual guidance information based on the real-time positioning information and the obtained guidance route, wherein the guidance route is a route determined based on the acquired initial position information and target position information of the AR device, match the real-time positioning information with the guidance route, and determine It is necessary to provide virtual guidance information to the AR device to improve the accuracy of navigation; use the first sending module to send the virtual guidance information to the AR device, so that the AR device can generate and dynamically display high-precision virtual navigation images based on the virtual guidance information, so as to Intuitively guide users to quickly go to the target location, improving navigation accuracy.

FIG. 5 shows a block diagram of an augmented reality AR device provided by an embodiment of the present application. As shown in Figure 5. The augmented reality device 500 includes the following modules.

The second sending module 501 is configured to send the actual location information of the augmented reality AR device indoors to the server, so that the server matches the actual location information with the preset navigation map, determines real-time positioning information, and uses the guidance route and real-time positioning information, generating and sending virtual guide information to the AR device, wherein the guide route is a route determined based on the acquired initial position information and target position information of the AR device; the generation module 502 is configured to respond to the virtual guide information sent by the server , to generate a virtual navigation image; the presentation module 503 is configured to dynamically display the virtual navigation image.

In this embodiment, the actual location information of the AR device indoors is sent to the server through the second sending module, so that the server can match the actual location information with the preset navigation map, determine real-time positioning information, and based on the guidance route and real-time Positioning information, generate and send virtual guidance information to the AR device, through the guidance route to clarify the path information that needs to be navigated, and match the guidance route with real-time positioning information, the mapping position of the AR device in the preset navigation map can be clarified Information, real-time dynamic adjustment of guidance routes to improve navigation accuracy; use the generation module to respond to the virtual guidance information sent by the server to generate virtual navigation images, and use the display module to dynamically display virtual navigation images, which is convenient for users to view navigation information in a three-dimensional and intuitive way, and improve Navigation accuracy.

FIG. 6 shows a block diagram of a terminal provided by an embodiment of the present application. As shown in Figure 6. The terminal 600 includes: at least one augmented reality device 500, and the augmented reality device 500 is configured to implement any indoor navigation method in the embodiments of the present application.

For example, the augmented reality device 500 includes: a second sending module 501 configured to send the actual location information of the augmented reality device 500 indoors to the server, so that the server matches the actual location information with the preset navigation map to determine the real-time location Information, and according to the guide route and real-time positioning information, generate and send virtual guide information to the augmented reality device 500, wherein the guide route is a route determined based on the acquired initial position information and target position information of the augmented reality device 500; the generation module 502, configured to generate a virtual navigation image in response to the virtual guidance information sent by the server; a display module 503, configured to dynamically display the virtual navigation image.

In this embodiment, the actual location information of the augmented reality device 500 indoors is sent to the server through the second sending module 501, so that the server can match the actual location information with the preset navigation map, determine real-time positioning information, and follow the guide route and real-time positioning information, generate and send virtual guidance information to the augmented reality device 500, through the guidance route to clarify the route information that needs to be navigated, and match the guidance route with the real-time positioning information, it can be determined that the augmented reality device 500 will Set the mapping position information in the navigation map, dynamically adjust the guiding route in real time, and improve the accuracy of navigation; use the generation module 502 to respond to the virtual guidance information sent by the server, generate a virtual navigation image, and use the display module to dynamically display the virtual navigation image, which is convenient for users View navigation information three-dimensionally and intuitively to improve navigation accuracy.

It should be clear that the present application is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the convenience and brevity of description, detailed descriptions of known methods are omitted here, and the specific working processes of the above-described systems, modules, and units can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.

Fig. 7 shows a block diagram of the composition of the indoor navigation system provided by the embodiment of the present application. As shown in Figure 7. The indoor navigation system includes the following devices.

An offline map creation device 710 , a cloud navigation server 720 , a terminal 730 and a preset navigation map generation device 740 .

Wherein, the offline map creation device 710 includes: a panorama image acquisition device 711 and a point cloud map creation device 712 . The cloud navigation server 720 includes: an identification and positioning module 721 , a path planning module 722 and a real-time navigation module 723 . The terminal 730 includes: an initial positioning module 731 , a destination selection module 732 , a real-time navigation image feedback module 733 , a virtual navigation image generation module 734 and a display module 735 .

Wherein, the terminal 730 may be a mobile phone terminal supporting AR functions (for example, supporting at least one of functions such as motion capture, environment perception, and light source perception), and the panoramic image acquisition device 711 may include: a panoramic camera device and/or a panoramic camera device .

Fig. 8 shows a schematic flowchart of a navigation method of an indoor navigation system provided by an embodiment of the present application. As shown in FIG. 8 , the navigation method of the indoor navigation system at least includes but not limited to the following steps.

In step S801 , the terminal 730 sends a map download request to the default navigation map generation device 740 .

Wherein, the download request is used to request to obtain a preset navigation map, and the preset navigation map is a map determined based on image data pre-collected by the panoramic image acquisition device 711 in the offline map creation device 710 .

For example, the download request may include information such as the identification of the terminal 730 and the number of the preset navigation map, where the number of the preset navigation map may be the number obtained from the historical interaction information of the terminal 730, or a real-time map uploaded by the terminal. The number identified by the image information.

For example, the panoramic image collection device 711 can send the collected image data based on the indoor environment (for example, the indoor environment of a shopping mall, etc.) to the point cloud map creation device 712, so that the point cloud map creation device 712 can follow the preset The 3D reconstruction algorithm processes the collected image data based on the indoor environment to obtain a preset navigation map.

For example, according to the image data collected by the panoramic image acquisition device 711, the point cloud map creation device 712 generates a dense map by calling an optimized three-dimensional reconstruction algorithm; view; determine the default navigation map.

Wherein, the optimized 3D reconstruction algorithm may include OpenMVG algorithm and OpenMVS algorithm. The OpenMVG algorithm can accurately solve common problems in multi-view geometry. For example, calibration based on scene structure information; self-calibration based on camera active information (eg, pure rotation information); self-calibration independent of scene structure and camera active information, etc.

The OpenMVS algorithm is suitable for scenarios such as dense point cloud reconstruction, surface reconstruction, surface refinement and texture mapping, and surface refinement can make images clearer. By optimizing the OpenMVG algorithm and the OpenMVS algorithm, the optimized algorithm obtained can be used in conjunction to realize the three-dimensional reconstruction of the image.

It should be noted that the point cloud map creation device 712 may perform dimensionality reduction processing on the three-dimensional map in a manner of orthographic projection to obtain a two-dimensional planar map. For example, map a 3D map onto a 2D planar map in an orthographic projection. The three-dimensional map and the two-dimensional planar map keep the horizontal and vertical coordinates (for example, x coordinates and y coordinates, etc.) unchanged, and the two-dimensional planar map can be aligned with the preset CAD view to maintain the consistency of the coordinates, In this way, the generated preset navigation map can have the characteristic of invariant zoom scale, and is a directional vector plane view.

In some specific implementations, the preset navigation map can be accurately displayed on display screens of different sizes (for example, display screens of mobile phones or display screens of tablet computers of different sizes, etc.), so as to improve user experience.

For example, it is also possible to use the two-dimensional planar CAD view corresponding to a certain floor in a shopping mall to reversely improve the accuracy of the dense map constructed from point cloud data, and at the same time remove the noise interference information in the dense map to make the dense map more accurate .

In step S802, the default navigation map generation device 740 sends the preset navigation map corresponding to the current scene of the terminal 730 to the terminal 730, and the map initialization of the terminal 730 has been completed.

Step S803 , the terminal 730 uploads the real scene image of its current location to the cloud navigation server 720 .

In step S804, the cloud navigation server 720 processes the real-scene image uploaded by the terminal 730 through the identification and positioning module 721, and matches the real-scene image with a preset navigation map to determine real-time positioning information.

For example, the identification and positioning module 721 invokes the deep learning hierarchical semantic description algorithm to classify the real-scene images uploaded by the terminal 730, and preliminarily clarify the major categories described in the real-scene images (for example, house images or person images, etc.); Further refinement analysis is performed to obtain the initial position of the terminal 730 .

Further, the real scene image uploaded by the terminal 730 may include: a partial area view or a panoramic image. Among them, the panoramic image can be a 360-degree image taken by a panoramic camera, and then the 360-degree image is divided into 12 projection surfaces on average, and then the encoding method based on Net VLAD is used to perform image retrieval and retrieval on the 12 projection surfaces respectively. The scene recognition is to obtain the initial position of the terminal 730, for example, the three-dimensional space coordinate information corresponding to the position of the terminal 730.

In some specific implementations, the real-time positioning information may include: the location of the terminal 730 corresponds to the two-dimensional coordinate information in the preset navigation map, and the two-dimensional coordinate information is obtained by mapping the three-dimensional space coordinate information corresponding to the location of the terminal 730 to the coordinate information obtained in the preset navigation map.

Step S805 , the cloud navigation server 720 sends the real-time positioning information to the terminal 730 , so that the terminal 730 uses the display module 735 to display the real-time positioning information.

Wherein, the real-time positioning information can represent the real-time position of the terminal 730 in the preset navigation map.

Step S806 , the destination selection module 732 in the terminal 730 obtains the target location information input by the user, and generates and sends a route navigation message to the cloud navigation server 720 based on the initial location information of the terminal 730 and the target location information.

For example, the target location information may be determined by the user operating the map displayed on the mobile phone terminal and directly clicking on a specific location, so that the user can easily operate when selecting the target location and ensure the ease of use of address selection.

Step S807, the cloud navigation server 720 performs message analysis on the received route navigation message, obtains the initial location information and the target location information of the terminal 730, and then uses the path planning module 722 to call the Rapidly-exploring Random Tree (RRT ) algorithm to process the initial position information and the target position information to obtain the guidance route.

It should be noted that the RRT algorithm is a tree-shaped data storage structure and algorithm. The data storage result is established through the method of path increment, and the distance between the randomly selected point and the tree is quickly reduced. The RRT algorithm can effectively search for non-convex The (Non Convex) high-dimensional space is especially suitable for path planning including obstacles and non-holonomic (Non-Holonomic) systems or reverse dynamics (Kino-Dynamic) differential constraints.

Step S808 , the cloud navigation server 720 sends the guiding route to the terminal 730 , so that the terminal 730 uses the display module 735 to display the guiding route.

Step S809 , the terminal 730 uses the real-time navigation image feedback module 733 to upload the location information and scene images acquired in real time to the real-time navigation module 723 in the cloud navigation server 720 .

In step S810, the real-time navigation module 723 determines virtual navigation information by performing motion capture, environment perception, and light source perception on the location information and scene images acquired in real time.

Wherein, the virtual navigation information may include: at least one of camera pose estimation information, environment perception information, and light source perception information; the environment perception information is used to characterize the position information of the terminal 730 in the building to be navigated, and the camera posture estimation information is used to The direction information corresponding to the terminal 730 is represented, and the light source perception information is used to represent the light source information acquired by the terminal 730 . It can comprehensively measure the information of the terminal 730 in real-time navigation, and improve the navigation accuracy.

Step S811, the cloud navigation server 720 sends the updated virtual navigation information to the terminal 730, so that the terminal 730 uses the virtual navigation image generation module 734 to generate an updated virtual navigation image, and uses the display module 735 to dynamically display the updated virtual navigation in real time image.

It should be noted that steps S809 to S811 may be performed repeatedly during the navigation process, so as to adjust the real-time virtual navigation image.

For example, during the navigation process, the terminal 730 will upload the image information corresponding to its location to the cloud navigation server 720 in real time, so that the cloud navigation server 720 can match the image information corresponding to the location of the terminal 730 with the guidance route, and real-time Dynamically adjust the guidance route to ensure the consistency and accuracy of the guidance route.

Step S812, after the terminal 730 arrives at the target location (i.e., the navigation destination), it will continue to upload the destination image corresponding to the navigation destination to the cloud navigation server 720, so that the cloud navigation server 720 can combine the previous guidance route to determine whether the navigation terminal is compatible with the preset The target location information matches the target location information, and if the match is determined, the navigation process ends.

In some specific implementations, the navigation beacons in the AR navigation process can be represented by cartoon characters to increase the fun of AR navigation.

In this embodiment, by using the panoramic camera device in the portable terminal to collect visual data of the scene in the building to be navigated, and perform three-dimensional reconstruction based on the collected visual data, a corresponding image corresponding to the physical space in the building to be navigated is generated. The dense map, and the visual data in the building to be navigated are segmented and mapped to generate a point cloud map, and then the point cloud map is aligned with the preset CAD view, so that the generated preset navigation map can have different zoom scales. Variable characteristics, and is a directional vector plane view. During the navigation process, the terminal uploads the image information corresponding to its location to the cloud navigation server in real time, so that the cloud navigation server can adjust the pre-planned guidance route in real time, generate virtual guidance information, and send the virtual guidance information to the terminal. The terminal can generate an updated virtual navigation image based on the virtual guidance information, and dynamically display the updated virtual navigation image through AR, so that users can view the navigation information dynamically and three-dimensionally, which is convenient for users to locate and navigate, and improves navigation accuracy .

Fig. 9 shows a structural diagram of an exemplary hardware architecture of a computing device capable of implementing the indoor navigation method and apparatus according to the embodiments of the present application.

As shown in FIG. 9 , the computing device 900 includes an input device 901 , an input interface 902 , a central processing unit 903 , a memory 904 , an output interface 905 , and an output device 906 . Wherein, the input interface 902, the central processing unit 903, the memory 904, and the output interface 905 are connected to each other through the bus 907, and the input device 901 and the output device 906 are respectively connected to the bus 907 through the input interface 902 and the output interface 905, and then connected to the computing device 900 other component connections.

Specifically, the input device 901 receives input information from the outside, and transmits the input information to the central processing unit 903 through the input interface 902; the central processing unit 903 processes the input information based on computer-executable instructions stored in the memory 904 to generate output information, temporarily or permanently store the output information in the memory 904, and then transmit the output information to the output device 906 through the output interface 905; the output device 906 outputs the output information to the outside of the computing device 900 for the user to use.

In one embodiment, the computing device shown in FIG. 9 may be implemented as an electronic device, and the electronic device may include: a memory configured to store a program; a processor configured to run the program stored in the memory to The indoor navigation method described in the above embodiments is executed.

In one embodiment, the computing device shown in FIG. 9 can be implemented as an indoor navigation system, and the indoor navigation system can include: a memory configured to store a program; a processor configured to run the program stored in the memory , so as to execute the indoor navigation method described in the above embodiments.

The above descriptions are merely exemplary embodiments of the present application, and are not intended to limit the protection scope of the present application. In general, the various embodiments of the present application can be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be executed by a controller, microprocessor or other computing device, although the application is not limited thereto.

Embodiments of the present application may be realized by a data processor of a mobile device executing computer program instructions, for example in a processor entity, or by hardware, or by a combination of software and hardware. Computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code written in any combination of one or more programming languages or object code.

Any logic flow block diagrams in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions. Computer programs can be stored on memory. The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, read-only memory (ROM), random-access memory (RAM), optical memory devices and systems (digital versatile disc DVD or CD), etc. Computer readable media may include non-transitory storage media. The data processor can be of any type suitable for the local technical environment, such as but not limited to general purpose computer, special purpose computer, microprocessor, digital signal processor (DSP), application specific integrated circuit (ASIC), programmable logic device (FGPA) and processors based on multi-core processor architectures.

The foregoing has provided a detailed description of exemplary embodiments of the present application by way of exemplary and non-limiting examples. However, considering the accompanying drawings and the claims, various modifications and adjustments to the above embodiments are obvious to those skilled in the art, but do not depart from the scope of the present application. Therefore, the proper scope of the application will be determined from the claims.

Claims (17)

1. A method for indoor navigation, characterized in that the method comprises: Acquire real-time indoor actual location information of the augmented reality AR device, where the actual location information is used to characterize the location information of the AR device in the world coordinate system; Matching the actual position information with the preset navigation map to determine real-time positioning information; determining virtual guidance information based on the real-time positioning information and the acquired guidance route, the guidance route being a route determined based on the acquired initial position information and target position information of the AR device; Sending the virtual guide information to the AR device, so that the AR device generates and dynamically displays a virtual navigation image according to the virtual guide information. 2. The method according to claim 1, wherein the actual location information includes: a real scene view corresponding to the actual indoor location of the AR device; The matching of the actual position information with the preset navigation map to determine the real-time positioning information includes: Based on the deep learning neural network, the real scene view corresponding to the actual position of the AR device in the room is processed to obtain the position information to be matched; Searching the preset navigation map according to the location information to be matched, and determining whether the location information to be matched exists in the preset navigation map; If the location information to be matched exists in the preset navigation map, the location information matching the location information to be matched in the preset navigation map is used as the real-time positioning information. 3. The method according to claim 2, wherein the processing of the real scene view corresponding to the actual indoor position of the AR device based on the deep learning neural network to obtain the position information to be matched includes: Extracting local features in the real scene view corresponding to the actual position of the AR device in the room; Inputting the local feature into the deep learning neural network to obtain a global feature corresponding to the actual indoor position of the AR device, the global feature is used to represent the position information of the AR device in the building to be navigated; The location information to be matched is determined based on the global features corresponding to the actual indoor location of the AR device. 4. The method according to claim 1, wherein before the real-time acquisition of the actual indoor location information of the augmented reality AR device, further comprising: Obtain the panoramic data in the building to be navigated; Carry out point cloud mapping based on the panoramic data and a preset algorithm to generate a dense map; The preset navigation map is determined according to the dense map and the plane view corresponding to the building to be navigated. 5. The method according to claim 4, wherein said point cloud mapping based on said panorama data and a preset algorithm to generate a dense map includes: Processing the panoramic data according to the principle of photogrammetry to generate point cloud data, the point cloud data including three-dimensional coordinate information and color information; The point cloud data is processed according to a preset three-dimensional reconstruction algorithm to generate the dense map. 6. The method according to claim 4, wherein the determining the preset navigation map according to the dense map and the plane view corresponding to the building to be navigated comprises: Mapping the dense map into a plane view to be processed by means of orthographic projection according to a preset scale factor; The plane view to be processed is matched with the plane view corresponding to the building to be navigated to determine the preset navigation map. 7. The method according to claim 6, wherein the plane view corresponding to the building to be navigated comprises: a computer-aided design CAD view, the CAD view being a vector plane view determined based on the preset scale factor . 8. The method according to claim 1, wherein the guiding route comprises a plurality of guiding position information; The determining virtual guidance information based on the real-time positioning information and the obtained guidance route includes: Matching the real-time positioning information with a plurality of the guiding position information, and determining the direction information to be moved and the distance information to be moved corresponding to the AR device; updating the guidance route according to the direction information to be moved, the distance information to be moved, and a plurality of pieces of guidance position information corresponding to the AR device; The virtual guidance information is determined according to the updated guidance route. 9. The method according to claim 1, wherein after sending the virtual guide information to the AR device so that the AR device generates and dynamically displays a virtual navigation image according to the virtual guide information, Also includes: receiving arrival location information fed back by the AR device; comparing the arrival location information with the target location information to determine whether the AR device has reached the target location; In case it is determined that the AR device has reached the target position, the navigation is ended. 10. The method according to any one of claims 1 to 9, wherein the virtual guide information includes: at least one of camera pose estimation information, environment perception information, and light source perception information; Wherein, the environment perception information is used to represent the position information of the AR device in the building to be navigated, the camera pose estimation information is used to represent the direction information corresponding to the AR device, and the light source perception information is used to represent Light source information acquired by the AR device. 11. A method for indoor navigation, characterized in that the method comprises: Sending the actual location information of the augmented reality AR device indoors to the server, so that the server matches the actual location information with the preset navigation map, determines real-time positioning information, and generates and sending virtual guidance information to the AR device, wherein the guidance route is a route determined based on the acquired initial position information and target position information of the AR device, and the actual position information is used to characterize the AR device Position information in the world coordinate system; generating a virtual navigation image in response to the virtual guidance information sent by the server; The virtual navigation image is dynamically displayed. 12. The method according to claim 11, wherein the generating a virtual navigation image in response to the virtual guidance information sent by the server comprises: receiving virtual guidance information sent by the server, the virtual guidance information including camera pose estimation information, environment perception information and light source perception information, the environment perception information being used to characterize the position information of the AR device in the building to be navigated , the camera pose estimation information is used to represent direction information corresponding to the AR device, and the light source perception information is used to represent light source information acquired by the AR device; Process the environmental perception information and light source perception information according to a preset three-dimensional reconstruction algorithm to obtain a virtual image of augmented reality AR; Matching the camera pose estimation information with the AR virtual image to determine the virtual navigation image. 13. A server comprising: An acquisition module configured to acquire real-time indoor actual location information of the augmented reality AR device, the actual location information being used to characterize the location information of the AR device in the world coordinate system; A matching module configured to match the actual location information with a preset navigation map to determine real-time positioning information; The determination module is configured to determine virtual guidance information according to the real-time positioning information and the obtained guidance route, the guidance route is a route determined based on the acquired initial position information and target position information of the AR device; The first sending module is configured to send the virtual guide information to the AR device, so that the AR device generates and dynamically displays a virtual navigation image according to the virtual guide information. 14. An augmented reality AR device comprising: The second sending module is configured to send the actual location information of the augmented reality AR device indoors to the server, so that the server will match the actual location information with the preset navigation map, determine real-time positioning information, and follow the guidance route and the real-time positioning information, generating and sending virtual guidance information to the AR device, wherein the guidance route is a route determined based on the acquired initial position information and target position information of the AR device; A generation module configured to generate a virtual navigation image in response to the virtual guidance information sent by the server; The display module is configured to dynamically display the virtual navigation image. 15. A terminal comprising: At least one augmented reality AR device as claimed in claim 14. 16. An electronic device comprising: one or more processors; A memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement any one of claims 1-10 item, or, the indoor navigation method according to any one of claims 11-12. 17. A readable storage medium, characterized in that the readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of claims 1-10 is implemented, or, as claimed in claim The indoor navigation method described in any one of 11-12.

Images