WO2019127320A1 - Information processing method and apparatus, cloud processing device, and computer program product - Google Patents

Abstract

An information processing method and apparatus, a cloud processing device, and a computer program product, relating to the technical field of data processing, and being capable of improving the accuracy and speed of recognising a static object to a certain degree. The information processing method provided in the embodiments of the present application comprises: acquiring image information photographed by a terminal (101); acquiring position information of the terminal (102); on the basis of the position information and the image information, retrieving an AR model (103); and sending the AR model to the terminal (104).

Description

Information processing method, device, cloud processing device and computer program product Technical field

The present application relates to the field of data processing technologies, and in particular, to an information processing method, apparatus, cloud processing device, and computer program product.

Background technique

AR (Augmented Reality) technology, also known as augmented reality, passes through the physical information (visual information, sound, taste, touch, etc.) that was difficult to experience in a certain time and space of the real world. After the simulation, it is superimposed on the real world and is perceived by human senses to achieve a sensory experience that transcends reality.

Moreover, through the AR technology, it is also possible to enhance the interaction between static things and users, which can present different overlapping scenes on static things. However, in the prior art, since the image taken by the user may be only a part of the static thing, part of the content will judge the whole of the static object, resulting in a large amount of retrieval, and thus the accuracy of identifying the static object is low.

Summary of the invention

The embodiment of the present application provides an information processing method, device, cloud processing device, and computer program product, which improves the recognition accuracy and speed of static objects, and the superimposed AR model is more accurate.

In a first aspect, an embodiment of the present application provides an information processing method, including:

Obtaining image information captured by the terminal;

Obtaining location information of the terminal;

Retrieving an AR model according to the location information and the image information;

The AR model is sent to the terminal.

The foregoing aspect and any possible implementation manner further provide an implementation manner for the AR model to be retrieved according to the location information and the image information, including:

Determining a search range according to the location information;

Retrieving all building information within the search range;

Obtaining image information corresponding to each building in the building information;

Determining a building included in the image information according to the image information;

The AR model corresponding to the building is retrieved.

The aspect as described above, and any possible implementation manner, further provide an implementation manner, the acquiring image information corresponding to each building in the building information, including:

Obtaining 3D model information corresponding to each building in the building information; or

If the 3D model information corresponding to the building information is not acquired, the 2D picture information corresponding to each building in the building information is acquired.

The aspect as described above and any possible implementation manner further provide an implementation manner, where the method further includes:

Establish at least one relational database based on the building information, the image information, and the AR model.

In a second aspect, the embodiment of the present application further provides an information processing apparatus, including:

a first acquiring unit, configured to acquire image information captured by the terminal;

a second acquiring unit, configured to acquire location information of the terminal;

a retrieval unit, configured to retrieve an AR model according to the location information and the image information;

And a sending unit, configured to send the AR model to the terminal.

The above-mentioned aspect and any possible implementation manner further provide an implementation manner, where the retrieval unit is specifically configured to:

Determining a search range according to the location information;

Retrieving building information within the search range;

Obtaining image information corresponding to each building in the building information;

Determining a building included in the image information according to the image information;

The AR model corresponding to the building is retrieved.

The above-mentioned aspect and any possible implementation manner further provide an implementation manner, where the retrieval unit is specifically configured to:

Obtaining 3D model information corresponding to each building in the building information; or

If the 3D model information corresponding to the building information is not acquired, the 2D picture information corresponding to each building in the building information is acquired.

The above-mentioned aspect and any possible implementation manner further provide an implementation manner, the device further includes:

The establishing unit is configured to establish at least one relational database according to the building information, the image information and the AR model.

In a third aspect, the embodiment of the present application further provides a cloud processing device, where the device includes a processor and a memory; the memory is configured to store an instruction, when the instruction is executed by the processor, causing the device to perform, for example, The method of any of the first aspects.

In a fourth aspect, the embodiment of the present application further provides a computer program product, which can be directly loaded into an internal memory of a computer and includes software code. After the computer program is loaded and executed by a computer, the first aspect can be implemented. One such method.

The information processing method, the device, the cloud processing device, and the computer program product provided by the embodiment of the present application acquire the corresponding AR model by acquiring the image information and the location information captured by the terminal, and then combining the location information and the image information, wherein, according to the location information, Retrieving the AR model can narrow the search range and improve the retrieval accuracy. Combined with the image information captured by the terminal to determine the corresponding AR model, the retrieval efficiency can be improved, the recognition rate of static objects can be improved, and the identification of static object retrieval in the prior art can be solved. Large quantity and low accuracy.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative labor for those skilled in the art.

FIG. 1 is a flowchart of an embodiment of an image processing method according to an embodiment of the present application;

FIG. 2 is another flowchart of an embodiment of an image processing method according to an embodiment of the present disclosure;

FIG. 3 is a schematic structural diagram of an embodiment of an information processing apparatus according to an embodiment of the present disclosure;

FIG. 4 is another schematic structural diagram of an embodiment of an information processing apparatus according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an embodiment of a cloud processing device according to an embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The terms used in the embodiments of the present application are for the purpose of describing particular embodiments only, and are not intended to limit the application. The singular forms "a", "the", and "the"

It should be understood that the term "and/or" as used herein is merely an association describing the associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, while A and B, there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting." Similarly, depending on the context, the phrase "if determined" or "if detected (conditions or events stated)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event) "Time" or "in response to a test (condition or event stated)".

When a user uses a terminal or a wearable device, the camera often uses a camera to capture video or images. When the user is in an outdoor environment, static objects such as buildings often appear in the camera, or the user actively performs static things such as buildings. Shooting. Since the surface of the outdoor building can be attached with various information, for example, service information, advertisement information, publicity information, etc., all of them have the disadvantages of time-consuming and labor-intensive replacement of the content, and high cost, and therefore, by means of the user-receiving terminal mentioned above or Wearable devices can be used to integrate AR technology into outdoor buildings, enhancing customization, low cost, and strong immersion.

However, since outdoor buildings are subject to illumination changes and image images are collected, and when the building is bulky, the captured images are only a part of the building, which reduces the accuracy of building identification to a certain extent. In order to be able to solve the above-mentioned situation, the embodiment of the present application provides an information processing method. Specifically, FIG. 1 is a flowchart of an embodiment of an image processing method according to an embodiment of the present disclosure. The information processing method may specifically include the following steps:

101. Obtain image information captured by the terminal.

In the embodiment of the present application, the terminal may include: a smart device with a camera function, such as a mobile phone, a tablet computer, a notebook computer, an in-vehicle device, and a wearable device. Moreover, in the embodiment of the present application, the terminal also needs to have a network communication function.

The image information captured by the terminal is image information obtained by the user controlling a camera or the like having a function of acquiring images to capture an outdoor environment. The image information may be obtained by a computing device having strong computing power, and the computing device includes at least an arithmetic unit, a wireless transmission unit, for example, a local computer, a cloud processing center, or the like. The terminal and the computing device can communicate with each other, and the communication mode can use wireless communication methods such as 2G, 3G, 4G, and WiFi.

In the embodiment of the present application, the terminal may actively upload the captured image information to the computing device for receiving by the computing device, or the computing device may actively acquire the image information captured by the terminal.

102. Obtain location information of the terminal.

In the embodiment of the present application, the location information of the terminal may be completed by using a location. Specifically, in a specific implementation process, a GPS (Global Positioning System) may be used to determine the latitude and longitude of the terminal, combined with the map information. The terminal performs positioning. In another specific implementation process, the location of the terminal may be determined by using the base station information, and the terminal reports the identity of the base station to which the current base station belongs and the signal strength to the positioning device, thereby determining the coordinates of the current location of the terminal.

In the embodiment of the present application, the terminal may be actively uploaded to the computing device by the computing device, and may be received by the computing device, or the computing device may acquire the information of the terminal to obtain the location information. By obtaining the location information of the terminal, the location of the terminal can be determined more accurately, and the difficulty of identifying subsequent image information is minimized, and the basis for improving the accuracy of image information recognition is established.

103. Retrieve the AR model according to the location information and the image information.

In the embodiment of the present application, since the location of the terminal may change, and the acquired location information may have a certain error, in order to improve accuracy, first, the retrieval range may be determined according to the location information. In a specific implementation process, it may be an area ranging from a certain range of position information, for example, a plane area 50 meters away from the positioning position.

Then, the building information within the search range is retrieved. In the process, information of the location of the building may be determined depending on the map information or information provided by the building supplier, and the result of the search is that the building includes the names of each building and each building, and each Information such as the number of the building.

Next, image information corresponding to each building in the building information is acquired. In the embodiment of the present application, the image information includes 3D model information and 2D picture information, and the image information may be stored in advance in a database. The 3D model information may be provided by a designer of the building, or may be obtained by the user scanning and reconstructing the appearance of the building using a camera or the like, and generally includes the complete three-dimensional shape of the building. The 2D picture information can be collected by the user and uploaded to the computing device. In general, it can be either the overall appearance of the building or a partial view of the building. In the embodiment of the present application, both the 3D model information and the 2D picture information need to be manually labeled to determine the area or location of the superimposed AR model. Therefore, to acquire image information corresponding to the building information, it is necessary to determine the 3D model information of the building according to the name or number of the building, or to determine the 2D picture information of the building according to the name or number of the building. In the embodiment of the present application, the image information corresponding to the building is acquired for comparison with the photographic information acquired by the user, and the building photographed by the user is determined.

Then, the image information included in the image information is determined according to the image information. In the embodiment of the present application, the image comparison method is used to determine whether the object in the image information is the same as the image information corresponding to the previously acquired building. Then you can determine which building the image information contains. Specifically, the feature points of the main content and the feature points of the image information in the image information are respectively extracted, and the feature points of the two are compared, and the comparison result shows that the error is within a certain range, which can be understood as the corresponding structure of the main content in the image information. The object is the same as the building represented by the image information.

Finally, the AR model corresponding to the building is retrieved. In the embodiment of the present application, different AR models, such as pictures, videos, 3D models, animations, and the like, are stored in advance in the cloud processing device. Correspondingly, different AR models correspond to different building image information, which can be manually matched or marked. When the building is determined, the corresponding AR model can be obtained by direct retrieval.

It should be noted that, in the embodiment of the present application, when acquiring image information corresponding to each building in the building information, the 3D model information corresponding to each building in the building information is preferentially acquired; The 3D model information corresponding to the building information is acquired, and 2D picture information corresponding to each building in the building information is acquired. The reason why the 3D model is preferably used is that the 3D model is a complete shape of the building, and when the AR model is superimposed, the operation can be performed more accurately, and the 3D model is more robust to changes in local details and changes in illumination, and Conducive to the subsequent calculation of pose. When there is no 3D model, the use of 2D images has the advantage of a large amount of data and a wide range of images.

104. Send the AR model to the terminal.

In the embodiment of the present application, after the AR model is sent to the terminal, the terminal starts the superimposition operation according to the received AR model, and in order to accurately superimpose the AR model into the image information, first calculate the building and the image information. The relative pose of the building in the image information. Specifically, in a specific implementation process, the feature points of the building in the image information are first extracted, and then the feature points of the building in the image information are extracted, and then the relative positions of each feature point are respectively calculated according to the feature point matching principle. The relationship determines the relative pose of the building in the image information based on the relative positional relationship.

Then, the AR model is superimposed on the image information according to the relative pose to form the superimposed image information, so that the pose of the AR is the same as the position of the building in the image information. The superimposed image information is displayed in the display unit.

The information processing method provided by the embodiment of the present invention acquires the corresponding AR model by acquiring the image information and the position information captured by the terminal, and then combining the position information and the image information, wherein the retrieval of the AR model according to the position information can narrow the search range and improve the search range. The retrieval accuracy, combined with the image information captured by the terminal to determine the corresponding AR model, can improve the retrieval efficiency, improve the recognition rate of static objects, and solve the problem of large amount of retrieval and low accuracy in the prior art.

When the user holds the terminal, or when the user is walking, the position or angle of the terminal is not fixed. Accordingly, the image information captured by the terminal is not fixed. Therefore, in order to enhance the immersion and visually more Smooth, on the basis of the foregoing content, the terminal can also collect inertial measurement data, determine the moving posture of the terminal according to the inertial measurement data, and then adjust the position of the AR model. In the embodiment of the present application, an IMU (Inertial Measurement Unit) can be used to calculate the mobile pose of the terminal when the position or angle of the terminal changes, and specifically, the acceleration of the IMU in the terminal is collected. The change value, the angular velocity change value, and the like, and then calculate the position change of the terminal according to the data value, that is, move the pose. Furthermore, the position of the AR model in the superimposed image information is updated according to the moving pose.

Therefore, the technical solution provided by the embodiment of the present application realizes that the AR model moves and moves along with the image information captured by the user, improves the timeliness, and is more visually smooth and enhances interaction with the user.

In the embodiment of the present application, a large amount of data is stored in the computing device. Therefore, in order to improve the speed and accuracy of the search, a relational database may be established in advance. Specifically, FIG. 2 is an implementation of the image processing method provided by the embodiment of the present application. Another flowchart of the example, as shown in FIG. 2, the information processing method provided by the embodiment of the present application may further include the following steps before the step 101:

100. Establish at least one relational database according to the building information, the image information, and the AR model.

In the embodiment of the present application, the image information may be determined for each building by using the map information and the image information uploaded by the user through various channels, and the image information is marked to determine the position of the AR model superposition or the AR model superposition. The area, and, for each building, match the corresponding AR model, establish a complete relational database about building information, image information, and AR models or separately establish a relational database.

Moreover, the content of the database can be constantly changed, adjusted according to actual needs, and content addition, deletion, modification, and the like.

The purpose of the database is to improve the retrieval accuracy and efficiency, so that the AR model can be superimposed on the image information captured by the user more accurately.

In order to implement the method in the foregoing, the embodiment of the present application further provides an information processing apparatus. FIG. 3 is a schematic structural diagram of an embodiment of an information processing apparatus according to an embodiment of the present application. As shown in FIG. 3, the apparatus of this embodiment may be used. The first acquisition unit 11, the second acquisition unit 12, the retrieval unit 13, and the transmission unit are included.

The first obtaining unit 11 is configured to acquire image information captured by the terminal;

The second obtaining unit 12 is configured to acquire location information of the terminal.

The searching unit 13 is configured to retrieve the AR model according to the location information and the image information;

The sending unit 14 is configured to send the AR model to the terminal.

In a specific implementation process, the retrieval unit 13 is specifically configured to:

Determining the search range based on the location information;

Searching for building information within the search scope;

Obtaining image information corresponding to each building in the building information;

Determining a building included in the image information according to the image information;

Retrieve the AR model corresponding to the building.

In a specific implementation process, the acquiring image information corresponding to each building in the building information includes:

Obtaining 3D model information corresponding to each building in the building information; or

If the 3D model information corresponding to the building information is not acquired, the 2D picture information corresponding to each building in the building information is acquired.

The information processing apparatus provided by the embodiment of the present application may be used to implement the technical solution of the method embodiment shown in FIG. 1 , and the implementation principle and technical effects thereof are similar, and details are not described herein again.

Further, the embodiment of the present application further provides an information processing apparatus, and FIG. 4 is another schematic structural diagram of an embodiment of an information processing apparatus according to an embodiment of the present application. As shown in FIG. 4, the apparatus of this embodiment is in the foregoing content. On the basis of, it may also include: establishing unit 15.

The establishing unit 15 is configured to establish at least one relational database according to the building information, the image information and the AR model.

The information processing apparatus provided by the embodiment of the present application may be used to implement the technical solution of the method embodiment shown in FIG. 2, and the implementation principle and technical effects are similar, and details are not described herein again.

FIG. 5 is a schematic structural diagram of an embodiment of a cloud processing device according to an embodiment of the present disclosure. As shown in FIG. 5, the cloud processing device provided by the embodiment of the present disclosure may specifically include: a processor 21 and a memory 22.

The memory 21 is configured to store instructions that, when executed by the processor 22, cause the device to perform any of the methods shown in FIG. 1 or FIG.

The cloud processing device provided by the embodiment of the present application may be used to implement the technical solution of the method embodiment shown in FIG. 1 or FIG. 2, and the implementation principle and the technical effect are similar, and details are not described herein again.

The embodiment of the present application further provides a computer program product, which can be directly loaded into an internal memory of a computer and contains software code. After the computer program is loaded and executed by the computer, any method as shown in FIG. 1 or FIG. 2 can be implemented. .

The computer program product provided by the embodiment of the present application may be used to implement the technical solution of the method embodiment shown in FIG. 1 or FIG. 2, and the implementation principle and the technical effect are similar, and details are not described herein again.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform the methods of the various embodiments of the present application. Part of the steps. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

The above is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., which are made within the spirit and principles of the present application, should be included in the present application. Within the scope of protection.

Claims (10)

An information processing method, comprising: Obtaining image information captured by the terminal; Obtaining location information of the terminal; Retrieving an AR model according to the location information and the image information; The AR model is sent to the terminal. The method according to claim 1, wherein the retrieving the AR model based on the location information and the image information comprises: Determining a search range according to the location information; Retrieving all building information within the search range; Obtaining image information corresponding to each building in the building information; Determining a building included in the image information according to the image information; The AR model corresponding to the building is retrieved. The method according to claim 2, wherein the acquiring image information corresponding to each building in the building information comprises: Obtaining 3D model information corresponding to each building in the building information; or If the 3D model information corresponding to the building information is not acquired, the 2D picture information corresponding to each building in the building information is acquired. The method according to any one of claims 1 to 3, wherein the method further comprises: Establish at least one relational database based on the building information, the image information, and the AR model. An information processing apparatus, comprising: a first acquiring unit, configured to acquire image information captured by the terminal; a second acquiring unit, configured to acquire location information of the terminal; a retrieval unit, configured to retrieve an AR model according to the location information and the image information; And a sending unit, configured to send the AR model to the terminal. The device according to claim 5, wherein the retrieval unit is specifically configured to: Determining a search range according to the location information; Retrieving building information within the search range; Obtaining image information corresponding to each building in the building information; Determining a building included in the image information according to the image information; The AR model corresponding to the building is retrieved. The device according to claim 6, wherein the acquiring image information corresponding to each building in the building information comprises: Obtaining 3D model information corresponding to each building in the building information; or If the 3D model information corresponding to the building information is not acquired, the 2D picture information corresponding to each building in the building information is acquired. The device according to any one of claims 5 to 7, wherein the device further comprises: The establishing unit is configured to establish at least one relational database according to the building information, the image information and the AR model. A cloud processing device, characterized in that the device comprises a processor and a memory; the memory is for storing instructions, when the instructions are executed by the processor, causing the device to perform as claimed in claims 1 to 4 Any of the methods described. A computer program product, which can be directly loaded into an internal memory of a computer and containing software code, which can be implemented by any one of claims 1 to 4 after being loaded and executed by a computer Methods.

Images