Disclosure of Invention
The invention aims to provide a digital exhibition interaction system and a digital exhibition interaction method.
To achieve the above object, in a first aspect, an embodiment of the present disclosure provides a digital exhibition interaction system, including:
The augmented reality module is used for generating augmented reality content related to the exhibition content and pushing the AR exhibition content to the exhibition terminal;
The intelligent display module is used for identifying the exhibits according to the images acquired by the image acquisition device and superposing the related information and the interactive elements on the display terminal, capturing the interactive actions of the exhibitors on the AR content of the exhibits, and adjusting the AR display content according to the interactive actions;
the intelligent positioning module is used for analyzing the distance between the exhibitor and the exhibited article according to the acquired image and adjusting the size and the precision of the AR exhibited content according to the distance;
the intelligent guiding module is used for guiding according to the visiting route and the acquired images, and is also used for adjusting basic explanation contents according to the stay time and interaction actions of the exhibitors.
The basic scheme has the advantages that the AR display content closely related to the display content is generated by the augmented reality module of the basic scheme and is pushed to the display terminal in real time to serve as the basis of digital display, so that the display form of the display is enriched, and the immersion and interestingness of the display are enhanced.
The intelligent display module utilizes the image acquisition device to identify the exhibited article and superimposes related information and interaction elements on the display terminal, so that the exhibitor can rapidly acquire accurate information related to the exhibited article. The intelligent positioning module adjusts the size and the precision of the AR display content according to the distance between the exhibitor and the exhibit, so that the clear and readable AR display content can be obtained no matter what position the exhibitor is located. The intelligent guide module is used for guiding according to the visiting route and the acquired images, so that the situation that a guide is lost or misses important exhibits is avoided, the exhibition viewing efficiency can be improved, the AR display module is used for guiding, the technical uniformity is better, and the intelligent guide module is more visual and accurate. In addition, the interest points of the exhibitors can be analyzed, the explanation content can be provided in a personalized manner, and the exhibitor experience is improved.
In addition, the technical characteristics are realized by using an image analysis technology, so that the hardware cost is reduced, and the system is convenient to deploy.
As an implementation preferred scheme, the intelligent display module collects the exhibit image through the image collecting device on the display terminal, compares the exhibit image with the exhibit image database pre-stored in the cloud platform to identify the exhibit, and superimposes AR display content related to the exhibit after the identification is successful.
As an implementation preferable scheme, the intelligent positioning module analyzes the distance between the exhibitor and the exhibited item according to the image acquired by the image acquisition device, and adjusts the size and the precision of the AR exhibited content.
As an implementation preferable scheme, the intelligent positioning module generates a visit route of a exhibitor according to the zoom record and sequence of display contents corresponding to the exhibitors and combining with the positions of the exhibitors stored in the cloud platform;
The intelligent positioning module takes the exhibited item closest to the exhibited item when the exhibited person stays last as a target point, and other exhibited items scaled in the section are only used for correcting the moving route from the previous target point to the next target point, and interest points are marked on the visiting route according to the action record of the exhibited item by the exhibited person.
As an implementation preferred scheme, the intelligent navigation module learns tracks of a large number of exhibitors by using the neural network model, analyzes the current nearest exhibits of the exhibitors through the images acquired by the image acquisition module, judges the directions of the exhibitors according to other exhibits in the images, and guides directions on the display interface of the display terminal according to the visit route.
The digital exhibition interactive system according to claim 1 is characterized in that the intelligent navigation module adjusts basic explanation contents by using a generated large model, each exhibit is provided with the basic explanation contents, and the generated large model adjusts the basic explanation contents according to the stay time of a exhibitor and the interaction action;
The intelligent guide module is also used for collecting characters or voices input by the exhibitors and answering questions of the exhibitors by using the generated large model or remotely connecting with a manual tour guide to conduct online explanation.
As an implementation preferred scheme, the intelligent navigation module is further used for tracking the attention points of the exhibitors according to the eyeball tracking device or the front camera, marking the attention points as interest points and expanding basic explanation contents according to the interest points.
As an implementation preferable scheme, the intelligent guide module is further used for analyzing the relevant part of the current exhibit and the previous explanation content and carrying out content linkage by taking the relevant part as the requirement of the current exhibit to generate the explanation content.
As an implementation preferred scheme, the intelligent navigation module is further used for conducting navigation route planning, identifying congestion points of the navigation route according to crowd density around image analysis exhibits of the image acquisition device, and analyzing congestion degrees and congestion duration time of the congestion points in different time periods according to historical visit data stored by the cloud platform.
In a second aspect, an embodiment of the present disclosure further provides a digital exhibition interaction method, which uses the digital exhibition interaction system described above.
Detailed Description
In order to make the technical scheme and the advantages thereof more clear, the technical scheme of the application will be described in further detail with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be regarded as being isolated, and they may be combined with each other to achieve a better technical effect. The same reference numerals appearing in the drawings of the embodiments described below represent the same features or components and are applicable to the different embodiments.
Furthermore, unless defined otherwise, technical or scientific terms used in the description of the invention should be given the ordinary meaning as understood by one of ordinary skill in the art to which the invention pertains.
Reference numeral 500, a processor 501, a communication interface 502, a memory 503, and a bus 504.
The invention is described in further detail below with reference to the accompanying drawings:
Referring to fig. 1, a digital display interaction system, comprising:
The augmented reality module comprises a content generation sub-module and a content pushing sub-module.
The content generation sub-module is used for generating AR content related to the exhibition content, and comprises a three-dimensional model, video, images and the like, and comprises the steps of data collection, model construction, rendering, release and the like.
And in the data collection stage, physical information of the exhibits, such as size, shape, color and the like, is acquired by scanning, photographing and the like. And then carrying out model construction, and constructing a three-dimensional model of the exhibit according to the collected data by utilizing three-dimensional modeling software. In the rendering stage, the system performs processing such as coloring, mapping and the like on the three-dimensional model. And finally, publishing the rendered AR content to the cloud platform.
And the content pushing sub-module is used for pushing the AR display content to a display terminal (such as a smart phone, a tablet personal computer, smart glasses and the like). When a guest enters the exhibition area, the guest's exhibition terminal will connect to the cloud platform, receiving the corresponding AR content.
The intelligent display module comprises an exhibit display sub-module and a gesture recognition sub-module.
The exhibit display submodule is used for identifying exhibits and superposing related information and interaction elements on a display terminal. And acquiring an exhibit image through an image acquisition device on the display terminal, comparing the exhibit image with an exhibit image database pre-stored in the cloud platform, identifying the exhibit, and superposing AR display content related to the exhibit on tourist equipment after the identification is successful.
And the gesture recognition sub-module is used for capturing interaction actions of the exhibitors on the exhibited AR content, wherein the interaction actions comprise zooming and rotating. The exhibition sub-module correspondingly adjusts the AR exhibition content according to the captured action instruction, for example, an exhibitor performs double-finger amplifying operation on a screen, the exhibition sub-module amplifies a three-dimensional model of the exhibition to display more details, or the exhibitor wears intelligent glasses, the exhibitor lifts hands to rotate the exhibition AR exhibition content in front of eyes, the image acquisition device of the intelligent glasses acquires interaction of the exhibitor, and the exhibition sub-module adjusts the exhibition AR content according to the rotating amplitude and direction of the exhibitor.
The intelligent positioning module comprises a distance analysis sub-module and a track generation sub-module.
The distance analysis sub-module is used for analyzing the distance between the exhibitor and the exhibited item according to the image acquired by the image acquisition device, so that the size and the precision of the AR exhibited content are adjusted, namely, when the image acquired by the image acquisition device contains a plurality of exhibited items, the size of the exhibited item is adjusted according to the distance between the exhibited item and the exhibitor, the smaller the size and the lower the precision of the AR exhibited content corresponding to the exhibited item far away from the exhibitor, the larger the size and the higher the precision of the AR exhibited content corresponding to the exhibited item near the exhibitor are, so that the AR exhibited content is distributed in a staggered manner, the emphasis is on, and the exhibitor is convenient to know the positions of other exhibited items which can have AR exhibited content. According to the technical scheme, an indoor positioning technology is not required, hardware requirements are lower, the popularization of the system is facilitated, meanwhile, due to the fact that indoor exhibits are distributed densely, the indoor positioning technology is sometimes disturbed, the problem is not accurate enough, the image acquired by the image acquisition device can be used for synchronizing the positions of the exhibitors, when a reference person approaches to an exhibit, the size of the exhibit corresponding to the AR exhibited content is gradually improved, the accuracy of the exhibit is improved, and the size and the accuracy of the exhibit corresponding to the AR exhibited content are gradually reduced when the exhibit is kept away from.
The track generation submodule is used for generating a visiting route of the exhibitor, the technical scheme does not depend on indoor positioning technology, the visiting route of the exhibitor is generated according to the zoom record and sequence of the display content corresponding to the exhibitions and combining with the display position stored in the cloud platform, the situation that a plurality of exhibitions zoom simultaneously in the moving process of the exhibitor is possible, under the situation, the exhibitor stays the nearest exhibitor as a target point in the last time, and other exhibitions zoomed in the section are only used for correcting the moving route from the last target point to the next target point, so that the situation that the visiting route is too complicated or turns around in situ is avoided, the interference of the intelligent navigation module during learning is reduced, and in addition, interest points are marked on the visiting route according to the action record of the exhibitor. Through generating the visit route of the exhibitor and marking the interest points, the overall preference of the exhibitor can be analyzed, the recommended visit route is set, or the recommended visit route is automatically generated according to the visit route overlapped by most exhibitors, the display content of the exhibits interested by the exhibitor can be further enriched, and meanwhile, the too dense interest points can be conveniently adjusted and optimized, so that the congestion of the visit route is avoided.
The intelligent guide module comprises a path guide sub-module and an intelligent explanation sub-module.
The path guiding sub-module learns the tracks of a large number of exhibitors by using the neural network model, and is used for guiding according to the recommended visit route stored in the cloud platform and combining the acquired images. The image collected by the image collection module is analyzed to obtain the current nearest exhibited item of the exhibitor, the direction of the exhibitor is judged according to other exhibited items in the image, and direction guidance is carried out on the display interface of the display terminal according to the recommended visit route, and the direction guidance can be in the modes of arrows or characters and the like. The track generation submodule records the visit routes of a large number of visitors, so that the visitors with different directions can be guided to the next target point of the recommended visit route, and the visitors can be guided to the next target point when the route congestion causes the visitors to deviate from the route in the recommended visit route, and the flexibility and usability of navigation are improved.
The intelligent explanation sub-module adjusts basic explanation contents by utilizing the generation type large model, each piece of exhibit is provided with the basic explanation contents, and the generation type large model expands, reduces, associates and other contents according to the residence time, interaction action and the like of the exhibitor. When the stay time of the exhibitor is longer than the average stay time corresponding to the exhibit, the intelligent explanation sub-module expands basic explanation contents by using the generated large model, and when the exhibitor starts to move to other exhibits (namely AR display contents corresponding to the current explanation start to shrink and AR display contents of other exhibits start to amplify), short summary is carried out by using the generated large model, and the current explanation is ended. When the exhibitor performs interaction on the current exhibited item, for example, a certain detail of the exhibited item is amplified, or the exhibition item is rotated to a certain specific angle, the intelligent explanation sub-module expands basic explanation contents by using the generated large model, enriches the explanation contents and meets the interests of the exhibitor.
In an embodiment, the intelligent navigation module further includes an interest analysis sub-module, configured to track an attention point of a exhibitor according to an eye tracking device on the intelligent glasses, a front camera on the mobile phone or the tablet, and an algorithm, mark the attention point as an interest point of the exhibitor, and the intelligent explanation sub-module expands basic explanation content according to the interest point.
In an embodiment, the intelligent guide module further comprises an intelligent question-answering sub-module, which is used for collecting characters or voices input by the exhibitors, answering questions of the exhibitors by using the generated large model, and timely solving questions of the exhibitors, for example, the questions of the exhibitors can not be solved, the intelligent question-answering sub-module can be used for remotely connecting with a manual tour guide, checking the questions of the exhibitors to conduct online explanation, so that the explanation efficiency is improved.
In an embodiment, the intelligent guide module further comprises an association analysis sub-module, wherein the association analysis sub-module is used for analyzing the relevant parts in the current exhibits and the previous explanation contents, the relevant parts comprise exhibits, features and interest points related to the current exhibits, and the relevant parts are used as requirements of the current exhibits for generating the explanation contents to carry out content linkage, so that the logic and depth of the explanation are improved, the exhibitors are helped to quickly establish a knowledge system of the exhibits, and the related explanation contents can be closer to the contents which the exhibitors want to know due to the fact that the previous explanation contents are originally generated according to the interest points of the exhibitors, and the explanation effects are improved.
Example two
The distinguishing technical features of this embodiment and the foregoing embodiment are that, the intelligent tour guide module is further configured to perform tour guide planning, analyze crowd density around the exhibits according to an image of the image acquisition device (or directly using data of the exhibition hall monitoring device), identify congestion points of the tour guide, and analyze congestion degrees and congestion durations of the congestion points in different time periods according to historical visit data stored by the cloud platform. The order of the exhibited item visit is reordered as follows:
taking the currently selected visit route as an initial scheme Setting an initial temperatureCooling rateAnd termination temperature. The initial temperature is dynamically adjusted according to the remaining distance of the current visit route, and the initial temperature T is higher as the distance is longer, so that more iterations and cooling rates are carried outDynamically adjusting according to the congestion degree of the next exhibit of the current visit route, wherein the higher the congestion degree is, the cooling rate isThe larger.
Setting fitness functionsThe formula is as follows:
Wherein, Representing the path distance from the kth exhibit to the kth+1th exhibit; representing the speed of movement of the exhibitor; Representing the crowd density of the kth+1st exhibit; representing the path congestion degree from the kth exhibit to the kth+1th exhibit; representing the total waiting time of the exhibits, and estimating according to the crowd density and the capacity of the exhibits; as fatigue factors, comprehensively evaluating factors such as movement difficulty among exhibits according to the total length of a visit path; 、、、 And the weight coefficient is adjusted according to actual conditions.
Random disturbance current schemeGenerating a proximity scheme. Calculating the performance of a new schemeIf the scheme performance is better, a new scheme is accepted, and if the performance is worse, the new scheme is accepted with a certain probability, and the formula is as follows:
Wherein, Is a constant value, and is used for the treatment of the skin,Indicating the current temperature of the device and,The receiving degree of the direction guide of the representing exhibitors is increased, the probability of receiving the new scheme is increased, and the probability of not receiving is reduced, so that the number of changing prompts on the exhibitor route can be reduced in actual experience, and if the exhibitors do not receive the direction guide, the guiding system repeatedly prompts the user, and the exhibitor experience is poor.
Updating temperature parameters。
And repeating the iteration until reaching the termination temperature, and selecting the weight allocation scheme with the highest performance as the final resource allocation scheme.
In the visit process of the exhibitor, the intelligent guide module continuously detects the changes of crowd density and congestion condition, adjusts the guide route in real time, ensures the exhibitor to visit along the optimal path,
The embodiment of the disclosure also provides a digital exhibition interaction method, which uses a digital exhibition interaction system.
The embodiment of the disclosure also provides a storage medium, in which a computer program is stored, and when the computer program is executed by a processor, all the steps of the digital exhibition interaction method can be realized.
Those skilled in the art will appreciate that implementing all or part of a digital display interaction method may be accomplished by computer programs in a non-volatile computer readable storage medium, which when executed, may comprise the steps of various embodiments of a digital display interaction method. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The embodiment of the application also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the digital exhibition interaction method when executing the program. In the embodiment of the application, the processor is a control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine.
Referring to fig. 2, the electronic device 500 comprises at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. Where bus 504 is used to enable connectivity communications between these components, communication interface 502 is used to communicate signaling or data with other node devices, and memory 503 stores machine readable instructions executable by processor 501. When the electronic device 500 is in operation, the processor 501 and the memory 503 communicate via the bus 504, and machine readable instructions, when invoked by the processor 501, perform the steps of a digital display interaction method as described above.
The foregoing is merely exemplary of the present application, and the specific structures and features well known in the art will not be described in any way, so that those skilled in the art will be able to ascertain all prior art in the field, and will not be able to ascertain any prior art to which the application pertains, without the general knowledge of the specific structures and features of the application, before the filing date or the priority date, with the ability to apply the conventional practice of the application, as it would be well known to those skilled in the art, with the benefit of this disclosure, to make various embodiments with the ability to work in mind, and to make certain typical structures or methods well known to those skilled in the art. It should be noted that modifications and improvements can be made by those skilled in the art without departing from the structure of the present application, and these should also be considered as the scope of the present application, which does not affect the effect of the implementation of the present application and the utility of the patent. The protection scope of the present application is subject to the content of the claims, and the description of the specific embodiments and the like in the specification can be used for explaining the content of the claims.