CN116258811B - Information transmission method, device, electronic device and computer readable medium - Google Patents

Abstract

Embodiments of the present disclosure disclose information sending methods, devices, electronic equipment, and computer-readable media. A specific implementation of the method includes: acquiring a real-time panoramic video; in response to detecting at least one target detection item in the real-time panoramic video, determining a real-time panoramic image corresponding to the current detection moment, a target real-time panoramic image in the real-time panoramic video, and a real-time observation perspective image corresponding to the current observation angle of view, wherein the real-time observation angle of view image and the target real-time panoramic image are images at the same time; according to the target real-time panoramic image and the real-time observation angle of view image, generating at least one candidate torsion information for the screen display device corresponding to the current observation angle of view; screening the candidate torsion information that meets the preset conditions from the at least one candidate torsion information, as the optimal twist Turn information; send the optimal twist information to the screen display device. In this embodiment, at least one target detection item can be observed quickly and efficiently through optimal twist information.

Description

Information transmission method, device, electronic device and computer readable medium

technical field

The embodiments of the present disclosure relate to the field of computer technology, and in particular, to an information sending method, device, electronic device, and computer-readable medium.

Background technique

At present, the technology of target recognition through the images captured by the panoramic camera is more and more applied to various fields. For observing the user to watch at least one target detection item, the usual way is to watch the real-time situation of each target detection item by continuously adjusting the panoramic display device artificially.

However, the inventors found that when using the above method to watch the real-time situation of each target detection item, there are often the following technical problems:

First, the adjustment efficiency of artificially adjusting the panoramic display device is low, resulting in poor user experience.

Second, the generation of the path length information and the corresponding path safety factor information is not accurate enough, which leads to the inaccuracy of the optimal torsion route screened subsequently.

The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art.

Contents of the invention

The Summary of the Disclosure is provided to introduce concepts in a simplified form that are described in detail in the Detailed Description that follows. The content of this disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

Some embodiments of the present disclosure propose an information sending method, device, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the background art section above.

In a first aspect, some embodiments of the present disclosure provide an information sending method, including: acquiring a real-time panoramic video of a target area captured by a panoramic camera; in response to detecting at least one target detection item in the real-time panoramic video, determining a real-time observation perspective image corresponding to a target real-time panoramic image in the real-time panoramic video and a current observation perspective corresponding to the current detection moment, wherein the real-time observation perspective image and the above-mentioned target real-time panoramic image are images at the same time; Twist information: Screen out candidate twist information satisfying preset conditions from the at least one piece of candidate twist information as optimal twist information; and send the optimal twist information to the screen display device.

In a second aspect, some embodiments of the present disclosure provide an information sending device, including: an acquisition unit configured to acquire a real-time panoramic video of a target area captured by a panoramic camera; a determination unit configured to respond to the detection of at least one target detection item in the real-time panoramic video, and determine a real-time observation perspective image corresponding to a target real-time panoramic image in the real-time panoramic video and a current observation angle corresponding to the current detection moment, wherein the real-time observation perspective image and the above-mentioned real-time panoramic image of the target are images at the same time; Observing the viewing angle image, generating at least one piece of candidate twist information for the screen display device corresponding to the current observation angle of view; the screening unit is configured to filter out the candidate twist information satisfying the preset condition from the above at least one piece of candidate twist information, as the optimal twist information; the sending unit is configured to send the above optimal twist information to the above screen display device.

In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; and a storage device, on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, wherein, when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the information sending method of some embodiments of the present disclosure, at least one target detection item can be viewed quickly and efficiently through optimal twist information. Specifically, the inaccurate reason for viewing at least one object detection object is that the adjustment efficiency of artificially adjusting the panoramic display device is low, resulting in poor user experience. Based on this, the information sending method of some embodiments of the present disclosure first acquires a real-time panoramic video of a target area captured by a panoramic camera. Here, the obtained real-time panoramic video is used for subsequent detection of at least one object detection object, and is also used for subsequent generation of at least one candidate torsion information. Then, in response to detecting at least one object detection object in the real-time panoramic video, determine the real-time observation perspective image corresponding to the target real-time panoramic image in the real-time panoramic video and the current observation perspective corresponding to the current detection moment. Wherein, the real-time observation perspective image and the target real-time panoramic image are images at the same moment. Here, the obtained real-time panoramic image of the target and the above-mentioned real-time observation perspective image are used to subsequently generate at least one piece of candidate torsion information. Furthermore, according to the target real-time panoramic image and the real-time observation angle image, at least one piece of candidate twist information for the screen display device corresponding to the current observation angle can be accurately generated. Next, the candidate twist information that satisfies the preset condition is selected from the above at least one piece of candidate twist information, and used as the optimal twist information for subsequent twisting of related users. Finally, the above-mentioned optimal twist information is sent to the above-mentioned screen display device for twisting by relevant users. To sum up, through the generation of at least one piece of candidate twist information and the screening of candidate twist information, the optimal twist information is sent to the screen display device for relevant users to view the target detection item.

Description of drawings

The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and elements and elements have not necessarily been drawn to scale.

FIG. 1 is a flowchart of some embodiments of an information sending method according to the present disclosure;

Fig. 2 is a schematic structural diagram of some embodiments of an information sending device according to the present disclosure;

FIG. 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these examples are provided so that the understanding of this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings. In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only, and are not used to limit the scope of these messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings and embodiments.

Referring to FIG. 1 , a process 100 of some embodiments of the information sending method according to the present disclosure is shown. The information sending method includes the following steps:

Step 101, acquiring a real-time panoramic video of a target area captured by a panoramic camera.

In some embodiments, the execution subject of the above information sending method may obtain the real-time panoramic video of the target area captured by the panoramic camera through a wired connection or a wireless connection. The panoramic camera may be a camera that captures panoramic images. A panoramic camera can include multiple cameras. The target area may be the area of the item to be detected.

Step 102, in response to detecting at least one target detection object in the real-time panoramic video, determine the real-time panoramic image of the target in the real-time panoramic video and the real-time observation perspective image corresponding to the current observation perspective corresponding to the current detection moment.

In some embodiments, in response to detecting at least one target detection item in the real-time panoramic video, the execution subject may determine the real-time panoramic image of the target in the real-time panoramic video and the real-time observation perspective image corresponding to the current observation perspective corresponding to the current detection moment. The target detection item may be an item to be detected. For example, the target detection item may be but not limited to at least one of the following: trees, vehicles, and houses. The target real-time panoramic image may be a panoramic image captured by a panoramic camera at the current detection moment. The real-time observation angle image may be an image under the current observation angle of the real-time observation user at the current detection moment.

Step 103: Generate at least one piece of candidate twist information for the screen display device corresponding to the current observation angle of view according to the target real-time panoramic image and the real-time observation angle of view image.

In some embodiments, the execution subject may generate at least one piece of candidate twist information for the screen display device corresponding to the current observation angle in various ways according to the target real-time panoramic image and the real-time observation angle image. Wherein, the current observation angle of view may be an observation angle of view that the real-time observation user can observe at the above-mentioned current detection moment. The screen display device may be a device for real-time observation of the screen of the area that the user can watch. For example, the screen display device may be a mobile phone terminal, or a VR wearable device. The candidate twist information may be twist information that the screen display terminal needs to twist to display at least one target detection item.

In some optional implementation manners of some embodiments, the candidate twist information in the at least one piece of candidate twist information is a candidate twist route. Wherein, the candidate twisting route may be a twisting route that the screen display terminal needs to twist in order to display at least one target detection item.

And the generating of at least one piece of candidate twist information for the screen display device corresponding to the above-mentioned current observation angle of view according to the above-mentioned real-time panoramic image of the target and the above-mentioned real-time observation angle of view image may include the following steps:

In the first step, according to the number of cameras included in the panoramic camera, image splitting is performed on the real-time panoramic image of the target to obtain a split image set. Wherein, the number of split images included in the split image set is the same as the number of cameras mentioned above. For example, if the number of cameras is 4, the number of split images included in the obtained split image set is 4.

As an example, the execution subject may use panoramic image splitting software to perform image splitting on the target real-time panoramic image to obtain a split image set.

In the second step, in a two-dimensional tiling and mosaic manner, the above split image sets are tiled and mosaiced to obtain tiled images.

As an example, in the manner of two-dimensional tiling and mosaicing, by splitting the original spatial positions corresponding to the images, the above split image set is subjected to tiling and mosaic processing to obtain tiled images.

The third step is to determine the same image area information as the corresponding image content of the above-mentioned real-time observation perspective image from the above-mentioned tiled image, and mark the position of at least one item corresponding to the above-mentioned at least one target detection item in the above-mentioned tiled image.

As an example, the execution subject may determine from the tiled image the image area information corresponding to the image corresponding to the real-time observation perspective image by means of area pixel comparison, and mark at least one item position corresponding to the at least one target detection item in the tiled image.

The fourth step is to generate at least one candidate torsion route according to the above image area information and the above at least one item position.

As an example, first, the execution subject may determine the position of the center of the area corresponding to the image area information. Then, randomly connect the position of the center of the area and the position of at least one item to obtain at least one connection curve as at least one candidate torsion route. Among them, the obtained connection curves do not have the same position.

Optionally, the above-mentioned generating at least one candidate twisting route according to the above-mentioned image area information and the above-mentioned at least one item position may include the following steps:

The first step is to obtain the positional relationship information between the above-mentioned screen display device and the real-time observation user. For example, at the current detection moment, the screen display device observes in real time the 20 centimeters directly opposite the user's viewing angle.

As an example, the execution subject may acquire the position relationship information between the screen display device and the real-time observation user through a position measuring device.

The second step is to determine the vertical point corresponding to the above-mentioned real-time observation perspective image according to the above-mentioned image area information and the above-mentioned positional relationship information.

As an example, the execution subject may generate a straight line for the screen display device and the real-time observation user according to the positional relationship information as the target straight line. Then, determine the vertical point of the target line in the area corresponding to the image area information.

In the third step, an item information list for the at least one item position is generated. Wherein, the item information includes: item category, item identification. Wherein, the item information list shows at least one item position corresponding to the item category and item identification of at least one item, so that the real-time observation user can select the item he wants to view in the item information list. The item category may be an item category to which the item belongs. The above item identifier may represent the identity information of the item.

Step 4: Send the item information list to the screen display device for the real-time observation user to select and view item information from the item information list.

In the fifth step, in response to receiving the selected item information set and not receiving the viewing sequence, determine the item position set corresponding to the above item information set. There is a one-to-one correspondence between the item information in the item information set and the item locations in the item location set.

Step 6: Input the above-mentioned item position set and the above-mentioned vertical point's position into the path generation model to output the above-mentioned at least one candidate torsion route. Wherein, the route generation model may be a model that generates a torsional route. In practice, the path generation model can be a sequential neural network model.

In some optional implementation manners of some embodiments, the candidate twist information in the at least one piece of candidate twist information is the candidate twist azimuth angle information.

Optionally, the above-mentioned generating at least one piece of candidate twist information for the screen display device corresponding to the above-mentioned current observation angle of view according to the above-mentioned real-time panoramic image of the target and the above-mentioned real-time observation angle of view image may include the following steps:

The first step is to establish a three-dimensional model for the real-time panoramic image of the above target.

Wherein, the three-dimensional stereo model can represent the three-dimensional spatial information of each target in the real-time panoramic image of the target.

The second step is to generate a three-dimensional sub-model for the above-mentioned real-time observation perspective images. Wherein, the above-mentioned three-dimensional sub-model is a sub-model of the above-mentioned three-dimensional three-dimensional model.

Wherein, the three-dimensional sub-model can represent the three-dimensional spatial information of each target in the real-time observation perspective image.

As an example, the execution subject may perform model segmentation on the 3D sub-model according to the image content of the actual observation perspective image and the image content of the target real-time panoramic image, so as to generate a 3D sub-model for the above-mentioned real-time observation perspective image.

The third step is to determine the three-dimensional coordinate points of each target detection item in the at least one target detection item in the above-mentioned three-dimensional solid model, and obtain at least one three-dimensional coordinate point.

In the fourth step, in response to receiving the selected item information set and not receiving the viewing order, respectively convert the sub-model preset 3D center coordinate point corresponding to the above-mentioned 3D three-dimensional sub-model, the model preset three-dimensional center coordinate point corresponding to the above-mentioned three-dimensional three-dimensional model, and the set of three-dimensional coordinate points corresponding to the above-mentioned item information set among the at least one three-dimensional coordinate point into a predetermined three-dimensional data format, and obtain the first three-dimensional center point data, the second three-dimensional center point data and the three-dimensional data set. Wherein, the predetermined three-dimensional data format may be in the form of triplets. The preset three-dimensional center coordinate point of the sub-model may be the coordinate point of the center position of the three-dimensional sub-model. Likewise, the model preset three-dimensional center coordinate point may be the coordinate point of the center position of the three-dimensional solid model.

Step 5: Input the first 3D center point data, the second 3D center point data and the 3D data set into the twist azimuth generation model to output the twist azimuth information set as the candidate twist azimuth information set.

Wherein, the torsion azimuth generation model may be a model that generates the torsion azimuth. In practice, the aforementioned torsional azimuth generation model may be a multi-layer serially connected convolutional neural network (Convolutional Neural Networks, CNN) model. Candidate twist azimuth angle information can represent the user's azimuth twist transformation.

Step 6: Remove candidate torsion azimuth angle information corresponding to a total torsion angle greater than a predetermined angle value from the above candidate torsion azimuth angle information set to obtain at least one piece of candidate torsion azimuth angle information. For example, the predetermined angle value may be 360 degrees.

Step 104 , selecting the candidate twist information satisfying the preset condition from the above at least one piece of candidate twist information, as the optimal twist information.

In some embodiments, the execution subject may select the candidate twist information satisfying a preset condition from the at least one piece of candidate twist information as the optimal twist information. Wherein, the preset condition may be a condition representing the least twisting operation of the screen display device.

Step 105, sending the above optimal twist information to the above screen display device.

In some embodiments, the execution subject may send the optimal twist information to the screen display device.

In some optional implementation manners of some embodiments, the foregoing preset conditions include: a path length subcondition and a path security subcondition. Wherein, the path length sub-condition may be that the path length corresponding to the candidate torsion route is located in a predetermined lower quantile of at least one path length corresponding to at least one candidate torsion information. For example, the predetermined lower quantile is 0.3. For example, at least one path length may include: 10 cm, 11 cm, 13 cm. Then the path length sub-condition is that the value of the path length corresponding to the candidate torsion route in at least one path length corresponding to at least one candidate torsion information is 10 cm. The path safety sub-condition may be that the safety factor of the path is higher than the target value. For example, the target value is 0.6.

And the above-mentioned screening of candidate torsion information satisfying preset conditions from at least one piece of candidate torsion information as optimal torsion information may include the following steps:

The first step is to obtain the user environment information corresponding to the real-time observed user.

Wherein, the user environment information may be real-time observation of real-time environment change information where the user is located. In practice, the above real-time environment change information may be in the form of video. Specifically, the real-time environment transformation information may include: pedestrian transformation information, vehicle transformation information, and position transformation information.

The second step is to determine the path length information and corresponding path safety factor information corresponding to each candidate twisting route in the at least one candidate twisting route according to the user environment information.

In the third step, according to the path length information corresponding to each candidate torsion route and the corresponding path safety factor information, a candidate torsion route that satisfies the above-mentioned path length sub-condition and the above-mentioned path safety sub-condition is selected from the at least one candidate torsion route as the optimal torsion route.

In some optional implementations of some embodiments, according to the above user environment information, determining the path length information and corresponding path safety factor information corresponding to each candidate twisting route in the at least one candidate twisting route may include the following steps:

The first step is to input each user image in the user image sequence corresponding to user environment information into the target recognition model to output the target information set to obtain the target information set group. Wherein, the target information may be position information of the target appearing in the image. In practice, the aforementioned objects can be one of the following: people, animals, plants, vehicles. The target recognition model may be a model of information that recognizes the target. In practice, the object recognition model can be a Faster RCNN model.

In the second step, according to the target time interval, the above-mentioned target information sets are grouped to obtain a grouping result. In practice, the target time interval may be preset. For example, the target time interval is 0.3 milliseconds.

In the third step, according to the grouping result, the transformation situation of each target information corresponding to the target in each group is determined.

Wherein, the change situation may include: the change speed of the target and the change direction of the target.

The fourth step is to summarize the conversion conditions corresponding to each target information in each group, so as to generate target conversion information for each target. The object transformation information may represent the transformation situation of the object in the corresponding time period of the video associated with the user environment information.

The fifth step is to input the user image sequence into the road information generation model to output road information. Wherein, the road information generation model may be a model for generating road information. The above road information may include: road length, road shape, and road sign. In practice, the above road information generation model can be a 50-layer serially connected residual network model.

Step 6: Generate path length information corresponding to each candidate twisting route in the at least one candidate twisting route according to the road information.

Step 7: For each candidate twisting route in at least one candidate twisting route, input the road information, the above-mentioned target transformation information for each target, and the candidate twisting route into the path safety factor information generation model to output the path safety factor information. Wherein, the route safety factor information generation model may be a model for generating safety factor information. In practice, the aforementioned path safety factor information generation model may be a Long Short-Term Memory (LSTM, Long Short-Term Memory) model.

The above-mentioned "first step-seventh step" as an invention point solves the technical problem two "the generation of the path length information and the corresponding path safety factor information is not accurate enough, which leads to the inaccuracy of the optimal torsion route screened subsequently". Therefore, the present disclosure accurately generates "target information", "road information" and "path safety factor information" through the "target recognition model", "road information generation model" and "path safety factor information generation model". Thus, more accurate path length information and corresponding path safety factor information can be generated.

In some optional implementation manners of some embodiments, the foregoing preset conditions include: a torsion angle sub-condition and an orientation sub-condition. Wherein, the twist angle sub-condition may be that the twist angle is smaller than a predetermined angle. The above azimuth sub-condition may be that the azimuth transformation direction is a preset designated direction.

Optionally, the above-mentioned screening of candidate twist information satisfying preset conditions from the above at least one piece of candidate twist information as optimal twist information may include the following steps:

The candidate twist information satisfying the twist angle sub-condition and the orientation sub-condition is selected from the at least one piece of candidate twist information as optimal twist information.

In some optional implementations of some embodiments, after step 105, the steps further include the following steps:

In the first step, in response to receiving the item information set and the viewing order, sort the item positions in the item position set according to the viewing order to obtain an item position sequence.

The second step is to use the position of the above-mentioned vertical point as the starting point, and according to the item position sequence corresponding to the above-mentioned item position sequence, sequentially connect the position of the above-mentioned vertical point with the positions of each item in a straight line to obtain a candidate twisting route.

The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the information sending method of some embodiments of the present disclosure, at least one target detection item can be viewed quickly and efficiently through optimal twist information. Specifically, the inaccurate reason for viewing at least one object detection object is that the adjustment efficiency of artificially adjusting the panoramic display device is low, resulting in poor user experience. Based on this, the information sending method of some embodiments of the present disclosure first acquires a real-time panoramic video of a target area captured by a panoramic camera. Here, the obtained real-time panoramic video is used for subsequent detection of at least one object detection object, and is also used for subsequent generation of at least one candidate torsion information. Then, in response to detecting at least one object detection object in the real-time panoramic video, determine the real-time observation perspective image corresponding to the target real-time panoramic image in the real-time panoramic video and the current observation perspective corresponding to the current detection moment. Wherein, the real-time observation perspective image and the target real-time panoramic image are images at the same moment. Here, the obtained real-time panoramic image of the target and the above-mentioned real-time observation perspective image are used to subsequently generate at least one piece of candidate torsion information. Furthermore, according to the target real-time panoramic image and the real-time observation angle image, at least one piece of candidate twist information for the screen display device corresponding to the current observation angle can be accurately generated. Next, the candidate twist information that satisfies the preset condition is selected from the above at least one piece of candidate twist information, and used as the optimal twist information for subsequent twisting of related users. Finally, the above-mentioned optimal twist information is sent to the above-mentioned screen display device for twisting by relevant users. To sum up, through the generation of at least one piece of candidate twist information and the screening of candidate twist information, the optimal twist information is sent to the screen display device for relevant users to view the target detection item.

Further referring to FIG. 2 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an information sending device. These device embodiments correspond to those method embodiments shown in FIG. 1 , and the information sending device can be specifically applied to various electronic devices.

As shown in FIG. 2 , an information sending device 200 includes: an acquiring unit 201 , a determining unit 202 , a generating unit 203 , a screening unit 204 and a sending unit 205 . Wherein, the acquiring unit 201 is configured to acquire the real-time panoramic video of the target area captured by the panoramic camera; the determining unit 202 is configured to respond to the detection of at least one target detection item in the above-mentioned real-time panoramic video, and determine the real-time observation perspective image corresponding to the target real-time panoramic image in the above-mentioned real-time panoramic video and the current observation angle corresponding to the current detection moment, wherein the above-mentioned real-time observation perspective image and the above-mentioned real-time panoramic image of the target are images at the same time; the generation unit 203 is configured. The current observation angle of view corresponds to at least one piece of candidate twist information of the screen display device; the screening unit 204 is configured to filter the candidate twist information satisfying a preset condition from the above at least one piece of candidate twist information as optimal twist information; the sending unit 205 is configured to send the above optimal twist information to the above screen display device.

It can be understood that the units recorded in the information sending device 200 correspond to the steps in the method described with reference to FIG. 1 . Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the information sending device 200 and the units contained therein, and will not be repeated here.

Referring now to FIG. 3 , it shows a schematic structural diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. The electronic device shown in FIG. 3 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 3 , the electronic device 300 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 301, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the electronic device 300 are also stored. The processing device 301 , ROM 302 and RAM 303 are connected to each other through a bus 304 . An input/output (I/O) interface 305 is also connected to the bus 304 .

Generally, the following devices may be connected to the I/O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication means 309 may allow the electronic device 300 to perform wireless or wired communication with other devices to exchange data. While FIG. 3 shows electronic device 300 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided. Each block shown in FIG. 3 may represent one device, or may represent multiple devices as required.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In some such embodiments, the computer program may be downloaded and installed from a network via communication means 309 , or from storage means 308 , or from ROM 302 . When the computer program is executed by the processing device 301, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.

It should be noted that the above-mentioned computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code thereon. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server can communicate using any currently known or future-developed network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (for example, a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the real-time panoramic video of the target area captured by the panoramic camera; in response to detecting at least one target detection item in the above-mentioned real-time panoramic video, determines the real-time observation angle image corresponding to the target real-time panoramic image in the above-mentioned real-time panoramic video and the current observation angle corresponding to the current detection moment, wherein the above-mentioned real-time observation angle image and the above-mentioned target real-time panoramic image are images at the same time; For at least one piece of candidate twist information of the screen display device corresponding to the current observation angle of view; screening candidate twist information satisfying preset conditions from the above at least one piece of candidate twist information as optimal twist information; sending the above optimal twist information to the above screen display device.

Computer program code for carrying out operations of some embodiments of the present disclosure may be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional procedural programming languages—such as the “C” language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. Where a remote computer is involved, the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (such as through the Internet using an Internet service provider).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more executable instructions for implementing specified logical functions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or by combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be realized by software or by hardware. The described units may also be set in a processor, for example, it may be described as: a processor includes an acquiring unit, a determining unit, a generating unit, a screening unit and a sending unit. Wherein, the names of these units do not constitute a limitation to the unit itself under certain circumstances, for example, the acquisition unit may also be described as “a unit that acquires real-time panoramic video of the target area captured by the panoramic camera”.

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

The above descriptions are only some preferred embodiments of the present disclosure and illustrations of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also covers other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with technical features having similar functions disclosed in (but not limited to) the embodiments of the present disclosure.

Claims (7)

1. An information transmission method, comprising:

acquiring a real-time panoramic video shot by a panoramic camera and aiming at a target area;

responding to detection of at least one target detection object in the real-time panoramic video, and determining a target real-time panoramic image in the real-time panoramic video corresponding to the current detection moment and a real-time observation visual angle image corresponding to the current observation visual angle, wherein the real-time observation visual angle image and the target real-time panoramic image are images at the same moment;

according to the number of cameras included in the panoramic camera, image splitting is carried out on the target real-time panoramic image to obtain a split image set, wherein the number of split images included in the split image set is the same as the number of the cameras, and candidate torsion information in at least one piece of candidate torsion information is a candidate torsion route;

carrying out tiling and splicing treatment on the split image set in a two-dimensional tiling and splicing mode to obtain a tiling image;

determining image area information with the same image content as that of the real-time observation visual angle image from the tiled image, and marking at least one object position corresponding to the at least one target detection object in the tiled image;

Acquiring position relation information between the picture display equipment and a real-time observation user;

determining a corresponding vertical point of the real-time observation visual angle image according to the image area information and the position relation information;

generating a list of item information for the at least one item location, wherein the item information comprises: item category, item identification;

transmitting the item information list to the picture display equipment so that the real-time observation user can select to watch item information from the item information list;

in response to receiving the selected item information set and not receiving a viewing order, determining a set of item positions corresponding to the item information set;

inputting the set of item locations and the location of the vertical point to a route generation model to output the at least one candidate twisted route;

in response to receiving the item information set and the viewing order, ordering each item position in the item position set according to the viewing order to obtain an item position sequence;

taking the position of the vertical point as a starting point, and sequentially connecting the position of the vertical point with each article position in a straight line according to the article position sequence corresponding to the article position sequence to obtain a candidate twisting route;

Screening candidate torsion information meeting preset conditions from the at least one piece of candidate torsion information to serve as optimal torsion information;

and sending the optimal torsion information to the picture display equipment.

2. The method of claim 1, wherein the preset condition comprises: path length sub-conditions and path security sub-conditions; and

the screening candidate torsion information meeting a preset condition from the at least one piece of candidate torsion information as optimal torsion information includes:

acquiring user environment information corresponding to the real-time observation user;

determining path length information and corresponding path safety coefficient information corresponding to each candidate twisted route in the at least one candidate twisted route according to the user environment information;

and screening the candidate twisted routes meeting the path length sub-conditions and the path safety sub-conditions from the at least one candidate twisted route according to the path length information corresponding to each candidate twisted route and the corresponding path safety coefficient information, and taking the candidate twisted routes as optimal twisted routes.

3. The method of claim 1, wherein the candidate twist information in the at least one piece of candidate twist information is candidate twist azimuth information; and

The generating at least one piece of candidate torsion information for the picture display device corresponding to the current observation view angle according to the target real-time panoramic image and the real-time observation view angle image comprises the following steps:

establishing a three-dimensional model aiming at the target real-time panoramic image;

generating a three-dimensional sub-model for the real-time observation view angle image, wherein the three-dimensional sub-model is a sub-model of the three-dimensional model;

determining three-dimensional coordinate points of each target detection object in the three-dimensional model to obtain at least one three-dimensional coordinate point;

in response to receiving the selected object information set and not receiving the viewing sequence, respectively converting a sub-model preset three-dimensional center coordinate point corresponding to the three-dimensional sub-model, a model preset three-dimensional center coordinate point corresponding to the three-dimensional sub-model and a three-dimensional coordinate point set corresponding to the object information set in the at least one three-dimensional coordinate point into a preset three-dimensional data format to obtain first three-dimensional center point data, second three-dimensional center point data and a three-dimensional data set;

inputting the first three-dimensional center point data, the second three-dimensional center point data and the three-dimensional data set into a torsion azimuth generating model to output a torsion azimuth information set as the candidate torsion azimuth information set;

And removing the candidate torsion azimuth information corresponding to the total torsion angle larger than a preset angle value from the candidate torsion azimuth information set to obtain at least one piece of candidate torsion azimuth information.

4. A method according to claim 3, wherein the preset conditions comprise: a twist angle sub-condition and an azimuth sub-condition; and

the screening candidate torsion information meeting a preset condition from the at least one piece of candidate torsion information as optimal torsion information includes:

and screening out candidate torsion information meeting the torsion angle sub-condition and the azimuth sub-condition from the at least one piece of candidate torsion information to serve as optimal torsion information.

5. An information transmitting apparatus comprising:

an acquisition unit configured to acquire a real-time panoramic video for a target area captured by a panoramic camera;

a determining unit configured to determine a target real-time panoramic image in the real-time panoramic video and a real-time observation view angle image corresponding to a current observation view angle corresponding to a current detection time in response to detection of at least one target detection object in the real-time panoramic video, wherein the real-time observation view angle image and the target real-time panoramic image are images at the same time;

The generation unit is configured to split the target real-time panoramic image according to the number of cameras included in the panoramic camera to obtain a split image set, wherein the number of split images included in the split image set is the same as the number of cameras, and candidate torsion information in at least one piece of candidate torsion information is a candidate torsion route; carrying out tiling and splicing treatment on the split image set in a two-dimensional tiling and splicing mode to obtain a tiling image; determining image area information with the same image content as that of the real-time observation visual angle image from the tiled image, and marking at least one object position corresponding to the at least one target detection object in the tiled image; acquiring position relation information between the picture display equipment and a real-time observation user; determining a corresponding vertical point of the real-time observation visual angle image according to the image area information and the position relation information; generating a list of item information for the at least one item location, wherein the item information comprises: item category, item identification; transmitting the item information list to the picture display equipment so that the real-time observation user can select to watch item information from the item information list; in response to receiving the selected item information set and not receiving a viewing order, determining a set of item positions corresponding to the item information set; inputting the set of item locations and the location of the vertical point to a route generation model to output the at least one candidate twisted route; in response to receiving the item information set and the viewing order, ordering each item position in the item position set according to the viewing order to obtain an item position sequence; taking the position of the vertical point as a starting point, and sequentially connecting the position of the vertical point with each article position in a straight line according to the article position sequence corresponding to the article position sequence to obtain a candidate twisting route;

A screening unit configured to screen candidate torsion information satisfying a preset condition from the at least one piece of candidate torsion information as optimal torsion information;

and a transmitting unit configured to transmit the optimal torsion information to the screen display device.

6. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.

7. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-4.