WO2019210667A1 - Screen image transmission method, device, server and storage medium - Google Patents

Abstract

Disclosed is a browser-based screen image transmission method. Said method comprises the steps of: capturing a display image at the current moment; transmitting data of the captured display image to a local browser; transmitting, on the basis of the local browser, the data of the display image to a server, the data of the display image being used for triggering the server to send, by means of a media channel, said data to a receiving end device having subscribed said data, and for triggering the receiving end device having received the data of the display image to render and display, by means of the browser, the data of the display image, the media channel being a communication link based on the UDP protocol of WebRTC. The embodiments of the present invention further provide a screen image transmission device, a storage medium and a system, being able to effectively improve the instantaneity and fluency of screen image transmission, and improving the upper limit of participants supportable in screen image transmission.

Description

Screen screen transmission method, device, server, system and storage medium Technical field

The present invention relates to the field of screen sharing, and in particular, to a screen picture transmission method, device, server, system, and storage medium.

Background technique

Screen sharing is of high application value in modern work and has a wide range of applications in modern teaching. For example, in the participation of the same event, such as a meeting or a class, a participant wants to share the screen screen that the source device is playing to other participants. For example, the speaker at the meeting will share the content to the audience. The player's device is played on the device, or the teacher shares the lecture content on the student's device for display, so that multiple participants of the event can see the same display content on their device.

In the implementation of screen sharing, screen transmission is the core technology. In the prior art, in order to realize screen screen transmission, the transmitting end often connects to the receiving end through the socket and transmits the screen image data, and the receiving end receives the screen screen data through the socket and displays it, thereby sharing the display content of the transmitting end. To the receiving end. In a specific implementation, one of the methods is that the transmitting end and the receiving end are directly connected through the socket, and the other method is that the sending end is connected to the external forwarding unit through the socket, and the transmitting end is received by the forwarding unit through the socket. The screen data sent is sent to the receiving end.

In the process of implementing the present invention, the inventors have found that in the existing screen picture transmission technology, since the underlying layer of the socket is based on the TCP protocol and is limited by the congestion control and retransmission strategy of the TCP protocol, the redundant information transmitted will gradually accumulate. The delay between the playback content of each receiving end and the playing content of the transmitting end is increased, the information transmission efficiency is low, affecting the real-time and smoothness of sharing, and the number of participants in the same network is greatly restricted.

Summary of the invention

The object of the embodiments of the present invention is to provide a screen picture transmission method, device, server, system, and storage medium, which can effectively improve the real-time and smoothness of screen picture transmission, and improve the upper limit of participants that can be supported in screen picture transmission.

To achieve the above objective, an embodiment of the present invention provides a screen picture transmission method, including:

Intercepting the display screen at the current time;

Transmitting the intercepted data of the display screen to a local browser;

Sending, according to the local browser, the data of the display screen to a server; the data of the display screen is used to trigger the server to send it to the subscribed receiving end device through the media channel, and trigger the receiving of the The receiving end device that displays the data of the screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of the UDP protocol based on WebRTC.

As an improvement of the above scheme, it also includes:

Sending a subscription instruction to the server; the subscription instruction is used to trigger the server to forward the subscription instruction to at least one receiving end device, and set a subscription status of the receiving end device that feeds the subscription request to be subscribed.

Compared with the prior art, a browser-based screen image transmission method provided by the embodiment of the present invention intercepts a display screen of a current time of the device, and sends the intercepted data of the display screen through a browser. The server is then sent to the receiving device through the media channel of the WebRTC-based UDP protocol for display, thereby realizing the transmission of the screen image. The transmission of the media channel based on the WebRTC-based UDP protocol solves the problem that the transmission efficiency of the prior art is low due to the congestion control and retransmission strategy of the TCP protocol, and the real-time and smoothness of the sharing is not high and can be supported. The technical problem with lower participant limit effectively improves the real-time and smoothness of screen transmission, and increases the upper limit of participants that can be supported in screen transmission.

The embodiment of the invention further provides another screen picture transmission method, including the steps:

Receiving, by the first media channel, data of a display screen sent by the sending end device through the browser; wherein the first media channel is a communication link of a UDP protocol based on WebRTC, and data of the display screen is used by the sending end The device intercepts the current display screen;

Obtain the currently subscribed receiving device;

Receiving, by the second media channel, the received data of the display screen to the currently subscribed receiving end device; wherein the second media channel is a communication link of a WebRTC-based UDP protocol.

As an improvement of the above scheme, it also includes:

Receiving a subscription request from at least one receiving device;

The subscription status of the receiving device that issued the subscription request is set to subscribed.

As an improvement of the foregoing solution, the communication link with the transmitting device further includes a first signaling channel of a TCP protocol-based TCP protocol; and before the receiving the subscription request from the receiving device, :

Receiving, by the sending end device, a subscription instruction sent by the first signaling channel; the subscription instruction includes address information of the local device;

Forwarding the received subscription instruction to the receiving device to be subscribed; wherein the subscription instruction is used to trigger the receiving device to initiate a subscription request to the local device according to the address information in the subscription instruction.

As an improvement of the foregoing solution, the subscription instruction further includes address information of the at least one receiving end device; the receiving end device to be subscribed to is the receiving end device corresponding to the address information of the receiving end device.

As an improvement of the foregoing solution, the communication link with the receiving end device further includes a second signaling channel of a TCP protocol-based TCP protocol; and the second signaling channel is used to the receiving end to be subscribed The device forwards the received subscription instruction and receives a subscription signal sent by the receiving device in response to the subscription instruction.

Compared with the prior art, a browser-based screen image transmission method provided by the embodiment of the present invention receives data of a display screen sent by a transmitting end device through a first media channel, and passes the data of the display screen through The two media channels are forwarded to the receiving device, thereby implementing transmission of the screen of the transmitting device. The first media channel and the second media channel are both communication links of the UDP protocol of the WebRTC, which solves the problem that the transmission efficiency of the prior art is low due to congestion control and retransmission policies of the TCP protocol. The technical problem of low real-time and fluency of sharing and low supportable upper limit of participants can effectively improve the real-time and smoothness of screen transmission and increase the upper limit of participants that can be supported in screen transmission.

The embodiment of the invention further provides a screen image transmission device, comprising:

a screen intercepting module, configured to intercept a display screen at a current time;

a picture transmission module, configured to transmit the intercepted data of the display screen to a local browser;

a screen forwarding module, based on the local browser, sending data of the display screen to a server; the data of the display screen is used to trigger the server to send it to the subscribed receiving device through the media channel, and trigger The receiving end device receiving the data of the display screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of a UDP protocol based on WebRTC.

An embodiment of the present invention further provides another screen picture transmission apparatus, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor executing the computer program The first screen transmission method as described above is implemented.

An embodiment of the present invention further provides a screen transfer server, including a first sub-server, where the first sub-server includes:

a receiving module, configured to receive data of a display screen sent by the sending end device through the browser through the first media channel, where the first media channel is a communication link of a UDP protocol based on WebRTC, and the data of the display screen Obtaining, by the sending end device, the current display screen;

a detecting module, configured to obtain a receiving device that is currently subscribed to;

And a forwarding module, configured to forward the received data of the display screen to the currently subscribed receiving end device by using the second media channel, where the second media channel is a communication link of a WebRTC-based UDP protocol.

As an improvement of the foregoing solution, the second sub-server is further included, and the second sub-server includes:

An instruction receiving module, configured to receive a subscription instruction sent by the sending end device; the subscription instruction includes address information of the first sub-server;

And an instruction forwarding module, configured to forward the received subscription instruction to the receiving end device to be subscribed, and receive a subscription signal sent by the receiving end device in response to the subscription instruction.

As an improvement of the above solution, the first sub-server and the second sub-server are provided on the same computer device or on different computer devices.

Another embodiment of the present invention provides a screen transfer server including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor executing the computer program The second screen transmission method as described above is implemented.

The embodiment of the present invention further provides a computer readable storage medium, comprising: a stored computer program, wherein, when the computer program is running, controlling a device where the computer readable storage medium is located performs any of the above A screen transmission method as described.

The embodiment of the present invention further provides a screen picture transmission system, comprising the screen picture transmission device according to any one of the preceding claims, and a plurality of receiving end devices, further comprising the screen picture transmission server according to any one of the preceding claims;

The screen picture transmission device is connected to the screen picture transmission server through a network as a source device; the receiving device is connected to the screen picture transmission server through a network.

Compared with the prior art, the screen picture transmission system provided by the embodiment of the present invention intercepts the display screen of the current time of the screen picture transmission device, and transmits the data of the display screen to the screen screen transmission through the first media channel. And transmitting, by the screen transmission server, the second screen channel to the receiving end device, thereby implementing transmission of the screen image of the screen image transmission device. The first media channel and the second media channel are both communication links of the UDP protocol of the WebRTC, which solves the problem that the transmission efficiency of the prior art is low due to congestion control and retransmission policies of the TCP protocol. The technical problem of low real-time and fluency of sharing and low supportable upper limit of participants can effectively improve the real-time and smoothness of screen transmission and increase the upper limit of participants that can be supported in screen transmission.

DRAWINGS

FIG. 1 is a schematic structural diagram of a screen picture transmission system according to an embodiment of the present invention.

FIG. 2 is a schematic flowchart of a screen picture transmission method according to an embodiment of the present invention.

FIG. 3 is a schematic flowchart diagram of another screen picture transmission method according to an embodiment of the present invention.

4 is a flow chart showing an alternative flow of the screen transfer method shown in FIG.

FIG. 5 is a schematic flowchart of a screen picture transmission method according to an embodiment of the present invention.

FIG. 6 is a schematic structural diagram of a screen picture transmission apparatus according to an embodiment of the present invention.

FIG. 7 is a schematic structural diagram of a screen picture transmission server according to an embodiment of the present invention.

FIG. 8 is a schematic structural diagram of a screen picture transmission apparatus according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In an embodiment of the present invention, the screen transmission system includes at least one transmitting device, at least one server, and a plurality of receiving devices; wherein the transmitting device and the receiving device have at least a display function and are installed with browsing Program related to the function.

In the screen transmission system, the transmitting device is connected to the server through a network, and the receiving device is connected to the server through a network.

1 is a schematic structural diagram of a screen image transmission system according to an embodiment of the present invention. By way of example, the screen transmission system 100 includes a sender device 110, a server 120, and three receiver devices 130, wherein Both the end device 110 and the sink device 130 are connected to the server 120 via a network. It can be understood that the structure shown in the screen transmission system 100 is only an example. In other cases, the number of devices in the screen transmission system may be adjusted, for example, two or four receiving devices may be included, or It is a larger number of sender devices, and can also be a larger number of servers, without affecting the beneficial effects that can be achieved by the present invention.

The embodiment of the invention provides a screen picture transmission method as shown in FIG. 2, and the screen picture transmission method can be performed by a sender device of the screen picture transmission system. For example, in a multi-person conference, when a participant has a device such as a tablet or a smart phone, any participant's tablet or smart phone can be used as the sender device, while other participants The network display device can be used as the receiving device; for example, in the teaching class, the teacher's tablet or smart phone can be used as the sending device, and the student's tablet or smart phone can be used as the receiving device.

It can be understood that the types of devices that can be used as the transmitting device include, but are not limited to, a tablet computer or a smart phone, and can also be a display computer having a display function and a related program with a browser function, such as a notebook computer, a desktop computer, or a learning machine. Electronic equipment.

The screen picture transmission method provided by the embodiment of the present invention includes the following steps S110 to S130.

S110: Intercepting a display screen of the current time.

The sending end device may intercept the display screen of the device itself at the current time, thereby obtaining data of the display screen. Optionally, the display screen may be all content being displayed by the display module of the sending end device, or may be part of content being displayed; the intercepting may be data extraction to obtain data of the display screen. It is also possible to obtain the data of the display screen by saving the display screen without affecting the advantageous effects obtained by the present invention.

S120. Transmit the intercepted data of the display screen to a local browser.

Optionally, the sending end device may transmit data of the display screen to the local browser by using an internal data channel of the device itself; and performing intercepting of the display screen by the local browser. In this case, the data of the display screen intercepted by the local browser may be transmitted and transmitted within the program.

S130. Send, according to the local browser, data of the display screen to a server, where the data of the display screen is used to trigger the server to send the data to the subscribed receiving end device through the media channel, and trigger the receiving. The receiving end device of the data of the display screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of a UDP protocol based on WebRTC.

Optionally, the subscription status of all or part of the receiving end device may be set to be subscribed by sending a subscription instruction to the server in advance; wherein the subscription instruction is used to trigger when received by the server The server forwards a subscription instruction to the at least one receiving end device; the subscription instruction forwarded by the server is configured to trigger the receiving end device to feed back a subscription request to the server when received by the receiving end device, the server The subscription status of the receiving device that feeds back the subscription request is set to subscribed. It can be understood that the timing of sending the subscription instruction to the server may be performed before step S110 or before step S120 or step S130, without affecting the beneficial effects obtained by the present invention.

The media channel connected between the sender device and the server is pre-built based on the UDP protocol of WebRTC. Specifically, the sending end device may establish a communication link to connect to the server according to a WebRTC UDP protocol by accessing an IP address of the server on the local browser, and perform subsequent steps via the communication link. The data of the display screen intercepted is sent to the server. Since the media channel is built based on the WebRTC UDP channel, and the function of the device is implemented based on a local browser, it is not necessary to develop different client programs for different operating systems, and the screen is improved while reducing the development difficulty. The compatibility and stability of the picture transmission method in different application environments.

The browser-based screen image transmission method provided by the embodiment of the present invention, the interception of the display screen of the current time of the device is performed, and the intercepted data of the display screen is sent to the server by the browser, and then The server sends the media channel based on the WebRTC-based UDP protocol to the receiving device for display, thereby realizing the transmission of the screen image. The transmission of the media channel based on the WebRTC-based UDP protocol solves the problem that the transmission efficiency of the prior art is low due to the congestion control and retransmission strategy of the TCP protocol, and the real-time and smoothness of the sharing is not high and can be supported. The technical problem with lower participant limit effectively improves the real-time and smoothness of screen transmission, and increases the upper limit of participants that can be supported in screen transmission.

The embodiment of the present invention further provides a browser-based screen image transmission method as shown in FIG. 3, which may be executed by a server of the screen image transmission system, and includes steps S210 to S230 as described below.

S210: Receive, by using a first media channel, data of a display screen sent by a sending device by using a browser, where the first media channel is a communication link of a WebRTC-based UDP protocol, and the data of the display screen is The sending device intercepts the current display screen.

The server is connected to the sending end device by using the first media channel. Specifically, the server may receive a network access request of the sending end device in advance and respond, so that the UDP protocol based on the WebRTC is constructed and The communication link between the sender devices is the first media channel.

S220. Obtain a currently-received receiving device.

Optionally, the server may directly obtain the subscribed receiving device by performing the step of performing the step, or the server may perform the steps in FIG. 4 by using the step. The flow shown in step S221 to step S224 or the flow as shown in step S223 to step S224 is performed to instantly update the subscription status of the receiving device to provide flexibility in adjusting the subscription status of the receiving device. It can be understood that the above two control modes for the subscription state of the receiving device are only used as an example, and the method for controlling the subscription state of the receiving device in the above manner does not affect the present invention. The beneficial effects achieved.

Referring to FIG. 4, steps S221 to S224 are as follows:

S221. Receive a subscription instruction sent by the sending end device by using the first signaling channel. The subscription instruction includes address information of the local device.

Preferably, the subscription instruction may further include address information of the at least one receiving end device, and the receiving end device corresponding to the address information of the receiving end device is used as the receiving end device to be subscribed.

The address information of the device may be an IP address of the server or other information including an address of the server in the network. The first signaling channel is a pre-built WebSocket-based TCP communication link, and the communication link connects the server and the sending end device, so that signaling between the server and the sending end device The stream and the media stream are transmitted through different communication links, which avoids the mutual influence of the transmission of the media stream and the signaling stream, and improves the bandwidth utilization and communication efficiency.

S222. Forward the received subscription instruction to the receiving device to be subscribed. The subscription instruction is used to trigger the receiving device to initiate a subscription request to the local device according to the address information in the subscription instruction.

Preferably, the server may be that the receiving end device forwards the received subscription instruction through the second signaling channel. The second signaling channel is a pre-built WebSocket-based TCP communication link, and the communication link connects the server and the receiving end device to enable signaling flow between the server and the receiving end device. The media stream is transmitted through different communication links, which avoids the interaction between the media stream and the signaling stream, and improves bandwidth utilization and communication efficiency.

S223. Receive a subscription request from at least one receiving device.

The subscription request may be request information for constructing a media channel that is sent by the receiving device to the server, or request information for subscribing to the server, or may be received by the receiving device. The request information sent to the server under the trigger of the subscription instruction.

Optionally, the server may be configured to receive the subscription request that is sent by the receiving end device, that is, directly perform step S223 without performing step S221 and step S222; or may receive the receiving device to send under a specific trigger condition. If the server performs step S221 and step S222, the receiving end device is triggered to send a subscription request, and then the content of step S223 is executed, thereby improving the controllability of the subscription status of the receiving end device and the subscription status. The convenience of regulation. It can be understood that regardless of whether the server receives the subscription request sent by the receiving device by using the above manner, the beneficial effects that can be achieved by the present invention are not affected.

S224. Set a subscription status of the receiving device that sends the subscription request to be subscribed.

S230. Forward, by the second media channel, the received data of the display screen to the currently subscribed receiving end device, where the second media channel is a communication link of a WebRTC-based UDP protocol.

The second media channel may be constructed when the receiving device accesses the server, or may be constructed when the server receives the subscription request sent by the receiving device, without affecting The beneficial effects that can be achieved by the present invention.

The screen picture transmission method provided by the embodiment of the invention receives the data of the display screen sent by the sending end device through the first media channel, and forwards the data of the display screen to the receiving end device through the second media channel, thereby realizing The transmission of the screen of the transmitting device. The first media channel and the second media channel are both communication links of the UDP protocol of the WebRTC, which solves the problem that the transmission efficiency of the prior art is low due to congestion control and retransmission policies of the TCP protocol. The technical problem of low real-time and fluency of sharing and low supportable upper limit of participants can effectively improve the real-time and smoothness of screen transmission and increase the upper limit of participants that can be supported in screen transmission.

The embodiment of the present invention further provides a screen image transmission method as shown in FIG. 5, which may be executed by a screen screen transmission system, and includes the following steps S310 to S370:

S310. The transmitting device intercepts the display screen at the current time.

S320. The sending end device transmits the intercepted data of the display screen to a local browser.

S330. The sending end device sends, according to the local browser, data of the display screen to a server by using a first media channel, where the first media channel is a communication link of a UDP protocol based on WebRTC.

S340. The server receives data of the display screen by using the first media channel.

S350. The server acquires a currently subscribed receiving end device.

S360. The server forwards the received data of the display screen to a currently subscribed receiving end device by using a second media channel, where the second media channel is a communication link of a WebRTC-based UDP protocol.

S370. The receiving end device receives data of the display screen from the second media channel, and performs rendering display on data of the display screen.

Taking the screen transmission system shown in FIG. 1 as an example, the screen transmission system 100 includes a transmitting device 110, a server 120, a receiving device 130A, a receiving device 130B, and a receiving device 130C. The source device 110 is connected to the sink devices 130A, 130B, and 130C through the server 120. The transmitting device 110 intercepts the display screen A of its current time to obtain the data of the display screen A. The source device 110 transmits the data of the display screen A to the local browser, and then transmits the data of the display screen A to the server 120 based on the local browser. The data is triggered by the server 120 to perform the forwarding operation, that is, the data of the display screen A is forwarded to the receiving device.

It is assumed that before the server 120 forwards the data of the display screen A, the subscription instruction B sent by the source device 110 is also received, and the server 120 forwards the subscription instruction B to the receiving device 130A and the receiving device 130B, and the receiving device After receiving the subscription instruction B forwarded by the server 120, the 130A feeds back the subscription request C to the server 120. After receiving the subscription request C, the server 120 sets the subscription status of the receiving device 130A to subscribed. When the server 120 forwards the data of the display screen A, since the subscription status of the receiving end device 130A is subscribed, the subscription status of the receiving end devices 130B and 130C is not set to subscribed, so the server 120 forwards the data of the display screen A. To the receiving end device 130A, the receiving end device 130A performs rendering display on the received data of the display screen A, so that the intercepted display screen of the transmitting end device 110 is shared with the receiving end device 130A.

The screen picture transmission method provided by the embodiment of the present invention, the current display screen is intercepted by the sending end device, and the data of the display screen is sent to the server through the first media channel, and the data of the display screen is passed by the server. The second media channel is forwarded to the receiving device, thereby implementing transmission of the screen of the transmitting device. The first media channel and the second media channel are both communication links of the UDP protocol of the WebRTC, which solves the problem that the transmission efficiency of the prior art is low due to congestion control and retransmission policies of the TCP protocol. The technical problem of low real-time and fluent sharing and low supportable participants has effectively improved the real-time and smoothness of screen transmission and increased the upper limit of participants that can be supported in screen transmission.

The embodiment of the invention further provides a screen picture transmission device. Referring to FIG. 6, the screen picture transmission device 40 can be used as a transmitting end device of the screen picture transmission system, or can be used as a component of the transmitting end device. The screen picture transmission device 40 includes:

The screen capture module 41 is configured to intercept the display screen at the current time;

The screen transmission module 42 is configured to transmit the intercepted data of the display screen to a local browser;

The screen forwarding module 43 sends the data of the display screen to the server based on the local browser; the data of the display screen is used to trigger the server to send it to the subscribed receiving device through the media channel, and The receiving end device that triggers the data received by the display screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of a WebRTC-based UDP protocol.

The workflow of the screen transfer device 40 is the same as the screen transfer method shown in FIG. 2, and details are not described herein.

The screen picture transmission device provided by the embodiment of the present invention intercepts the display screen of the current time of the device, sends the intercepted data of the display screen to the server through the browser, and then passes the server through the WebRTC. The media channel of the UDP protocol is sent to the receiving device for display, thereby realizing the transmission of the screen image. The transmission of the media channel based on the WebRTC-based UDP protocol solves the problem that the transmission efficiency of the prior art is low due to the congestion control and retransmission strategy of the TCP protocol, and the real-time and smoothness of the sharing is not high and can be supported. The technical problem with lower participant limit effectively improves the real-time and smoothness of screen transmission, and increases the upper limit of participants that can be supported in screen transmission.

The embodiment of the present invention further provides a screen transfer server. As shown in FIG. 7, the screen transfer server 50 can serve as a server of the screen transfer system or as a component of the server. The screen transfer server 50 includes a first sub-server 51, and the first sub-server 51 includes a receiving module 51a, a detecting module 51b, and a forwarding module 51c.

The receiving module 51a is configured to receive data of a display screen sent by the sending end device through the browser through the first media channel, where the first media channel is a communication link of a UDP protocol based on WebRTC, and the display screen is The data is obtained by intercepting the current display screen by the transmitting device. The detecting module 51b is configured to obtain the receiving device that is currently subscribed. The forwarding module 51c is configured to forward, by using the second media channel, the received data of the display screen to the currently subscribed receiving end device, where the second media channel is a communication link of a WebRTC-based UDP protocol.

Preferably, the screen transfer server 50 may further include a second sub-server 52, and the second sub-server 52 includes an instruction receiving module 52a and an instruction forwarding module 52b.

The instruction receiving module 52a is configured to receive a subscription instruction sent by the sender device, where the subscription instruction includes address information of the first subserver. The instruction forwarding module 52b is configured to forward the received subscription instruction to the receiving end device to be subscribed, and receive a subscription signal sent by the receiving end device in response to the subscription instruction.

Optionally, the first sub-server 51 and the second sub-server 52 may be located on the same computer device; the first sub-server 51 and the second sub-server 52 may also be located on different computer devices and share the same IP address. In the event of a child server failure, another child server can remain in normal operation, providing stability to the system screen transfer server.

The working process of the screen transfer server 50 is the same as the screen transfer method shown in FIG. 3, and details are not described herein.

The browser-based screen image transmission server provided by the embodiment of the present invention receives the data of the display screen sent by the sending end device through the first media channel, and forwards the data of the display screen to the receiving end device through the second media channel. Thereby, the transmission of the screen of the transmitting device is realized. The first media channel and the second media channel are both communication links of the UDP protocol of the WebRTC, which solves the problem that the transmission efficiency of the prior art is low due to congestion control and retransmission policies of the TCP protocol. The technical problem of low real-time and fluency of sharing and low supportable upper limit of participants can effectively improve the real-time and smoothness of screen transmission and increase the upper limit of participants that can be supported in screen transmission.

The embodiment of the invention further provides a screen picture transmission device. As shown in FIG. 8 , the screen picture transmission device 60 can be used as a component of the sender device in the screen picture transmission system or as a sender device or as a screen image. A component or server of the server in the transmission system, the screen transfer device 60 includes a processor 61, a memory 62, and a computer program stored in the memory and operable on the processor, such as a screen transfer program . When the processor 61 executes the computer program, the steps in the above embodiments of the respective screen image transmission methods are implemented, for example, step S120 shown in FIG. Alternatively, when the processor executes the computer program, the functions of each module in the foregoing device embodiments are implemented, such as the screen screen transmission device or the screen screen transmission server described in the above embodiments.

Illustratively, the computer program can be partitioned into one or more modules that are stored in the memory 62 and executed by the processor 61 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program in the screen transmission device 60. For example, the computer program may be divided into a screen intercepting module, a screen transmission module, and a screen forwarding module, and the specific functions of each module are as follows: the screen intercepting module is configured to intercept a current time display screen; the screen transmission module Transmitting, by the screen forwarding module, the data of the display screen to the server according to the local browser; the data of the display screen is used by the screen forwarding module And the receiving end device that triggers the server to send the data to the subscribed receiving end device through the media channel, and triggers the receiving the data of the display screen to display and display the data of the display screen by using a browser; The media channel is a communication link of the WebRTC-based UDP protocol.

The computer program can also be divided into a receiving module, a detecting module and a forwarding module. The specific functions of each module are as follows: the receiving module is configured to receive, by using the first media channel, data of a display screen sent by the sending end device through the browser. The first media channel is a communication link of the UDP protocol based on the WebRTC, and the data of the display screen is obtained by the sending end device to intercept the current display image; the detecting module is configured to acquire the current The subscribed receiving device is configured to forward the received data of the display screen to the currently subscribed receiving end device through the second media channel; wherein the second media channel is a WebRTC-based The communication link of the UDP protocol.

The screen transmission device 60 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The screen transfer device 60 may include, but is not limited to, a processor 61 and a memory 62. It will be understood by those skilled in the art that the schematic diagram is merely an example of a screen transmission device, does not constitute a limitation of the screen transmission device 60, may include more or less components than those illustrated, or may combine certain components. Or different components, such as the screen transmission device 60, may also include input and output devices, network access devices, buses, and the like.

The processor 61 may be a central processing unit (CPU), or may be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 61 is a control center of the screen transmission device 60, and connects the entire screen transmission by using various interfaces and lines. Various parts of device 60.

The memory 62 can be used to store the computer program and/or module, by executing or executing computer programs and/or modules stored in the memory 62, and recalling data stored in the memory 62, Various functions of the screen transmission device 60 are implemented. The memory 62 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be Stores data created based on the use of the phone (such as audio data, phone book, etc.). In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), and a secure digital (SD). Card, flash card, at least one disk storage device, flash device, or other volatile solid state storage device.

Wherein, the module integrated by the screen transmission device 60 can be stored in a computer readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the present invention implements all or part of the processes in the foregoing embodiments, and may also be completed by a computer program to instruct related hardware. The computer program may be stored in a computer readable storage medium. The steps of the various method embodiments described above may be implemented when the program is executed by the processor. Wherein, the computer program comprises computer program code, which may be in the form of source code, object code form, executable file or some intermediate form. The computer readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM). , random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media.

The screen picture transmission apparatus provided by the embodiment of the present invention can obtain the beneficial effects of the screen picture transmission apparatus shown in FIG. 6 when it is used as a component of the transmission end device of the screen picture transmission system or as a transmission end device; When the components of the server of the screen transmission system or the server are used as the server, the beneficial effects of the screen transmission server shown in FIG. 7 can be obtained, and details are not described herein.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It is the scope of protection of the present invention.

Claims (15)

A screen picture transmission method, comprising: Intercepting the display screen at the current time; Transmitting the intercepted data of the display screen to a local browser; Sending, according to the local browser, the data of the display screen to a server; the data of the display screen is used to trigger the server to send it to the subscribed receiving end device through the media channel, and trigger the receiving of the The receiving end device that displays the data of the screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of the UDP protocol based on WebRTC. The method for transmitting a screen image according to claim 1, further comprising: Sending a subscription instruction to the server; the subscription instruction is used to trigger the server to forward the subscription instruction to at least one receiving end device, and set a subscription status of the receiving end device that feeds the subscription request to be subscribed. A screen picture transmission method, comprising the steps of: Receiving, by the first media channel, data of a display screen sent by the sending end device through the browser; wherein the first media channel is a communication link of a UDP protocol based on WebRTC, and data of the display screen is used by the sending end The device intercepts the current display screen; Obtain the currently subscribed receiving device; Receiving, by the second media channel, the received data of the display screen to the currently subscribed receiving end device; wherein the second media channel is a communication link of a WebRTC-based UDP protocol. The method for transmitting a screen image according to claim 3, further comprising: Receiving a subscription request from at least one receiving device; The subscription status of the receiving device that issued the subscription request is set to subscribed. The screen transmission method according to claim 4, wherein the communication link with the transmitting device further comprises: a first signaling channel of a TCP protocol based on WebSocket; Before the subscription request of the receiving device, the method further includes: Receiving, by the sending end device, a subscription instruction sent by the first signaling channel; the subscription instruction includes address information of the local device; Forwarding the received subscription instruction to the receiving device to be subscribed; wherein the subscription instruction is used to trigger the receiving device to initiate a subscription request to the local device according to the address information in the subscription instruction. The screen transmission method of claim 5, wherein the subscription instruction further comprises address information of at least one receiving device; and the receiving device to be subscribed corresponds to address information of the receiving device Receiver device. The screen transmission method according to claim 5, wherein the communication link with the receiving device further comprises: a second signaling channel of a TCP protocol based on WebSocket; and the second signaling And the channel forwards the received subscription instruction to the receiving device to be subscribed, and receives a subscription signal sent by the receiving device in response to the subscription instruction. A screen picture transmission device, comprising: a screen intercepting module, configured to intercept a display screen at a current time; a picture transmission module, configured to transmit the intercepted data of the display screen to a local browser; a screen forwarding module, based on the local browser, sending data of the display screen to a server; the data of the display screen is used to trigger the server to send it to the subscribed receiving device through the media channel, and trigger The receiving end device receiving the data of the display screen performs rendering display on the data of the display screen by using a browser; wherein the media channel is a communication link of a UDP protocol based on WebRTC. A browser-based screen transfer device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the computer program as claimed The screen transmission method according to any one of claims 1 to 2. A browser-based screen transfer server, comprising: a first sub-server, the first sub-server comprising: a receiving module, configured to receive data of a display screen sent by the sending end device through the browser through the first media channel, where the first media channel is a communication link of a UDP protocol based on WebRTC, and the data of the display screen Obtaining, by the sending end device, the current display screen; a detecting module, configured to obtain a receiving device that is currently subscribed to; And a forwarding module, configured to forward the received data of the display screen to the currently subscribed receiving end device by using the second media channel, where the second media channel is a communication link of a WebRTC-based UDP protocol. The screen transfer server of claim 10, further comprising a second sub-server, the second sub-server comprising: An instruction receiving module, configured to receive a subscription instruction sent by the sending end device; the subscription instruction includes address information of the first sub-server; And an instruction forwarding module, configured to forward the received subscription instruction to the receiving end device to be subscribed, and receive a subscription signal sent by the receiving end device in response to the subscription instruction. The screen transfer server according to claim 11, wherein the first sub-server and the second sub-server are provided on the same computer device or on different computer devices. A browser-based screen transfer server includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the computer program as claimed The screen transmission method according to any one of 3 to 7. A computer readable storage medium, comprising: a stored computer program, wherein the computer readable storage medium is controlled to execute the apparatus as claimed in claim 1 The screen picture transmission method according to any one of the items 7. A browser-based screen transmission system, comprising the screen transmission device according to any one of claims 8 to 9, a plurality of receiving end devices, and the apparatus according to any one of claims 10 to 13 Screen transfer server; The screen picture transmission device is connected to the screen picture transmission server through a network as a source device; the receiving device is connected to the screen picture transmission server through a network.

Images