CN103414751B - A kind of PC screen content sharing/interaction control method - Google Patents

Abstract

The present invention relates to the technical field of a PC screen content sharing interactive control method, and more particularly, relates to the improvement of the existing PC screen content sharing method. A PC screen content sharing interactive control method, comprising the following steps, S1. The remote PC host designates a window for content sharing, S2. obtains the screen RGB data in the shared memory data buffer, and S3. utilizes a software encoder to convert the YUV420 format The data is coded and compressed to become an H.264 video stream; S4. Upload the encoded and compressed screen content video stream to the streaming media server S5 through the RTMP protocol. The user logs in to the webpage client address S6 that publishes the media stream of the remote PC host through the browser. The control information is transmitted to the remote PC host through the TCP protocol; S7. The remote PC host analyzes the received control information of the client and presents it on the PC screen; S8. Presents the effect of the screen response on the webpage client logged in by the user. This enables the user to directly browse the screen content of the host through the web browser. Customize the remote interactive control protocol to realize a lightweight PC screen content sharing interaction.

Description

A PC screen content sharing interactive control method

technical field

The present invention relates to the technical field of a PC screen content sharing interactive control method, and more particularly, relates to the improvement of the existing PC screen content sharing method.

Background technique

Internet technology has become increasingly mature. In order to meet the current needs of screen content sharing, content display, and interactive communication among multiple Internet users, a lightweight screen content sharing system will well popularize multimedia interactive applications among ordinary users.

The multimedia teaching system is usually built with a specific server and client. When implementing multimedia teaching, it is necessary to install the corresponding client software on all client PCs first, which has poor operability and applicability. Moreover, most multimedia teaching systems lack interaction, and sometimes they only accept the teacher's lectures one-way on the screen, which does not play a very good role in the teaching of some courses that require operability. Therefore, it is of practical significance to design a light-weight teaching platform based on PC screen sharing and interaction from the perspective of rapid deployment of the teaching system and low-cost realization of simple teaching interaction.

The current video conferencing system usually uses the SIP protocol for session management, and the media stream is transmitted using the RTP protocol. The system needs to install an independent system and client. Compared with operating directly on the browser, the operation is complicated and the applicability is relatively poor. In addition, the SIP protocol is relatively complex, and simple applications do not require complex protocols, and it is inconvenient to expand for specific services. In addition, for the needs of lightweight applications, it is more practical to customize a simple interactive control protocol.

Contents of the invention

In order to overcome at least one of the defects described in the above-mentioned prior art, the present invention provides a PC screen content sharing interactive control method, which uses video encoding and compression technology to encode the screen content (RGB image data) into H.264, and packs it It is uploaded to an open source streaming media server in FLV format, making it universal and applicable to ordinary PCs, with low cost and resource overhead. And use a common streaming media transmission protocol RTMP to send the screen content encoded into video stream on the streaming media server to each user on the Internet, so that the user can directly browse the screen content of the host computer through the web browser. Customize the remote interactive control protocol to realize a lightweight system for sharing and interacting with PC screen content. Multiple users can use this lightweight system to carry out video conferences on the Internet to display the contents of the computer screen of the host computer, and can be used in interactive education scenarios for users to realize interactive and intuitive learning.

In order to solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a PC screen content sharing interactive control method, including the following steps, wherein steps S1 to S4 are the content sharing steps of the PC screen, and steps S5 to S8 are the PC screen interactive control step.

S1. The remote PC host specifies the window that needs to be shared, such as the name of the PPT window and the name of the picture window. The screen capture program will intercept the content of the specified window, and the obtained screen data will be stored in the shared memory data buffer in RGB format;

S2. Get the screen RGB data in the shared memory data buffer, convert it to YUV420 data, and compress the data volume to meet the needs of video encoding and real-time transmission;

S3. Utilize a software encoder to encode and compress YUV420 format data into an H.264 video stream;

S4. Upload the encoded and compressed screen content video stream to the streaming media server through the RTMP protocol, and the streaming media server publishes it to the network, and sends the media stream with the RTMP protocol according to the user's request;

S5. The user logs in to the address of the webpage client publishing the media stream of the remote PC host through the browser, and the webpage client captures the mouse or keyboard operation actions of the user end;

S6. According to the self-defined remote control protocol, transmit the control information to the remote PC host through the TCP protocol;

S7. The remote PC host analyzes the received control information of the client, performs corresponding control operations, and presents them on the PC screen;

S8. Present the effect of the screen response on the webpage client logged in by the user.

Further, in the step S1, after the name of the specified windows window is obtained, the API of the windows system is called to take a screenshot of the specified windows window, and the screenshot is automatically performed according to the acquired length and width parameters of the actual window.

Further, in the step S3, it is necessary to ensure that the code rate cannot consume too much bandwidth, so that it can smoothly browse the screen content of the host computer even in a poor network environment, and both the screen capture frame rate and the output frame rate are limited. It is 10 frames, which makes a compromise between smoothness and low consumption.

Further, in the step S4, according to the RTMP protocol, the compressed video stream content needs to be encapsulated into FLV, and then uploaded to the RTMP server in the form of a media stream through the RTMP protocol.

In the step S5, the webpage client calls the Flash plug-in of the user's browser to realize decoding and playback, without installing specific client software, only need to use the web browser to log in to the webpage client address, you can browse to the remote PC host screen content sharing.

Generally speaking, the step S1 and the steps S2, S3, and S4 belong to two asynchronous threads, and the two asynchronous threads work in parallel to improve program efficiency. At the same time, the processing strategy adopted in the whole processing flow is to capture a picture and immediately encode a frame of video and then upload it to the streaming media server immediately.

Further, in the described steps S5 and S6, the user realizes the control of the remote PC host screen through the web client, and the web client uses the interface of ActionScript to capture the mouse action or keyboard action for the web client, and the mouse operation Record the relative coordinates in the web client at any time, and transmit its coordinates and action information to the remote PC host through a custom protocol; or capture the keyboard action for the web client, and transmit it to the remote PC through a custom protocol host.

In the step S7, after the remote PC host receives the control information from the client, the corresponding operation is realized through the API of the Windows system, and the corresponding operation effect is presented on the screen display of the remote PC host, and the user can realize remote interactive control. Effect. For example, if the screen displays a PPT content, the user can realize the remote page turning operation by clicking the mouse, and remotely modify the PPT content by keyboard operation.

Compared with the prior art, the beneficial effect is:

1. Using the method of intercepting the PC screen, the image data obtained by the screen capture is converted into an H.264 video stream by means of image compression and encoding into an H.264 video stream and uploaded to the server, and then published to the network for users to remotely browse the host screen content. The use of fixed frame rate screen capture and encoding output ensures real-time performance, fluency and picture continuity, making the user experience good.

2. Use open source RTMP server, low resource consumption, easy installation and setting. Instead of using the commonly used protocols of common video conferencing systems such as the SIP protocol to share and display screen content, the RTMP protocol is used to transmit screen video streams, so that the client does not need to install any software, and only needs to browse the URL published by the screen content through the browser. You can remotely browse the screen sharing content. The simple RTMP transmission protocol is aimed at relatively light and simple system applications, so that there is no need to consume unnecessary resources caused by using complex protocols on small systems, and it can most effectively use a small amount of resources to achieve simple effects.

3. Relying on efficient compression and encoding algorithms, the processing strategy adopted in the entire processing process is to capture a picture and immediately encode a frame of video and then upload it to the streaming media server immediately, which simplifies the implementation of the system and does not need to use a complicated memory buffer mechanism, saving memory resources.

4. Do not use the complicated Microsoft remote desktop control protocol to realize interactive operation, customize a simple protocol, only need the client to capture mouse and keyboard events, send them to the PC host through TCP, realize the response on the remote host, and send the response The effect is fed back to the web client through the screenshot video stream of the screen content, realizing the simple remote desktop control effect. And because it does not involve complex control protocols, it is easy to expand for specific usage scenarios.

Description of drawings

FIG. 1 is a schematic diagram of the system architecture of the present invention.

Fig. 2 is a schematic flow chart of the content sharing steps of the PC screen of the present invention;

Fig. 3 is a schematic flow chart of the PC screen interactive control steps of the present invention;

Fig. 4 is a schematic diagram of the remote PC host end flow chart of the present invention;

Fig. 5 is a schematic flow chart of the webpage client of the present invention;

FIG. 6 is a schematic diagram of the system architecture of the user perspective of the present invention.

detailed description

The accompanying drawings are for illustrative purposes only, and should not be construed as limitations on this patent; in order to better illustrate this embodiment, certain components in the accompanying drawings will be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art It is understandable that some well-known structures and descriptions thereof may be omitted in the drawings.

As shown in Figures 1, 2, and 3, a PC screen content sharing interactive control method includes the following steps, wherein steps S1 to S4 are steps for sharing content on a PC screen, and steps S5 to S8 are steps for PC screen interaction control.

S1. The remote PC host specifies the window that needs to be shared, such as the name of the PPT window and the name of the picture window. The screen capture program will intercept the content of the specified window, and the obtained screen data will be stored in the shared memory data buffer in RGB format;

S2. Get the screen RGB data in the shared memory data buffer, convert it to YUV420 data, and compress the data volume to meet the needs of video encoding and real-time transmission;

S3. Utilize a software encoder to encode and compress YUV420 format data into an H.264 video stream;

S4. Upload the encoded and compressed screen content video stream to the streaming media server through the RTMP protocol, and the streaming media server publishes it to the network, and sends the media stream with the RTMP protocol according to the user's request;

S5. The user logs in to the address of the webpage client publishing the media stream of the remote PC host through the browser, and the webpage client captures the mouse or keyboard operation actions of the user end;

S6. According to the self-defined remote control protocol, transmit the control information to the remote PC host through the TCP protocol;

S7. The remote PC host analyzes the received control information of the client, performs corresponding control operations, and presents them on the PC screen;

S8. Present the effect of the screen response on the webpage client logged in by the user.

Further, in the step S1, after the name of the specified windows window is obtained, the API of the windows system is called to take a screenshot of the specified windows window, and the screenshot is automatically performed according to the acquired length and width parameters of the actual window.

Further, in the step S3, it is necessary to ensure that the code rate cannot consume too much bandwidth, so that it can smoothly browse the screen content of the host computer even in a poor network environment, and both the screen capture frame rate and the output frame rate are limited. It is 10 frames, which makes a compromise between smoothness and low consumption.

Further, in the step S4, according to the RTMP protocol, the compressed video stream content needs to be encapsulated into FLV, and then uploaded to the RTMP server in the form of a media stream through the RTMP protocol.

In the step S5, the webpage client calls the Flash plug-in of the user's browser to realize decoding and playback, without installing specific client software, only need to use the web browser to log in to the webpage client address, you can browse to the remote PC host screen content sharing.

Generally speaking, the step S1 and the steps S2, S3, and S4 belong to two asynchronous threads, and the two asynchronous threads work in parallel to improve program efficiency. At the same time, the processing strategy adopted in the whole processing flow is to capture a picture and immediately encode a frame of video and then upload it to the streaming media server immediately.

Further, in the described steps S5 and S6, the user realizes the control of the remote PC host screen through the web client, and the web client uses the interface of ActionScript to capture the mouse action or keyboard action for the web client, and the mouse operation Record the relative coordinates in the web client at any time, and transmit its coordinates and action information to the remote PC host through a custom protocol; or capture the keyboard action for the web client, and transmit it to the remote PC through a custom protocol host.

In the step S7, after the remote PC host receives the control information from the client, the corresponding operation is realized through the API of the Windows system, and the corresponding operation effect is presented on the screen display of the remote PC host, and the user can realize remote interactive control. Effect. For example, if the screen displays a PPT content, the user can realize the remote page turning operation by clicking the mouse, and remotely modify the PPT content by keyboard operation.

As shown in Figure 4, it is a schematic diagram of the process of the remote PC host terminal, and the steps are as follows:

a. The remote PC host specifies the windows that need to share content, such as the name of the PPT window and the name of the picture window;

b. According to whether to accept remote control for branch processing, if not necessary, do not open a remote interactive control management process, perform the following operations; otherwise, open a new process, parallel to the following operations;

Thread 1: The screen capture program will capture the content of the specified shared window, and the obtained screen data will be stored in the shared memory data buffer in RGB format;

c. Thread 2: Convert the screen RGB data in the shared memory data buffer to YUV420 data, use a software encoder to encode and compress the YUV420 format data into an H.264 video stream; upload the encoded and compressed screen content video stream data through the RTMP protocol To the streaming media server, the streaming media server publishes it to the network, and sends the media stream with RTMP protocol according to the user's request;

d. If it is necessary to open up a new remote interactive control process, it waits in a loop to receive the control information from the client, then parses it, and responds on the host.

As shown in Figure 5, it is a schematic diagram of the flow of the web client, and the steps are as follows:

A． Accept user login;

B． The web client requests the media stream of the remote PC screen from the streaming server;

C． Call the browser Flash plug-in to decode and play the video stream;

D． Determine whether the remote host accepts the user's interactive control, if not, do not perform the following operations, otherwise, perform the following operations;

E． Capture mouse or keyboard actions on the client side;

F． According to the custom remote control protocol, the control information is transmitted to the remote PC host through the TCP protocol.

As shown in Figure 6, the system implementation process is described from the perspective of the user by integrating the remote PC host terminal and the webpage client, and the steps are as follows:

(1) The host specifies the content sharing window, and the program executes the screen capture operation, and compresses and encodes it into a video stream and uploads it to the streaming media server, and the streaming media server publishes the video stream on the network;

(2) The user learns the URL of the web client corresponding to the video stream published by the remote host, and logs in to the web client through a web browser;

(3) The web client requests the corresponding video stream data from the streaming media server, and calls the browser's Flash plug-in to decode and play, and presents the content of the remote host sharing window on the user's screen;

(4) If the remote host is open to receive the user's remote interactive control, the web client will continuously capture the mouse and keyboard actions, and encapsulate them according to the customized control protocol and transmit them to the remote host through the TCP protocol;

(5) The remote host receives the user's control information, analyzes and responds, and presents it on the screen changes, so as to achieve the effect of remote interactive control by the user.

The same or similar symbols correspond to the same or similar components; the positional relationship described in the drawings is only for illustrative purposes, and should not be construed as a limitation on this patent.

The above description is only the preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, some improvements suitable for the actual situation can be made to the technical solution of the invention. Therefore, the protection scope of the present invention is not limited thereto, and any insubstantial changes based on the technical solution of the present invention are included in the protection scope of the present invention.

Claims (7)

1. a PC screen content sharing/interaction control method, it is characterised in that comprise the following steps:

S1. remotely the appointment of PC main frame needs to carry out the window that content is shared, and screenshotss program will intercept specified window content, it is thus achieved that On-screen data data in an rgb format be stored in shared drive data buffer zone；

S2. obtain the screen RGB data in shared drive data buffer zone, be converted into YUV420 data, compress data Amount, with adaptive video coding and the demand of real-time Transmission；

S3. utilize software encoder that YUV420 formatted data compression coding is become H.264 video flowing；

S4. screen content video flowing good for compression coding is uploaded to streaming media server, streaming media service by RTMP agreement Device is issued long-range PC host media and is flowed on webpage client, issues long-range PC host media by user's request with RTMP agreement Stream；

S5. user signs in the address of described webpage client by browser, the mouse of webpage client capture user side or Keyboard operation action；

S6. according to custom remote control protocol, by Transmission Control Protocol to long-range PC main frame transmission capture user side mouse or Keyboard operation action message；

S7. remotely PC aster Resolution receives the mouse of user side or the keyboard operation action message of capture, controls accordingly System operation, is presented on the screen of long-range PC main frame；

S8. the effect that the screen of long-range PC main frame responds being presented on the webpage client that user logs in, webpage client calls The Flash plug-in unit of user browser realizes decoding and plays, and makes user browse to long-range PC main frame screen content.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step In S1, after obtaining the name specifying windows window, its windows window specified is entered by the API calling windows system Row screenshotss, intercept according to the length and width parameter of the actual window obtained automatically.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step In S3, screenshotss frame per second and output frame rate are all 10 frames.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step In S4, need the video streaming content compressed is encapsulated as FLV according to RTMP agreement, then by RTMP agreement with Media Stream Pass on streaming media server in form.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step S1 and step S2, S3, S4, adhere to two asynchronous threads, the two asynchronous thread concurrent working separately.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step In S5, S6, user is to realize the control to long-range PC main frame screen by webpage client, and webpage client uses The interface capture of ActionScript for the mouse action of webpage client or keyboard action, and by during mouse action at net Relative coordinate in page client is recorded, and its coordinate and action message is sent in long-range PC main frame by making agreement by oneself Go；Or capture the keyboard action for webpage client, be transmitted in long-range PC main frame again by certainly concluding an agreement.

A kind of PC screen content sharing/interaction control method the most according to claim 1, it is characterised in that described step In S7, after long-range PC main frame receives mouse or the keyboard operation action message of the user side of capture, by Windows system API realizes corresponding operation, the operating effect of response is presented on long-range PC main frame screen and shows, to realize remote interaction control System.