CN114157903B - Redirection method, device, apparatus, storage medium and program product - Google Patents

Abstract

The embodiments of the present application disclose a redirection method, apparatus, device, storage medium and program product, which belong to the field of screen projection technology. The method includes: creating a source redirection instance based on a source device identifier and a target redirection object identifier, the target redirection object identifier is used to characterize the target redirection object; when the same source redirection instance corresponds to at least two destination redirection instances, the redirection data corresponding to the source redirection instance is generated by a multiplexing redirection data generator, and the destination redirection instance is created based on the destination device identifier and the target screen identifier; transmitting the redirection data to the destination device so that the destination device displays the target redirection object through the target screen based on the redirection data. The scheme provided by the embodiments of the present application can reduce the encoding and decoding overhead generated by data transmission between the source device and the destination device.

Description

Redirection method, device, equipment, storage medium and program product

Technical Field

The embodiments of the present application relate to the field of screen projection technology, and in particular to a redirection method, device, equipment, storage medium and program product.

Background Art

Redirection refers to the technology of transferring certain capabilities from one device to another. The capabilities transferred include video, audio, control, etc. Common redirection technologies in daily life include device screen projection.

With the help of the screen projection function, the source device can project the currently displayed image onto the destination device to achieve a better display effect. The screen projection solution provided by the relevant technology is suitable for simple screen projection scenarios, such as one-to-one screen projection scenarios or many-to-one screen projection scenarios. However, as the complexity of the screen projection scenario increases, such as many-to-many screen projection scenarios, the data flow between the source device and the destination device increases, resulting in increased encoding and decoding overhead.

Summary of the invention

The embodiments of the present application provide a redirection method, apparatus, device, storage medium and program product. The technical solution is as follows:

On the one hand, an embodiment of the present application provides a redirection method, which is used for a source device, and the method includes:

Creating a source redirection instance based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

In the case where the same source-end redirection instance corresponds to at least two destination-end redirection instances, the redirection data corresponding to the source-end redirection instance is generated by a multiplexing redirection data generator, and the destination-end redirection instance is created based on a destination-end device identifier and a target screen identifier, wherein the destination-end device identifier is used to characterize the destination-end device, and the target screen identifier is used to characterize the target screen of the destination-end device;

The redirection data is transmitted to the target device, so that the target device displays the target redirection object through the target screen based on the redirection data.

On the other hand, an embodiment of the present application provides a redirection method, which is used for a destination device, and the method includes:

Creating a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device;

Receiving redirection data corresponding to a source redirection instance transmitted by a source device, wherein the source redirection instance is created based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

In the case that there are at least two destination redirection instances corresponding to the same source redirection instance, parsing the redirection data by using a multiplexing redirection data parser;

Based on the parsing result of the redirection data, the target redirection object is displayed through the target screen.

On the other hand, an embodiment of the present application provides a redirection device, the device comprising:

A creation module, used to create a source redirection instance based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

a multiplexing module, configured to generate redirection data corresponding to the source redirection instance by a multiplexing redirection data generator when the same source redirection instance corresponds to at least two destination redirection instances, wherein the destination redirection instance is created based on a destination device identifier and a target screen identifier, wherein the destination device identifier is used to characterize the destination device, and the target screen identifier is used to characterize the target screen of the destination device;

The first transmission module is used to transmit the redirection data to the target device, so that the target device displays the target redirection object through the target screen based on the redirection data.

On the other hand, an embodiment of the present application provides a redirection device, the device comprising:

A creation module, used to create a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device;

A first receiving module, configured to receive redirection data corresponding to a source redirection instance transmitted by a source device, wherein the source redirection instance is created based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

A parsing module, configured to parse the redirection data by using a multiplexing redirection data parser when at least two destination redirection instances correspond to the same source redirection instance;

A display module is used to display the target redirection object through the target screen based on the parsing result of the redirection data.

On the other hand, an embodiment of the present application provides a redirection system, the redirection system comprising: at least one source device and at least one destination device;

The source-end device is used to create a source-end redirection instance based on a source-end device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to characterize a target redirection object; when the same source-end redirection instance corresponds to at least two target-end redirection instances, generate redirection data corresponding to the source-end redirection instance by using a multiplexing redirection data generator; and transmit the redirection data to the target-end device so that the target redirection object is displayed on the target screen by the target-end device based on the redirection data;

The destination device is used to create a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device; receive redirection data corresponding to the source redirection instance transmitted by the source device; when there are at least two destination redirection instances corresponding to the same source redirection instance, parse the redirection data by a multiplexing redirection data parser; and based on the parsing result of the redirection data, display the target redirection object through the target screen.

On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the redirection method described in the above aspect.

On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the redirection method as described in the above aspects.

On the other hand, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the redirection method provided in various optional implementations of the above aspects.

During the redirection process, the source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, and the destination device creates a destination redirection instance corresponding to the source redirection instance based on the destination device identifier and the target screen identifier. When the same source redirection instance corresponds to at least two destination instances, the source device generates redirection data through a multiplexing redirection data generator and transmits it to the destination device. The destination device receives the redirector data, parses the redirection data through a multiplexing redirection data parser, and displays the target redirection object through the target screen indicated by the destination redirection instance. The solution provided in the embodiment of the present application is adopted, and the encoding overhead of the source device and the decoding overhead of the destination device are reduced during the data stream transmission between the source device and the destination device by multiplexing the redirection data generator and the redirection data parser, thereby improving the data transmission performance in complex redirection scenarios.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

FIG2 is a schematic diagram of an implementation of the encoding, transmission and decoding process provided by an exemplary embodiment of the present application;

FIG3 is a schematic diagram of an implementation of the encoding, transmission and decoding process provided by another exemplary embodiment of the present application;

FIG4 is a flow chart of a redirection method provided by an exemplary embodiment of the present application;

FIG5 is a schematic diagram of a hierarchical architecture of a redirection module according to an exemplary embodiment of the present application;

FIG6 is a schematic diagram of a hierarchical architecture of a redirection module according to another exemplary embodiment of the present application;

FIG7 is a flowchart of a redirection method provided by another exemplary embodiment of the present application;

FIG8 is a flowchart of a video encoding, decoding and transmission process shown in an exemplary embodiment of the present application;

FIG9 is a flowchart of an exemplary embodiment of the present application showing a method for splitting and supplementing frames;

FIG10 is a flowchart of a redirection method for asynchronous access shown in an exemplary embodiment of the present application;

FIG11 is a flowchart of a concurrent access redirection method shown in an exemplary embodiment of the present application;

FIG12 is a schematic diagram of an implementation of a concurrent access redirection method according to an exemplary embodiment of the present application;

FIG13 is a structural block diagram of a redirection device provided by an embodiment of the present application;

FIG14 is a structural block diagram of a redirection device provided by another embodiment of the present application;

FIG15 is a system architecture diagram of a redirection system provided by an embodiment of the present application;

FIG. 16 is a structural block diagram of a computer device provided by an exemplary embodiment of the present application.

DETAILED DESCRIPTION

In order to make the objectives, technical solutions and advantages of the present application more clear, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

Fig. 1 shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment may include: at least one source device 110 and at least one destination device 120.

The source device 110 is an electronic device with redirection requirements, which may be a mobile terminal such as a smart phone, a tablet computer, a personal computer, etc. In FIG1 , the source device 110 includes a tablet computer 111 and a smart phone 112 as an example for illustration.

The object with redirection requirements in the source device 110 is called a redirection object, which can be a screen mirror or a redirection application, and the redirection content of the redirection object can include at least one of video, audio, and control. The redirection application can be a navigation application, a video application, a game application, etc. The embodiment of the present application does not limit the specific type of the screen projection application. Optionally, when the redirection object is an application, the application can be in the background running state during the redirection process.

In an embodiment of the present application, the same source device can redirect at least two redirection objects at the same time. For example, screen mirroring and video applications can be cast at the same time, or video applications and navigation applications can be cast at the same time. Among them, the number of redirection objects that the source device supports to redirect simultaneously has an upper limit, and the upper limit is related to the processing performance of the source device.

The destination device 120 is an electronic device for displaying redirection results, which may be a smart phone, a tablet computer, a personal computer, a smart TV, or a vehicle-mounted terminal (such as a vehicle computer), etc. FIG1 is taken as an example in which the destination device 120 includes a first vehicle-mounted terminal 121 and a second vehicle-mounted terminal 122, wherein the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 are arranged in the same vehicle, and the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 are respectively controlled by different SoCs (System on Chip). For example, the first vehicle-mounted terminal 121 is a vehicle-mounted terminal arranged in the front row of the vehicle, and the second vehicle-mounted terminal 122 is a vehicle-mounted terminal arranged in the back row of the vehicle.

Optionally, the destination device 120 has at least one screen. As shown in FIG1 , the first vehicle terminal 121 has a first screen 1211 and a second screen 1212 (both controlled by the first SoC 1213 of the first vehicle terminal 121), and the second vehicle terminal 122 has a third screen 1221 and a fourth screen 1222 (both controlled by the second SoC 1223 of the second vehicle terminal 122).

In some embodiments, when the destination device 120 has at least two screens, at least two screens can display different projection contents at the same time, or the same projection contents, and the projection contents displayed on each screen can be a single projection object or multiple projection objects (displaying multiple projection objects in split screen).

In addition to the function of displaying the projected content, the destination device 120 may also have an audio playback function. The audio played during the screen projection process is the audio of the projected object.

In addition to the audio and video playback function, the destination device 120 can also realize the reverse control function of the screen projection content through the input and output components. Optionally, the input and output components can be a touch screen of the destination device 120, an external control device of the destination device 120, such as a handle, keyboard, mouse or touchpad, etc., which is not limited in this embodiment.

The many-to-many redirection in the embodiment of the present application is embodied in multiple source devices (multiple source devices participate in redirection), multiple redirection objects (multiple redirection objects in the same source device participate in redirection), multiple destination devices (multiple destination devices participate in redirection) and multiple screens (multiple screens of the same destination device participate in redirection). In a possible many-to-many screen projection scenario, as shown in FIG1 , the video application in the tablet computer 111 is projected to the first screen 1211 of the first vehicle terminal 121; the first game application is projected to the second screen 1212 of the first vehicle terminal 121. The navigation application in the smartphone 112 is projected to the first screen 1211 of the first vehicle terminal 121 and is displayed in split screen with the video application; the second game application in the smartphone 112 is projected to the third screen 1221 and the fourth screen 1222 of the second vehicle terminal 122, that is, the third screen 1221 and the fourth screen 1222 display the same projection content. In addition, since each screen of the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 has a touch function, the user can operate the projection application on the screen, thereby achieving reverse control of the projection application.

It should be noted that the connection method used between the source device 110 and the destination device 120 may include at least one of WlANAP (Wireless Local Area Network Access Point) mode, WiFi-P2P (Wireless Fidelity-Peer to Peer), USB (Universal Serial Bus) wired connection, and mobile data network, and the embodiments of the present application are not limited to this.

In addition, the above embodiment is only schematically explained using the example of screen projection from a mobile terminal to a vehicle-mounted terminal (i.e., the vehicle-mounted terminal scenario). The solution provided in the embodiment of the present application can be applied to other many-to-many redirection scenarios, and this embodiment does not limit the specific application scenario.

In an embodiment of the present application, before transmitting redirection data to the destination device, the source device needs to collect the redirection object and encode the collected content through the redirection data generator to generate redirection data. This process will generate encoding overhead. Correspondingly, after the destination device receives the redirection data, it needs to use the redirection data parser to parse the redirection data to restore the redirection object. This process will generate decoding overhead. Taking the redirection object as a video as an example, the source device needs to collect the video screen first, and then use the video encoder to encode the collected screen to obtain a video data stream. This process generates encoding overhead. Correspondingly, after the destination device receives the video data stream, it needs to decode it through a video decoder to restore the video screen. This process will generate decoding overhead.

In a possible screen projection scenario, the same projection object in the source device needs to be projected to different destination devices, or to different screens of the same destination device. The related technology adopts the method shown in Figure 2, that is, when the projection application in the source device needs to be simultaneously projected to screen 1 and screen 2 of destination device 1, and screen 3 of destination device 2, video encoding is performed by video encoders 1, 2, and 3 respectively, video decoding is performed by video decoders 1, 2, and 3 respectively, and video data stream transmission in three paths will generate three times the encoding and decoding overhead.

In the embodiment of the present application, the source device and the destination device can adopt a multiplexing strategy, that is, the source device encodes the redirection object by a multiplexing redirection data generator to generate redirection data, and the destination device parses the redirection data by a multiplexing redirection data parser to restore the redirection object. Compared with the related art, the source device and the destination device respectively adopt a multiplexing strategy to reduce the encoding and decoding overhead. As shown in Figure 3, when it is necessary to simultaneously project the screen projection application in the source device to screen 1 and screen 2 of the destination device 1, and screen 3 of the destination device 2, the source device multiplexes the video encoder 1 to perform video encoding to obtain a video data stream. After the destination device 1 receives the video data stream, it performs video decoding by multiplexing the video decoder 1, so that the decoded picture is displayed through screen 1 and screen 2. After the destination device 2 receives the video data stream, it performs video decoding by video decoder 2, so that the decoded picture is displayed through screen 3. Compared with the related art, the source device and the destination device respectively adopt a multiplexing strategy encoder and decoder, which reduces the encoding overhead by 2 times and the decoding overhead by 1 times. The solution provided by the embodiments of the present application is introduced below.

Please refer to FIG. 4 , which shows a flowchart of a redirection method provided by an exemplary embodiment of the present application. The present application embodiment takes the method applied to the implementation environment shown in FIG. 1 as an example for explanation. The method includes:

Step 401: The source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, where the target redirection object identifier is used to represent the target redirection object.

The type of the redirection object includes at least one of screen mirroring or redirection application, and the type of redirection content of the redirection object includes at least one of audio, video and control.

When redirection is required through the source device, the source device creates a source redirection instance based on the source device identifier (i.e., its own device identifier) and the target redirection object identifier of the target redirection object to be redirected, that is, the source redirection instance is formed in the unit of "source device + redirection object".

Among them, when different source-end devices redirect the same redirection object (for example, different source-end devices redirect the same application), the source-end redirection instances created by different source-end devices are different; when the same source-end device needs to redirect different redirection objects, the source-end redirection instances created by the same source-end device based on different redirection objects are also different.

Optionally, the source device creates a source redirection instance when receiving a redirection operation, or creates a source redirection instance when the source device receives a redirection instruction from a third-party device, that is, redirection through the source device can be triggered on the source device side or by a third-party device, wherein the third-party device and the source device are in the same scheduling domain, for example, the third-party device and the source device have the same user account for logging in.

Regarding the specific content of the source device redirection, in some embodiments, the source device can redirect the screen image or the installed and running application (i.e., the redirected application). Correspondingly, when the target redirection object is the screen image, the target redirection object identifier is the image identifier; when the target redirection object is the redirected application, the target redirection object identifier is the application identifier of the redirected application.

Step 402: The destination device creates a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent the target screen of the destination device.

Corresponding to the source device, the destination device for display needs to create a destination redirection instance corresponding to the source redirection instance. The destination device creates the destination redirection instance based on the destination device identifier (i.e., its own device identifier) and the target screen identifier of the target screen for display, i.e., the destination redirection instance is formed in the unit of "destination device + target screen".

In the embodiment of the present application, the destination device has at least one screen, different screens correspond to different screen identifiers, and different screens are controlled by the same SoC in the destination device. For example, the destination device is a vehicle-mounted terminal, which has a main driver's screen and a co-driver's screen.

When different destination devices display the same redirection object, the destination redirection instances created by the different destination devices are different; when the same destination device displays through different screens, the destination redirection instances created by the same destination device based on different screens are also different.

Optionally, the destination device creates a destination redirection instance when it receives a redirection request sent by the source device, or the destination device creates a destination redirection instance when it receives a redirection instruction from a third-party device, where the third-party device and the destination device are in the same scheduling domain, for example, the third-party device and the destination device have the same user account to log in.

Step 403: When the same source-end redirection instance corresponds to at least two destination-end redirection instances, the source-end device generates redirection data corresponding to the source-end redirection instance by using a multiplexing redirection data generator.

In a possible implementation, the same source redirection instance corresponds to at least two destination redirection instances, which means that the same redirection object of the same source device is redirected to at least two target screens of the same destination device. For example, as shown in FIG3 , the screen projection application of the same source device is projected to screen 1 and screen 2 of destination device 1.

In another possible implementation, the same source redirection instance corresponds to at least two destination redirection instances, which means that the same redirection object of the same source device is redirected to the target screens of different destination devices. For example, as shown in FIG3 , the screen projection application of the same source device is projected to screen 1 and screen 2 of destination device 1 and screen 3 of destination device 2.

In the above case, after the source device completes the creation of the source redirection instance, in order to reduce the encoding overhead of the source device, the source device adopts a multiplexing strategy, that is, a multiplexing redirection data generator converts the redirection object into redirection data. The type of the redirection data generator is related to the media type corresponding to the target redirection object. In the embodiment of the present application, the media type corresponding to the target redirection object includes at least one of video, audio, and control.

Optionally, when the media type of the target redirection object is video, the redirection data generator is a video encoder; when the media type of the target redirection object is audio, the redirection data generator is an audio encoder; when the media type of the target redirection object is control, the redirection data generator is a control encoder.

In addition, the redirection data includes at least a video data stream, and may also include at least one of an audio data stream and a control data stream.

In some embodiments, the source device performs video encoding on the screen of the target redirection object to obtain a video data stream, wherein the source device can use video encoding formats such as H264 and H265 for encoding, which is not limited to this embodiment of the present application; when the redirection data includes an audio data stream, the source device performs audio encoding on the sound of the target redirection object to obtain an audio data stream, wherein the source device can use audio encoding formats such as AAC for encoding, which is not limited to this embodiment of the present application; when the redirection data includes a control data stream, the source device can encode the control instructions in an agreed instruction format to obtain a control data stream.

Step 404: The source device transmits redirection data to the destination device.

Regarding the transmission method of redirected data, optionally, the source device can transmit the redirected data wirelessly, such as Wlan AP, WiFi-P2P or mobile data network; it can also transmit the redirected data by wired means, such as USB wired connection; it can also transmit the redirected data by a hybrid wired and wireless method, which is not limited to the embodiments of the present application.

Step 405: The destination device receives the redirection data transmitted by the source device.

In a possible implementation manner, the same destination device receives redirected data transmitted by the same source device.

In another possible implementation, different destination devices receive redirected data transmitted by the same source device.

Step 406: When there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device parses the redirection data by using the multiplexing redirection data parser.

In one possible implementation, at least two destination redirection instances corresponding to the same source redirection instance may be different destination devices displaying the same redirection object of the same source device; in another possible implementation, the same destination device may display the same redirection object of the same source device through different screens.

At this time, in order to reduce the decoding overhead of the destination device, the destination device adopts a multiplexing strategy, that is, multiplexing the redirection data parser to parse the redirection data.

The type of the redirection data parser is related to the media type corresponding to the target redirection object. Optionally, when the media type of the target redirection object is video, the redirection data parser is a video decoder, and when the media type of the target redirection object is audio, the redirection data parser is an audio decoding player. When the media type of the target redirection object is control, the redirection data parser is a control decoder.

In some embodiments, when the redirected data includes a video data stream, the destination device parses the video data stream through a video decoder; when the redirected data includes an audio data stream, the destination device parses the audio data stream through an audio decoding player; when the redirected data includes a control data stream, the destination device parses the control data stream through a control decoder.

Step 407: The destination device displays the target redirection object through the target screen based on the parsing result of the redirection data.

Furthermore, the destination device determines the target screen indicated by the destination redirection instance, so as to display the target redirection object through the target screen.

In some embodiments, after the destination device parses the video data stream, it plays the image of the target redirection object through the video playback component; after the destination device parses the audio data stream, it plays the sound of the target redirection object through the audio playback component.

To summarize, in an embodiment of the present application, during the redirection process, the source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, and the destination device creates a destination redirection instance corresponding to the source redirection instance based on the destination device identifier and the target screen identifier. When the same source redirection instance corresponds to at least two destination instances, the source device generates redirection data through a multiplexing redirection data generator and transmits it to the destination device. The destination device receives the redirector data, parses the redirection data through a multiplexing redirection data parser, and displays the target redirection object through the target screen indicated by the destination redirection instance. By adopting the solution provided in the embodiment of the present application, the encoding overhead of the source device and the decoding overhead of the destination device during the data stream transmission between the source device and the destination device are reduced by multiplexing the redirection data generator and the redirection data parser, thereby improving the data transmission performance in complex redirection scenarios.

In order to achieve multi-level many-to-many redirection, the redirection modules of the source device and the destination device are designed in layers. As shown in Figure 5, the hierarchical structure of the redirection module includes:

Device layer: includes multi-channel screen projection services, which are used to provide the device's screen projection API (Application Programming Interface) to the outside world.

Screen projection application layer: includes the source redirection instance (when acting as a source device) or the destination redirection instance (when acting as a destination device) created by the screen projection manager.

Streaming media processing layer: includes video processing module, audio processing module and control processing module, which are used to process data of specific media types in the redirection instance of the source or destination.

Redirection instance layer: When used as a source device, it includes a source video redirector, a source audio redirector, and a source control redirector; when used as a destination device, it includes a destination video redirector, a destination audio redirector, and a destination control redirector. The transmission link between the source redirector and the destination redirector of a specific media type is used to transmit redirected data of a specific media type.

Display layer: includes projection window applications, and different projection window applications correspond to different screens of the destination device.

In the embodiment of the present application, the source device and the destination device adopt a multiplexing strategy, and reduce the encoding and decoding overhead of the source device and the destination device during data stream transmission by multiplexing the structure in the redirection module. The multiplexing strategy of the source device and the destination device is described below in conjunction with a more detailed hierarchical structure of the redirection module as shown in FIG6.

1. Source-side screen projection instance: The same redirection object is reused and is responsible for managing various types of media processing modules and redirectors under the screen projection application at the source end.

2. Target-side screen projection instance: The same source device + redirection object is reused, responsible for managing various types of redirectors under the screen projection application on the target side.

3. Redirection context: The same redirection object is reused to maintain the context data and state transition of the redirection object.

4. Source-side video processing module: reuses the same media type under the source-side screen projection instance, and is responsible for maintaining and uniformly processing the business logic of a certain media type.

5. Video encoder: The video media type is reused under the source screen projection instance to ensure that the same redirection object will only be encoded once and is responsible for distributing it to multiple screen projection links at the lower level.

6. Source video redirector: corresponds to each specific screen projection instance and its physical channel on the source side, is responsible for receiving the screen projection request from the destination side and transmitting the media stream to it without multiplexing.

7. Destination video redirector: Corresponding to each specific screen projection instance and its physical channel on the destination, it is responsible for initiating screen projection requests to the source and receiving and processing media streams from it. The same source device + multiplexing under the redirection object provides unified media stream reception and processing for multiple displays.

8. Video decoder: The same source device + redirection object is reused to ensure that the same redirection object will only be decoded one way and provide decoded images for multiple displays.

9. Target screen projection window application: corresponds to the target display screen one by one, and multiplexes multiple media streams from the same display screen. Responsible for displaying multiple video images on the display screen.

10. Audio capture encoder: The device-level audio capture encoder is reused by the entire source device. The application-level audio capture encoder is reused at the level of a certain screen projection application. Audio recording and encoding are performed uniformly to provide data for multiple audio links.

11. Audio decoding player: Device-level audio is multiplexed per source device, and application-level audio is multiplexed per source device + redirection object.

In combination with the above hierarchical structure, the reuse strategy and redirection process of the source device and the destination device are further explained below.

Please refer to FIG. 7 , which shows a flowchart of a redirection method provided by another exemplary embodiment of the present application. The present application embodiment takes the method applied to the implementation environment shown in FIG. 1 as an example for explanation. The method includes:

Step 701: The source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, where the target redirection object identifier is used to represent the target redirection object.

The implementation of this step may refer to the above step 401, and this embodiment will not be described in detail here.

Step 702: The destination device creates a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent the target screen of the destination device.

The implementation of this step can refer to the above-mentioned step 402, and the embodiment of the present application will not be described in detail here.

Step 703: When the same source-end redirection instance corresponds to at least two destination-end redirection instances, the source-end device generates redirection data corresponding to the source-end redirection instance by using a multiplexing redirection data generator.

The implementation of this step can refer to the above-mentioned step 403, and the embodiment of the present application will not be described in detail here.

Step 704: When at least two destination redirection instances correspond to the same destination device, the source device creates a source redirector; or, when at least two destination redirection instances correspond to at least two destination devices, the source device creates at least two source redirectors.

In a possible implementation, after the source device creates the source redirection instance, it further creates a source redirector based on the source redirection instance, and the source redirector is used to transmit the redirection data corresponding to the source redirection instance. Optionally, the source device creates the source redirector based on the source redirection instance and the destination device identifier.

In a possible implementation, when the same source device projects the same redirection object to multiple screens of a destination device, the source device creates a source redirector. For example, when a game application needs to be simultaneously projected to the first screen and the second screen of the first destination device, the source device creates a source redirector based on the source device identifier, the first destination identifier, and the game application identifier.

In another possible implementation, when the same source device projects the same redirection object to the screens of different destination devices, the source device creates multiple source redirectors. For example, when a game application needs to be projected to the first destination device and the second destination device, the source device creates a first source redirector based on the source device identifier, the first destination identifier, and the game application identifier, and creates a second source redirector based on the source device identifier, the second destination identifier, and the game application identifier.

Optionally, when redirecting, the media type of the redirection object can be set (can be set manually by the user, or can be automatically set by the source device based on the capabilities of the destination device), and the media type includes at least one of video, audio, and control. When the media type of the source redirection instance includes video, the destination device can display the target redirection object through the screen; when the media type of the source redirection instance includes audio, the destination device can play the sound of the target redirection object through the audio playback component; when the media type of the source redirection instance includes control, the destination device can reversely control the target redirection object.

Accordingly, the source-end device needs to create source-end redirectors corresponding to different media types according to the media type corresponding to the target redirection object. In another possible implementation, the source-end device creates at least one source-end redirector based on the source-end redirection instance, the destination-end device identifier, and the media type corresponding to the target redirection object, wherein different source-end redirectors correspond to different media types.

Exemplarily, when the media type corresponding to the target redirection object includes video, the source-end device creates a source-end video redirector, which is used to redirect the video data stream of the target redirection object; when the media type corresponding to the target redirection object includes audio, the source-end device creates a source-end audio redirector, which is used to redirect the audio data stream of the target redirection object; when the media type corresponding to the target redirection object includes control, the source-end device creates a source-end control redirector, which is used to redirect the control data stream of the target redirection object.

For example, when it is necessary to project a game application to the first destination device, and the first destination device supports displaying the game application screen, playing the game application sound, and reverse controlling the game application, the source device creates a source video redirector based on the source device identifier, the first destination identifier, the game application identifier, and the video type, and creates a source audio redirector based on the source device identifier, the first destination identifier, the game application identifier, and the audio type; and creates a source control redirector based on the source device identifier, the first destination identifier, the game application identifier, and the control type.

Step 705: The destination device creates a destination redirector, and the destination redirector is multiplexed by at least two destination redirection instances.

Corresponding to the source device, the destination device needs to create a destination redirection corresponding to the source redirector, so as to subsequently transmit the redirection data corresponding to the source redirection instance through the link between the source redirector and the destination redirector.

Regarding the method of creating a destination redirector, in a possible implementation, the destination device knows the source device and the target redirection object, and thus creates the destination redirector based on the source device identifier, the target redirection object identifier, and the destination device identifier.

In a possible implementation, when the same redirection object of the source device is projected to different destination devices, different destination devices create different destination redirectors. For example, when the source device projects a game application to the first destination device and the second destination device, the first destination device creates a first destination redirector based on the source device identifier, the first destination identifier, and the game application identifier, and the second destination device creates a second destination redirector based on the source device identifier, the second destination identifier, and the game application identifier.

In another possible implementation, when multiple screens of a destination device of a source device are projected onto the same redirection object, the destination device creates a single destination redirector, and multiple destination redirection instance identifiers reuse the same destination redirector. For example, when the source device projects a game application to the first screen and the second screen of a first destination device at the same time, the first destination device creates a first destination redirector based on the source device identifier, the first destination identifier, and the game application identifier. At this time, the first destination redirection instance created based on the first destination device identifier and the first screen identifier and the second destination redirection instance created based on the first destination device identifier and the second screen identifier reuse the first destination redirector.

Optionally, when the user sets the media type of the source redirection instance during the redirection process, the source device will create different source redirectors for different media types, and the destination device needs to correspond to the destination redirectors for different media types.

In another possible implementation, the destination device creates at least one destination redirector based on the source redirection instance, the destination device identifier, and the media type corresponding to the target redirection object, wherein different destination redirectors correspond to different media types.

Exemplarily, when the media type corresponding to the target redirection object includes video, the destination device creates a destination video redirector for receiving the video data stream transmitted by the source video redirector; when the media type corresponding to the target redirection object includes audio, the destination device creates a destination audio redirector for receiving the audio data stream transmitted by the source audio redirector; when the media type corresponding to the target redirection object includes control, the destination device creates a destination control redirector for transmitting the control data stream to the source control redirector.

For example, when the source device projects a game application to the first destination device, and the first destination device supports displaying the game application screen, playing the game application sound, and reverse controlling the game application, the destination device creates a destination video redirector based on the source device identifier, the first destination identifier, the game application identifier, and the video type; creates a destination audio redirector based on the source device identifier, the first destination identifier, the game application identifier, and the audio type; and creates a destination control redirector based on the source device identifier, the first destination identifier, the game application identifier, and the control type.

Step 706: The source-end device transmits redirection data to the destination-end device through the link between the source-end redirector and the destination-end redirector.

After the redirector is created, the source device transmits the redirection data to the corresponding destination redirector (the destination redirector and the source redirector correspond to the same source redirection instance) through the source redirector.

In one possible implementation, when a source device and a destination device create different redirectors for different media types, the source device transmits redirection data of at least one media type to the destination device through at least one link between at least one source redirector and one destination redirector, wherein the links between different source redirectors and destination redirectors are used to transmit redirection data of different media types.

For example, when the source device projects a game application to the destination device, and the destination device supports displaying the game application screen and playing the game application sound, the source device transmits the video data stream to the destination video redirector of the destination device through the source video redirector, and transmits the audio data stream to the destination audio redirector of the destination device through the source audio redirector.

Step 707: The destination device receives the redirection data transmitted by the source device through the link between the destination redirector and the source redirector.

In a possible implementation manner, the destination device receives redirection data of at least one media type transmitted by the source device through at least one link between at least one destination redirector and at least one source redirector.

Step 708: When there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device parses the redirection data by using the multiplexing redirection data parser.

The implementation of this step can refer to the above-mentioned step 406, and this embodiment of the present application will not be described in detail.

Step 709: The destination device displays the target redirection object through the target screen based on the parsing result of the redirection data.

In a possible implementation, when the destination device has at least two screens, the destination device determines the target screen according to the target screen identifier in the destination redirection instance, and then displays through the target screen.

In some possible application scenarios, multiple screen projections (from the same source device or different source devices) need to be displayed on the same screen of the destination device at the same time (multiple displays on the same screen); multiple screen projections need to be displayed on different screens of the destination device (different displays on multiple screens); the same projection object of the source device needs to be displayed on different screens of the destination device at the same time (same display on multiple screens).

In order to ensure the accuracy of the screen projection display, in some embodiments, each screen of the target device corresponds to its own screen projection window application. After receiving the redirection data, the target device displays it through the target screen projection window application corresponding to the target screen. Taking the Android system as an example, the screen projection window application can be an Activity component.

Optionally, a window application agent is set up corresponding to the projection window application to control the life cycle of the projection window application.

In addition, for the scenario of multiple displays on the same screen, the destination device needs to reuse the projection window application to display multiple redirected objects in split screens on the same screen.

In a possible implementation manner, the destination device may display the redirection data after receiving the redirection data in the following two possible situations.

1. In response to the target projection window application corresponding to the target screen being in an unstarted state, start the target projection window application; and display the target redirection object through the target projection window application.

Optionally, after determining the target screen to be displayed, the destination device detects whether the target projection window application corresponding to the target screen is in an activated state. If it is in an unactivated state, it indicates that there is currently no other projection object being displayed through the target screen, thereby starting the target projection window application, and then displaying it through the target projection window application. At this time, the target screen only displays the screen of the target redirection object.

2. In response to the target projection window application being in the startup state, the target redirection object is displayed by reusing the target projection window application, wherein after reusing the target projection window application, each projection object is displayed in split screen on the target screen.

Optionally, when it is detected that the target projection window application is in the startup state, it indicates that the target screen is currently displaying other projection objects. The destination device needs to reuse the target projection window application (that is, multiple source redirection instances reuse the same projection window application), and then perform split-screen display of other projection objects currently displayed on the target screen and the target redirection object. Among them, the destination device can perform split-screen control based on the projection direction (horizontal or vertical) of each projection object and the number of projection objects, which is not limited in the embodiments of the present application.

In some embodiments, when all source-end redirection instances within the projection window application exit, the destination-end device ends the life cycle of the projection window application, that is, destroys the projection window application.

In an embodiment of the present application, the source device generates redirection data by a multiplexing redirection data generator, and the destination device parses the redirection data by a multiplexing redirection data parser, thereby reducing the encoding and decoding overhead of the source device and the destination device. In addition, the source device and the destination device respectively create a source redirector and a destination redirector, and transmit the redirection data through the link between the redirectors to ensure the accurate transmission of the redirection data.

In an embodiment of the present application, taking the media type of the target redirection object as video as an example, the multiplexing strategy adopted by the source device and the destination device is explained. As shown in Figure 8, the video encoding, decoding and transmission process may include the following steps.

Step 801: A source-end device creates a source-end redirection instance based on a source-end device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object.

The implementation of this step may refer to the above step 401, and this embodiment will not be described in detail here.

Step 802: The destination device creates a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent the target screen of the destination device.

The implementation of this step can refer to the above-mentioned step 402, and the embodiment of the present application will not be described in detail here.

Step 803: The source device performs video encoding on the pictures corresponding to the same source redirection instance by using a multiplexed video encoder to generate a video data stream.

Optionally, when the same redirected object of the same source device needs to be projected to different screens of the same destination device or different screens of different destination devices, the source device will reuse the video encoder to encode the picture of the source redirection instance to obtain a video data stream, rather than using multiple video encoders to perform video encoding separately according to the number of screens.

Schematically, as shown in Figure 3, when the screen projection application in the source device needs to be simultaneously projected to screen 1 and screen 2 of destination device 1, and screen 3 of destination device 2, the source device multiplexes video encoder 1 for video encoding to obtain a video data stream.

Step 804, when at least two destination redirection instances correspond to the same destination device, the source device creates a source video redirector, or, when at least two destination redirection instances correspond to at least two destination devices, the source device creates at least two source video redirectors.

Exemplarily, as shown in Figure 3, when the screen projection application in the source device needs to be projected to screen 1 and screen 2 of destination device 1, the source device creates a source video redirector. When the screen projection application in the source device needs to be projected to destination device 1 and destination device 2 at the same time, the source device creates two source video redirectors.

Step 805: The destination device creates a destination video redirector.

Exemplarily, as shown in Figure 3, when the screen projection application in the source device needs to be projected to screen 1 and screen 2 of destination device 1, the destination device creates a destination video redirector. When the screen projection application in the source device needs to be projected to screen 1 and screen 2 of destination device 1 and screen 3 of destination device 2 at the same time, the destination device creates a first destination video redirector for destination device 1 and a second destination video redirector for destination device 2.

Step 806: The source device transmits the video data stream to the destination device via the link between the source video redirector and the destination video redirector.

Schematically, as shown in Figure 3, since the screen projection application needs to be projected to destination device 1 and destination device 2, the source device needs to create a first source video redirector for destination device 1 and a second source video redirector for destination device 2, and transmit the encoded video data stream to the first source video redirector and the second source video redirector, and then transmit the video data stream to destination device 1 through the first source video redirector and the first destination video redirector, and transmit the video data stream to destination device 2 through the second source video redirector and the second destination video redirector.

Step 807: The destination device receives the video data stream transmitted by the source device through the link between the destination video redirector and the source video redirector.

Schematically, as shown in Figure 3, the screen projection application of the source device is projected to the destination device 1 and the destination device 2, the destination device 1 receives the video data stream through the link between the first destination video redirector and the first source video redirector, and the destination device 2 receives the video data stream through the link between the second destination video redirector and the second source video redirector.

Step 808: When there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device decodes the video data stream by using a multiplexed video decoder to obtain a picture corresponding to the source redirection instance.

Optionally, when the same redirected object needs to be projected to at least two screens of the same destination device, the destination device will reuse the video decoder to decode the video data stream of the source redirection instance to obtain a video picture, rather than using multiple video decoders to perform video decoding separately according to the number of screens.

Schematically, as shown in FIG. 3 , after receiving the video data stream, the destination device 1 multiplexes the video decoder 1 to perform video decoding, thereby displaying the decoded images through the screen 1 and the screen 2 .

Step 809: The destination device displays the target redirection object through the target screen based on the parsing result of the redirection data.

Schematically, as shown in FIG. 3 , target device 1 displays the video through screen 1 and screen 2 , and target device 2 displays the video through screen 3 .

In the embodiment of the present application, the source device multiplexes the video encoder, and the destination device multiplexes the video decoder, thereby reducing the encoding and decoding overhead in the process of video data stream transmission between the source device and the destination device.

When the media type corresponding to the target redirection object is audio, you can refer to the above steps, which are briefly introduced below.

In another possible scenario, when the media type corresponding to the target redirection object includes audio, the source device needs to collect audio from the redirection object and encode the collected audio to obtain an audio data stream, thereby transmitting the audio data stream to the destination device for parsing and playback by the destination device. In a possible implementation, when the type of the target redirection object is screen mirroring, the source device collects and encodes audio by multiplexing a device-level audio acquisition encoder to obtain an audio data stream, wherein the device-level audio acquisition encoder is used to collect and encode the audio of the device; when the type of the target redirection object is a redirection application, the source device collects and encodes audio by multiplexing an application-level audio acquisition encoder to obtain an audio data stream, wherein the application-level audio acquisition encoder is used to collect and encode the audio of the redirection application.

Furthermore, the source device transmits an audio data stream to the destination device via a link between the source audio redirector and the destination audio redirector. The destination device creates an audio redirector and receives the audio data stream transmitted by the source device via a link between the destination audio redirector and the source audio redirector. After receiving the audio data stream, the destination device performs audio decoding on the audio data stream through a multiplexed audio decoder to obtain the audio corresponding to the source redirection instance. In an embodiment of the present application, the source device reduces the encoding and decoding overhead during the transmission of the audio data stream between the source device and the destination device by multiplexing the audio acquisition encoder and the destination device by multiplexing the audio decoder.

In a possible implementation, when there are at least two destination devices, the source device transmits redirected data to at least two destination devices through at least two links between at least two source redirectors and at least two destination redirectors. Exemplarily, as shown in FIG3, since the screen projection application needs to be projected to destination device 1 and destination device 2, the source device needs to create a first source video redirector for destination device 1 and a second source video redirector for destination device 2, and shunt the encoded video data stream to the first source video redirector and the second source video redirector, and then transmit the video data stream to destination device 1 through the first source video redirector and the first destination video redirector, and transmit the video data stream to destination device 2 through the second source video redirector and the second destination video redirector, thereby realizing the shunt transmission of the video data stream. Due to the adoption of the shunt strategy, the IO overhead of the data stream transmission of the source device and the destination device is reduced.

In a possible implementation, the first destination device and the second destination device asynchronously send screen projection requests to the source device, and the source device successively transmits the video data stream to the first destination device and the second destination device through the link between the first source video redirector and the first destination video redirector, and the link between the second source video redirector and the second destination video redirector. However, in this process, due to the different access times of the first screen projection of the first destination device and the second screen projection of the second destination device, abnormal situations such as screen distortion may occur.

First, we take the Android system MediaCodec encoding module as an example to briefly introduce the video encoding process.

In an embodiment of the present application, the media type of the target redirection object is video, and the MediaCodec encoding module of the source device samples the video screen at a certain frequency and encodes it into multiple I (Intra-coded picture) frames and multiple P (Predictive-coded picture) frames. The I frame is a key frame, which is an independent frame with complete display information. It contains full data and can be decoded independently without relying on other frames, and will serve as a reference value for other frames. The P frame is a non-key frame, which stores the difference between the current frame and the adjacent previous frame (I frame and P frame), and requires reference to the forward frame to form an image. If any frame data of an I frame and multiple P frames before the P frame is lost or erroneous, it will cause the screen to be distorted for a period of time until the next I frame is restored.

In addition, the MediaCodec encoding module will also generate a configuration frame during the encoding process of the video screen, that is, the header information of the video stream, which will record the relevant parameters of the current video stream encoding and save them in the SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) strings. In addition, it contains the information parameters required to initialize the video decoder, such as the encoded profile, level, image width, etc.

Further, in combination with the above-mentioned video encoding process, the abnormal conditions such as screen distortion generated when the first destination device and the second destination device are asynchronously connected are explained. For example, as shown in Figure 9, when the first projection screen of the first destination device is connected, the source device triggers the video encoder, and the configuration frame, I frame, and P frame generated by the video encoder are all sent to the first source video redirector, so the first video stream contains complete configuration frames, I frames, and P frames. Since the second destination device is connected after the first destination device, the second projection screen is usually connected at a certain P frame. At this time, if the second source video redirector receives the video frame, the configuration frame and I frame will be missing, which will cause the destination device to display abnormally. In one possible implementation, if the configuration frame is missing, the destination device decoder cannot decode; in another possible implementation, if the I frame and the P frame after the I frame are missing, the destination device will display a screen distortion. Therefore, in the embodiment of the present application, the second projection screen is used to adopt a shunt frame supplement method to solve the above problem. The following introduces the shunt frame supplement method using the first destination device and the second destination device as examples. In addition, it should be noted that in the embodiment of the present application, there are at least two destination devices, and the embodiment of the present application only takes the first destination device and the second destination device as examples for explanation, and the first destination device is accessed before the second destination device. Please refer to Figure 10, which shows a flowchart of the redirection method during asynchronous access provided by an exemplary embodiment of the present application.

Step 1001: The second destination device sends a screen projection request to the source device through the link between the second destination video redirector and the second source video redirector.

In one possible implementation, the first destination device is accessed before the second destination device, that is, after the first destination device sends a first screen projection request to the source device through the link between the first destination video redirector and the first source video redirector, the second destination device sends a second screen projection request to the source device through the link between the second destination video redirector and the second source video redirector.

Step 1002, in response to the screen projection request of the second destination device, the source device obtains a configuration frame from the redirection context of the source redirection instance through the second source video redirector.

In an embodiment of the present application, the first destination device first sends a screen projection request to the source device. After receiving the screen projection request sent by the first destination device, the source device starts the video encoder, and synchronously reports the configuration frame generated by the video encoder to the redirection context, and sends all frames, namely the configuration frame, I frame, and P frame, to the first source video redirector. Then, the source device transmits the video data stream to the first destination video redirector through the link between the first source video redirector and the first destination video redirector.

The source device maintains the key data and status of each screen projection by reusing the redirection context. Among them, the key data may include various video frames, source device projection parameters, destination device projection parameters, etc., which are not limited in this embodiment of the present application. The status is used to indicate the connection status between the destination device and the source device, such as access status, projection status, exit status, etc., which are not limited in this embodiment of the present application.

When the second destination device sends a screen projection request to the source device, after the source device receives the screen projection request sent by the second destination device, the second source video redirector determines that it is not the first screen projection. Therefore, the source device obtains the configuration frame from the redirection context through the second source video redirector. The configuration frame is generated by the video encoder during the first screen projection.

Step 1003: The source device transmits a configuration frame to the second destination device through the link between the second source video redirector and the second destination video redirector.

Furthermore, after the source device obtains the configuration frame, it transmits the configuration frame to the second destination video redirector of the second destination device solely through the link between the second source video redirector and the second destination video redirector.

Step 1004: The second destination device receives the configuration frame transmitted by the source device.

Further, the second destination device receives the configuration frame through the second destination video redirector.

Step 1005: The second destination device performs video decoding configuration based on the configuration frame.

Furthermore, the video decoder of the second destination device configures decoding parameters of the video decoder by configuring the frame, wherein the decoding parameters may be the width and height of the image.

Step 1006, in response to the screen projection request of the second destination device, the source device obtains the rotated frame from the redirection context through the second source video redirector.

In a possible implementation, since the video screen may rotate horizontally or vertically during projection, a rotation frame is generated during the video encoder encoding the video screen. Since the second destination device is connected after the first destination device, when the second projection is connected, usually in a certain P frame, if the second source video redirector receives the video frame, the rotation frame will be missing, which will cause the display direction of the destination device screen to be inconsistent with the display direction of the source device screen, which may differ by 90°, 180° or 270°.

In an embodiment of the present application, the first destination device first sends a screen projection request to the source device. The source device receives the screen projection request sent by the first destination device, starts the video encoder, and synchronously reports the rotated frames generated by the video encoder to the redirection context, and sends all frames, namely configuration frames, I frames, P frames and rotated frames, to the first source video redirector.

When the second destination device sends a screen projection request to the source device, after the source device receives the screen projection request sent by the second destination device, the second source video redirector determines that it is not the first screen projection. Therefore, the source device obtains the rotated frame from the redirection context through the second source video redirector. The rotated frame is generated by the video encoder during the first screen projection.

Step 1007: The source device transmits the rotated frame to the second destination device via the link between the second source video redirector and the second destination video redirector.

Furthermore, after the source-end device obtains the rotated frame, it transmits the rotated frame to the second destination-end device solely through the link between the second source-end video redirector and the second destination-end video redirector.

Step 1008: The second destination device receives the rotated frame transmitted by the source device.

Further, the second destination device receives the rotated frame through a second destination video redirector.

Step 1009: The second destination device configures the screen display direction based on the rotation frame.

The video decoder of the second destination device configures the display direction of the destination device screen by rotating the frame, which is consistent with the display direction of the source device screen.

Step 1010, starting from the first key frame after the asynchronous access request, the source device transmits the video data stream to the second destination device through the link between the second source video redirector and the second destination video redirector.

In one possible implementation, the source device sends a RequesKeyFrame request to the video encoder, requesting the arrival of the first I frame after the asynchronous access request of the second destination device. Before the I frame arrives, the P frames will be skipped until the I frame arrives. The source device transmits the video data stream to the second destination device through the link between the second source video redirector and the second destination video redirector.

In another possible implementation, the source device does not need to send a request to the video encoder. If the first I frame after the asynchronous access request of the second destination device has not arrived, it waits for the I frame to arrive. Before the I frame arrives, all P frames will be skipped. Until the I frame arrives, the source device transmits the video data stream to the second destination device through the link between the second source video redirector and the second destination redirector.

Step 1011: The second destination device receives the video data stream transmitted by the source device.

Furthermore, the second destination device receives the video data stream transmitted by the source device through the second destination video redirector.

Exemplarily, in conjunction with Figure 10, the above-mentioned method of splitting and supplementing frames is schematically illustrated. The first screen projection of the first destination device is first connected, and the source device triggers the video encoder to generate configuration frames, I frames, P frames and rotation frames, and simultaneously reports the configuration frames and rotation frames to the redirection context. Further, all frames are sent to the first source video redirector, and then the source device transmits the video data stream through the link between the first source video redirector and the first destination video redirector.

The second projection of the second destination device is accessed at a certain P frame. At this time, the second source video redirector of the source device directly obtains the configuration frame and the rotation frame from the redirection context, and transmits the configuration frame and the rotation frame to the second destination separately through the link between the second source video redirector and the second destination video redirector. Further, the source device sends a RequestKeyFrame request to the video encoder, requesting the next I frame to arrive as soon as possible. Before the I frame arrives, the P frame will be skipped until the I frame arrives. Then the source device sends the video data stream to the second destination device through the link between the second source video redirector and the second destination video redirector.

In an embodiment of the present application, when the first destination device is asynchronously connected to the second destination device, a method of splitting and supplementing frames is adopted for the second screen projection of the second destination device, that is, the configuration frame and the rotation frame are obtained from the redirection context by the second source video redirector, and transmitted to the second destination video redirector. The source device waits until the first key frame after the asynchronous access request of the second destination device is used as the starting point, and sends the video data stream to the second destination device to avoid abnormal situations such as screen distortion on the display of the destination device.

In another possible implementation, the first destination device and the second destination device synchronously send a screen projection request to the source device. At this time, the first destination device and the second destination device are accessed concurrently, so the data and status shared by the two will compete and be modified at the same time, thereby affecting the quality of the screen projection. In an embodiment of the present application, the above-mentioned problem is solved by negotiating the screen projection parameters with the first screen projection as the main negotiation object. Please refer to Figure 11, and take the first destination device and the second destination device as an example to introduce the redirection method during concurrent access. It should be noted that in the embodiment of the present application, there are at least two destination devices. The embodiment of the present application only takes the first destination device and the second destination device as an example for explanation. Among them, the first destination device is the first access, and the second destination device is the second access.

Step 1101: The second destination device sends an access request to the source device through the link between the second destination video redirector and the second source video redirector.

The access request is used to establish a connection between the second destination-end video redirector and the second source-end video redirector to create a data stream channel.

Step 1102: In response to the access request of the second destination device, the source device obtains the instance identifier from the redirection context of the source redirection instance through the second source video redirector.

In an embodiment of the present application, the destination device as the first access sends an access request to the source device through the link between the first destination video redirector and the first source video redirector. Further, the source device receives the first access request, and the first source video redirector creates an instance identifier and synchronously reports the instance identifier to the redirection context. At the same time, the source device sends the instance identifier to the first destination device through the link between the first source video redirector and the first destination video redirector. The source device manages the key data and status of each video stream by multiplexing the redirection context.

Therefore, when the source-end device receives the access request sent by the second destination-end device as the second path, the source-end device directly obtains the instance identifier from the redirection context through the second source-end video redirector.

The instance identifier is determined based on the source device identifier and the target redirection object identifier of the target redirection object to be redirected.

Step 1103: The source device sends an instance identifier to the second destination device through a link between the second source video redirector and the second destination video redirector.

Further, after the source device obtains the instance identifier, the second source-end video redirector sends the instance identifier to the second destination-end video redirector through the link between the second source-end video redirector and the second destination-end video redirector.

Step 1104: The second destination device receives the instance identifier of the source redirection instance sent by the source device.

Further, the second destination video redirector receives the instance identifier, determines the source device based on the device identifier, confirms the content displayed by the destination device based on the target redirection object identifier, and further initializes the destination window application.

In a possible implementation, if the first destination device of the first access has initialized the destination window application, the second destination device directly reuses the destination window application without reinitializing the destination window application through the second destination video redirector.

In another possible implementation, if the first destination device accessed through the first path has not initialized the destination window application or the destination window application has been destroyed, the second destination device needs to initialize the destination window application through the second destination video redirector, and other destination devices accessed thereafter can directly reuse the destination window application.

Step 1105: The second destination device sends a parameter acquisition request to the source device through the link between the second destination video redirector and the second source video redirector.

Furthermore, the second destination device sends a parameter acquisition request to the source device for capability negotiation.

Step 1106, in response to the parameter acquisition request of the second destination device, the source device obtains the target screen projection parameters from the redirection context of the source redirection instance through the second source video redirector.

In an embodiment of the present application, a source-end video redirector (including a first source-end video redirector and a second source-end video redirector) of a source-end device receives a parameter acquisition request from a destination-end device, and determines whether it is a first destination-end device accessed through a first path or a second destination-end device accessed through a second path. In a possible implementation, if it is a first destination-end device accessed through a first path, the first source-end video redirector determines the target screen projection parameters based on the source screen projection parameters and the first destination screen projection parameters, and synchronously reports them to the redirection context.

In another possible implementation, if it is a second destination device accessed through a second path, the second source-end video redirector directly obtains the target screen projection parameters from the re-projection screen context.

In addition, it should be noted that, in another possible implementation, the target screen projection parameters can also be determined in real time based on the source screen projection parameters and the current destination screen projection parameters. For example, if it is a first destination device accessed through the first path, the first source video redirector determines the target screen projection parameters based on the source screen projection parameters and the first destination screen projection parameters; if it is a second destination device accessed through the second path, the second source video redirector determines the target screen projection parameters based on the source screen projection parameters and the second destination screen projection parameters.

Among them, the screen projection parameters may be screen resolution, refresh rate, delay, etc., which are not limited in the embodiments of the present application.

Step 1107: The source device sends target screen projection parameters to the second destination device through the link between the second source video redirector and the second destination video redirector.

Further, the source-end device sends the finally determined target screen projection parameters to the second destination-end device through the link between the second source-end video redirector and the second target-end video redirector.

Step 1108: The second destination device receives the target screen projection parameters sent by the source device.

Further, the second destination video redirector of the second destination device receives the target screen projection parameters.

Step 1109: The second destination device initializes the video decoder based on the target screen projection parameters.

Further, after receiving the target screen projection parameters, the second destination video redirector initializes the video decoder according to the target screen projection parameters and sends a screen projection request to the source device.

Exemplarily, the above process is described in conjunction with FIG. 12 .

The second destination device sends an access request to the source device, and the second destination video redirector establishes a connection with the second source video redirector to create a video data stream channel. Further, the second source video redirector obtains the instance identifier by reusing the redirection context, and notifies the second destination video redirector to bind the instance identifier. The second destination video redirector obtains the instance identifier and requests to initialize the destination window application. If the destination window application already exists, it is directly reused. Further, the second destination video redirector sends a parameter acquisition request to the second source video redirector. After receiving the parameter request, the second source video redirector determines whether it is the first destination device accessed as the first route or the second destination device accessed as the second route. If it is the destination device accessed by the first route, the target screen projection parameters are determined by combining the source screen projection parameters and the first destination screen projection parameters, and are synchronously reported to the redirection context. If it is the second destination device accessed by the second route, the second source video redirector directly obtains the target screen projection parameters from the redirection context and sends the target screen projection parameters to the second destination video redirector. Further, the second destination-end video redirector receives the target screen projection parameters, initializes the video decoder according to the parameters, and after the video decoder is initialized, sends a screen projection request to the second source-end video redirector. If the screen projection request is sent to the first destination-end device connected by the first channel, the source-end device creates a video encoder, sends the configuration frame, I frame, P frame and rotation frame generated by the video encoder to the first source-end video redirector, and synchronously reports the configuration frame and the rotation frame to the redirection context. Further, the source-end device sends a video data stream to the first destination-end video redirector through the first source-end video redirector. If the screen projection request is sent to the second destination-end device connected by the second channel, the second source-end video redirector directly obtains the configuration frame and the rotation frame from the redirection context. Further, the source-end device sends a video data stream to the second destination-end video redirector through the second source-end video redirector. After receiving the video data stream, the second destination-end video redirector parses the video data stream through the video decoder and sends it for rendering.

In an embodiment of the present application, when the first destination device and the second destination device are concurrently connected, the second destination device obtains the target screen projection parameters that have been determined based on the source screen projection parameters and the first destination screen projection parameters from the redirection context, and then initializes the video decoder, thereby avoiding modification of shared data when the first destination device and the second destination device are concurrently connected, thereby improving the quality of the displayed image.

It should be noted that the above embodiments are described by taking the interaction process between the source device and the destination device during the redirection process as an example, wherein the steps with the source device as the execution subject can be independently implemented as a redirection method on the source device side, and the steps with the destination device as the execution subject can be independently implemented as a redirection method on the destination device side, which will not be elaborated in the embodiments of the present application.

The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Please refer to Figure 13, which shows a structural block diagram of a redirection device provided by an embodiment of the present application. The device may include:

A creation module 1301, configured to create a source redirection instance based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

A multiplexing module 1302 is used to generate redirection data corresponding to the source redirection instance by a multiplexing redirection data generator when the same source redirection instance corresponds to at least two destination redirection instances, wherein the destination redirection instance is created based on a destination device identifier and a target screen identifier, wherein the destination device identifier is used to characterize the destination device, and the target screen identifier is used to characterize the target screen of the destination device;

The first transmission module 1303 is used to transmit the redirection data to the target device, so that the target device displays the target redirection object through the target screen based on the redirection data.

Optionally, the first transmission module 1303 includes:

a creating unit, configured to create a source-end redirector when at least two of the destination-end redirection instances correspond to the same destination-end device, or to create at least two of the source-end redirectors when at least two of the destination-end redirection instances correspond to at least two destination-end devices;

A transmission unit is used to transmit the redirection data to the destination device through a link between the source redirector and the destination redirector, and the destination redirector is created by the destination device.

Optionally, there are at least two destination devices;

The transmission unit is further used for:

The redirection data is transmitted to at least two destination devices through at least two links between at least two source-end redirectors and at least two destination-end redirectors.

Optionally, the media type corresponding to the target redirection object is video;

The multiplexing module 1302 is further used for:

Performing video encoding on the pictures corresponding to the same source end redirection instance by using a multiplexing video encoder to generate a video data stream;

The transmission unit is further used for:

The video data stream is transmitted to the destination device through a link between the source video redirector and the destination video redirector.

Optionally, the at least two destination devices include a first destination device and a second destination device;

The transmission unit is further used for:

Transmitting the video data stream to the first destination device via a link between the first source-end video redirector and the first destination-end video redirector;

The video data stream is transmitted to the second destination device via a link between the second source-end video redirector and the second destination-end video redirector.

Optionally, the first destination device and the second destination device are accessed asynchronously, and the first destination device is accessed before the second destination device;

The device also includes:

A first acquisition module, configured to respond to a screen projection request of the second destination device, and to acquire a configuration frame from a redirection context of the source redirection instance through the second source video redirector, wherein the configuration frame is reported by the video encoder to the redirection context;

The second transmission module is used to transmit the configuration frame to the second destination device through the link between the second source video redirector and the second destination video redirector, so that the second destination device performs video decoding configuration based on the configuration frame.

Optionally, the transmission unit is further used for:

Taking the first key frame after the asynchronous access request as the starting point, the video data stream is transmitted to the second destination device through the link between the second source video redirector and the second destination video redirector.

Optionally, the device further comprises:

A second acquisition module, configured to obtain a rotated frame from the redirection context through the second source-end video redirector in response to the screen projection request, wherein the rotated frame is reported to the redirection context by the video encoder when the source-end device is rotated;

The third transmission module is used to transmit the rotated frame to the second destination device through the link between the second source video redirector and the second destination video redirector, so that the second destination device configures the screen display direction based on the rotated frame.

Optionally, the first destination device and the second destination device are accessed concurrently, and the first destination device is accessed via a first path, and the second destination device is accessed via a second path;

The device also includes:

A third acquisition module, configured to obtain, in response to an access request from the second destination device, an instance identifier from a redirection context of the source redirection instance through the second source video redirector;

A first sending module, configured to send the instance identifier to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector;

a fourth acquisition module, configured to respond to a parameter acquisition request of the second destination device, and to obtain target screen projection parameters from a redirection context of the source redirection instance through the second source video redirector, wherein the target screen projection parameters are determined by the first source video redirector based on the source screen projection parameters and the destination screen projection parameters of the first destination device, and reported to the redirection context;

The second sending module is used to send the target screen projection parameters to the second destination device through the link between the second source video redirector and the second destination video redirector.

Optionally, the media type corresponding to the target redirection object is audio;

The multiplexing module 1302 is further used for:

When the type of the target redirection object is screen mirroring, audio acquisition and encoding are performed by multiplexing a device-level audio acquisition encoder to obtain an audio data stream;

When the type of the target redirection object is an application, audio acquisition and encoding are performed by multiplexing an application-level audio acquisition encoder to obtain an audio data stream;

The transmission unit is further used for:

The audio data stream is transmitted to the destination device via a link between the source audio redirector and the destination audio redirector.

To summarize, in an embodiment of the present application, during the redirection process, the source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, and the destination device creates a destination redirection instance corresponding to the source redirection instance based on the destination device identifier and the target screen identifier. When the same source redirection instance corresponds to at least two destination instances, the source device generates redirection data through a multiplexing redirection data generator and transmits it to the destination device. The destination device receives the redirector data, parses the redirection data through a multiplexing redirection data parser, and displays the target redirection object through the target screen indicated by the destination redirection instance. By adopting the solution provided in the embodiment of the present application, the encoding overhead of the source device and the decoding overhead of the destination device during the data stream transmission between the source device and the destination device are reduced by multiplexing the redirection data generator and the redirection data parser, thereby improving the data transmission performance in complex redirection scenarios.

Please refer to FIG14, which shows a structural block diagram of a redirection device provided by another embodiment of the present application. The device may include:

A creation module 1401 is used to create a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device;

A first receiving module 1402 is configured to receive redirection data corresponding to a source redirection instance transmitted by a source device, wherein the source redirection instance is created based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object;

A multiplexing module 1403, configured to parse the redirection data by using a multiplexing redirection data parser when at least two destination redirection instances correspond to the same source redirection instance;

The display module 1404 is used to display the target redirection object through the target screen based on the parsing result of the redirection data.

Optionally, the first receiving module 1402 includes:

A creating unit, configured to create a destination redirector, wherein the destination redirector is multiplexed by at least two destination redirection instances;

The receiving unit is used to receive the redirection data transmitted by the source-end device through a link between the destination-end redirector and the source-end redirector, wherein the source-end redirector is created by the source-end device.

Optionally, the media type corresponding to the target redirection object is video;

The creation unit is further used for:

Create a destination video redirector;

The receiving unit is further used for:

Receiving the video data stream transmitted by the source device through the link between the destination video redirector and the source video redirector;

The multiplexing module 1403 is further used for:

The video data stream is decoded by a multiplexing video decoder to obtain a picture corresponding to the source end redirection instance.

Optionally, the device further comprises:

A first sending module, configured to send a screen projection request to the source device through a link between the destination video redirector and the source video redirector;

A second receiving module is used to receive a configuration frame transmitted by the source device, wherein, in the case where other destination devices are connected before the destination device, the configuration frame is obtained by the source video redirector from the redirection context of the source redirection instance;

The first configuration module is used to perform video decoding configuration based on the configuration frame.

Optionally, the device further comprises:

a third receiving module, configured to receive a rotated frame transmitted by the source device, wherein, in the case where other destination devices are connected before the destination device, the rotated frame is obtained by the source video redirector from the redirection context, and the rotated frame is reported to the redirection context by the video encoder when the source device is rotated;

The second configuration module is used to configure the picture display direction based on the rotation frame.

Optionally, the device further comprises:

A second sending module, used for sending an access request to the source device through a link between the destination video redirector and the source video redirector;

a fourth receiving module, configured to receive an instance identifier of the source redirection instance sent by the source device, wherein, in the case where other destination devices are connected before the destination device, the instance identifier is obtained by the source video redirector from the redirection context of the source redirection instance;

A third sending module, configured to send a parameter acquisition request to the source device through a link between the destination video redirector and the source video redirector;

a fifth receiving module, configured to receive target screen projection parameters sent by the source-end device, wherein, in the case where other destination-end devices are connected before the destination-end device, the target screen projection parameters are obtained by the source-end video redirector from the redirection context of the source-end redirection instance;

An initialization module is used to initialize the video decoder based on the target screen projection parameters.

Optionally, the media type corresponding to the target redirection object is audio;

The creation unit is further used for:

Create a destination audio redirector;

The receiving unit is further used for:

Receiving the audio data stream transmitted by the source device through the link between the destination video redirector and the source video redirector;

The multiplexing module 1403 is used to:

The audio data stream is decoded by a multiplexing audio decoder to obtain audio corresponding to the source end redirection instance.

To summarize, in an embodiment of the present application, during the redirection process, the source device creates a source redirection instance based on the source device identifier and the target redirection object identifier, and the destination device creates a destination redirection instance corresponding to the source redirection instance based on the destination device identifier and the target screen identifier. When the same source redirection instance corresponds to at least two destination instances, the source device generates redirection data through a multiplexing redirection data generator and transmits it to the destination device. The destination device receives the redirector data, parses the redirection data through a multiplexing redirection data parser, and displays the target redirection object through the target screen indicated by the destination redirection instance. By adopting the solution provided in the embodiment of the present application, the encoding overhead of the source device and the decoding overhead of the destination device during the data stream transmission between the source device and the destination device are reduced by multiplexing the redirection data generator and the redirection data parser, thereby improving the data transmission performance in complex redirection scenarios.

Please refer to FIG15 , which shows a system architecture block diagram of a redirection system provided by an embodiment of the present application. The system includes: at least one source device 1510 and at least one destination device 1520 .

The source device 1510 is configured to create a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object; when the same source redirection instance corresponds to at least two target redirection instances, generate redirection data corresponding to the source redirection instance by using a multiplexing redirection data generator; and transmit the redirection data to the target device so that the target device displays the target redirection object on a target screen based on the redirection data.

The destination device 1520 is used to create a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent the target screen of the destination device; receive redirection data corresponding to the source redirection instance transmitted by the source device; when there are at least two destination redirection instances corresponding to the same source redirection instance, parse the redirection data by a multiplexing redirection data parser; and display the target redirection object through the target screen based on the parsed result of the redirection data.

It should be noted that the specific process of redirection between the source device 1510 and the destination device 1520 can be found in the above method embodiment, and the embodiment of the present application will not be repeated here.

Please refer to Figure 16, which shows a block diagram of a computer device provided by an exemplary embodiment of the present application. The computer device 1600 can be implemented as a source device or a destination device in the above-mentioned embodiments. The computer device 1600 may include one or more of the following components: a processor 1610 and a memory 1620.

The processor 1610 may include one or more processing cores. The processor 1610 uses various interfaces and lines to connect the various parts of the entire computer device 1600, and executes various functions and processes data of the computer device 1600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1620, and calling data stored in the memory 1620. Optionally, the processor 1610 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1610 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1610, but may be implemented by a separate chip.

The memory 1620 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1620 includes a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book), etc., created according to the use of the computer device 1600.

In addition, those skilled in the art can understand that the structure of the computer device 1600 shown in the above drawings does not constitute a limitation on the computer device, and the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the computer device 1600 also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

An embodiment of the present application further provides a computer-readable storage medium, which stores at least one program code, and the program code is loaded and executed by a processor to implement the redirection method described in the above embodiments.

The embodiment of the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the redirection method provided in various optional implementations of the above aspects.

It should be understood that the "multiple" mentioned in this article refers to two or more than two. "And/or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims (23)

1. A redirection method, characterized in that the method is used for a source device, and the method comprises: Creating a source redirection instance based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object; In the case where the same source redirection instance corresponds to at least two destination redirection instances, the redirection data corresponding to the source redirection instance is generated by a multiplexing redirection data generator, and the destination redirection instance is created based on a destination device identifier and a target screen identifier, wherein the destination device identifier is used to characterize the destination device, and the target screen identifier is used to characterize the target screen of the destination device; The redirection data is transmitted to the target device, so that the target device parses the redirection data through a multiplexing redirection data parser and displays the target redirection object through the target screen. 2. The method according to claim 1, wherein transmitting the redirection data to the destination device comprises: In the case where at least two of the destination redirection instances correspond to the same destination device, creating a source redirector, or, in the case where at least two of the destination redirection instances correspond to at least two destination devices, creating at least two of the source redirectors; The redirection data is transmitted to the destination device via a link between the source redirector and the destination redirector, and the destination redirector is created by the destination device. 3. The method according to claim 2, characterized in that there are at least two destination devices; The transmitting the redirection data to the destination device through the link between the source redirector and the destination redirector comprises: The redirection data is transmitted to at least two destination devices through at least two links between at least two source-end redirectors and at least two destination-end redirectors. 4. The method according to claim 2, characterized in that the media type corresponding to the target redirection object is video; The step of generating the redirection data corresponding to the source end redirection instance by multiplexing the redirection data generator includes: Performing video encoding on the pictures corresponding to the same source end redirection instance by using a multiplexing video encoder to generate a video data stream; The transmitting the redirection data to the destination device through the link between the source redirector and the destination redirector comprises: The video data stream is transmitted to the destination device through a link between the source video redirector and the destination video redirector. 5. The method according to claim 4, characterized in that the at least two destination devices include a first destination device and a second destination device; The transmitting the video data stream to the destination device through a link between the source video redirector and the destination video redirector comprises: Transmitting the video data stream to the first destination device via a link between the first source-end video redirector and the first destination-end video redirector; The video data stream is transmitted to the second destination device via a link between the second source-end video redirector and the second destination-end video redirector. 6. The method according to claim 5, characterized in that the first destination device and the second destination device are accessed asynchronously, and the first destination device is accessed before the second destination device; The method further comprises: In response to the screen projection request of the second destination device, obtaining a configuration frame from the redirection context of the source redirection instance through the second source video redirector, and the configuration frame is reported to the redirection context by the video encoder; The configuration frame is transmitted to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device performs video decoding configuration based on the configuration frame. 7. The method according to claim 6, wherein the transmitting the video data stream to the second destination device through a link between the second source video redirector and the second destination video redirector comprises: Taking the first key frame after the asynchronous access request as the starting point, the video data stream is transmitted to the second destination device through the link between the second source video redirector and the second destination video redirector. 8. The method according to claim 6, characterized in that the method further comprises: In response to the screen projection request, obtaining a rotated frame from the redirection context through the second source-end video redirector, wherein the rotated frame is reported to the redirection context by the video encoder when the source-end device is rotated; The rotated frame is transmitted to the second destination device via a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device configures a screen display direction based on the rotated frame. 9. The method according to claim 5, characterized in that the first destination device and the second destination device are accessed concurrently, and the first destination device is accessed by a first path, and the second destination device is accessed by a second path; The method further comprises: In response to the access request of the second destination device, obtaining, by the second source-end video redirector, an instance identifier from a redirection context of the source-end redirection instance; Sending the instance identifier to the second destination device via a link between the second source-end video redirector and the second destination-end video redirector; In response to a parameter acquisition request of the second destination device, the target screen projection parameters are acquired from the redirection context of the source redirection instance by the second source video redirector, and the target screen projection parameters are determined by the first source video redirector based on the source screen projection parameters and the destination screen projection parameters of the first destination device, and reported to the redirection context; The target screen projection parameters are sent to the second destination device through the link between the second source video redirector and the second destination video redirector. 10. The method according to claim 2, characterized in that the media type corresponding to the target redirection object is audio; The step of generating the redirection data corresponding to the source end redirection instance by multiplexing the redirection data generator includes: When the type of the target redirection object is screen mirroring, audio acquisition and encoding are performed by multiplexing a device-level audio acquisition encoder to obtain an audio data stream; When the type of the target redirection object is an application, audio acquisition and encoding are performed by multiplexing an application-level audio acquisition encoder to obtain an audio data stream; The transmitting the redirection data to the destination device through the link between the source redirector and the destination redirector comprises: The audio data stream is transmitted to the destination device via a link between the source audio redirector and the destination audio redirector. 11. A redirection method, characterized in that the method is used for a destination device, and the method comprises: Creating a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device; Receiving redirection data corresponding to a source redirection instance transmitted by a source device, wherein the source redirection instance is created based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object; In the case that there are at least two destination redirection instances corresponding to the same source redirection instance, parsing the redirection data by using a multiplexing redirection data parser; Based on the parsing result of the redirection data, the target redirection object is displayed through the target screen. 12. The method according to claim 11, characterized in that the receiving of redirection data corresponding to the source redirection instance transmitted by the source device comprises: Creating a destination redirector, where the destination redirector is multiplexed by at least two of the destination redirection instances; The redirection data transmitted by the source-end device is received through a link between the destination-end redirector and the source-end redirector, and the source-end redirector is created by the source-end device. 13. The method according to claim 12, characterized in that the media type corresponding to the target redirection object is video; The step of creating a destination redirector includes: Create a destination video redirector; The receiving the redirection data transmitted by the source-end device through a link between the destination-end redirector and the source-end redirector comprises: Receiving the video data stream transmitted by the source device through the link between the destination video redirector and the source video redirector; The step of parsing the redirection data by multiplexing the redirection data parser comprises: The video data stream is decoded by a multiplexing video decoder to obtain a picture corresponding to the source end redirection instance. 14. The method according to claim 13, characterized in that before receiving the video data stream transmitted by the source device, the method further comprises: Sending a screen projection request to the source device through a link between the destination video redirector and the source video redirector; Receiving a configuration frame transmitted by the source device, wherein, in the case where other destination devices are connected before the destination device, the configuration frame is obtained by the source video redirector from the redirection context of the source redirection instance; Video decoding configuration is performed based on the configuration frame. 15. The method according to claim 14, characterized in that the method further comprises: Receiving a rotated frame transmitted by the source device, wherein, in the case where there is another destination device connected before the destination device, the rotated frame is obtained by the source video redirector from the redirection context, and the rotated frame is reported to the redirection context by the video encoder when the source device is rotated; The screen display direction is configured based on the rotation frame. 16. The method according to claim 13, characterized in that the method further comprises: Sending an access request to the source device via a link between the destination video redirector and the source video redirector; Receiving an instance identifier of the source redirection instance sent by the source device, wherein, in the case where other destination devices are connected before the destination device, the instance identifier is obtained by the source video redirector from the redirection context of the source redirection instance; Sending a parameter acquisition request to the source device through a link between the destination video redirector and the source video redirector; Receiving target screen projection parameters sent by the source device, wherein, in the case where other target devices are connected before the target device, the target screen projection parameters are obtained by the source video redirector from the redirection context of the source redirection instance; Initialize the video decoder based on the target screen projection parameters. 17. The method according to claim 12, characterized in that the media type corresponding to the target redirection object is audio; The step of creating a destination redirector includes: Create a destination audio redirector; The receiving the redirection data transmitted by the source-end device through a link between the destination-end redirector and the source-end redirector comprises: Receiving the audio data stream transmitted by the source device through the link between the destination video redirector and the source video redirector; The step of parsing the redirection data by multiplexing the redirection data parser comprises: The audio data stream is decoded by a multiplexing audio decoder to obtain audio corresponding to the source end redirection instance. 18. A redirection device, characterized in that the device is used for a source device, and the device comprises: A creation module, used to create a source redirection instance based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object; a multiplexing module, configured to generate redirection data corresponding to the source redirection instance by a multiplexing redirection data generator when the same source redirection instance corresponds to at least two destination redirection instances, wherein the destination redirection instance is created based on a destination device identifier and a target screen identifier, wherein the destination device identifier is used to characterize the destination device, and the target screen identifier is used to characterize the target screen of the destination device; The first transmission module is used to transmit the redirection data to the target device, so that the target device parses the redirection data through a multiplexing redirection data parser and displays the target redirection object through the target screen. 19. A redirection device, characterized in that the device is used for a destination device, and the device comprises: A creation module, used to create a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device; A first receiving module, configured to receive redirection data corresponding to a source redirection instance transmitted by a source device, wherein the source redirection instance is created based on a source device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to represent a target redirection object; a multiplexing module, configured to parse the redirection data by a multiplexing redirection data parser when at least two destination redirection instances correspond to the same source redirection instance; A display module is used to display the target redirection object through the target screen based on the parsing result of the redirection data. 20. A redirection system, characterized in that the redirection system comprises: at least one source device and at least one destination device; The source-end device is used to create a source-end redirection instance based on a source-end device identifier and a target redirection object identifier, wherein the target redirection object identifier is used to characterize a target redirection object; when the same source-end redirection instance corresponds to at least two target-end redirection instances, generate redirection data corresponding to the source-end redirection instance by a multiplexing redirection data generator; transmit the redirection data to a target-end device, so that the target-end device parses the redirection data by a multiplexing redirection data parser, and displays the target redirection object by a target screen; The destination device is used to create a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used to represent a target screen of the destination device; receive redirection data corresponding to the source redirection instance transmitted by the source device; when there are at least two destination redirection instances corresponding to the same source redirection instance, parse the redirection data by a multiplexing redirection data parser; and based on the parsing result of the redirection data, display the target redirection object through the target screen. 21. A computer device, characterized in that the computer device includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the redirection method as described in any one of claims 1 to 10, or to implement the redirection method as described in any one of claims 11 to 17. 22. A computer-readable storage medium, characterized in that at least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by a processor to implement the redirection method as described in any one of claims 1 to 10, or to implement the redirection method as described in any one of claims 11 to 17. 23. A computer program product, characterized in that the computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the redirection method as described in any one of claims 1 to 10, or the redirection method as described in any one of claims 11 to 17.