CN113434103B - Multi-screen interaction method and system based on screen virtualization and application thereof - Google Patents

Abstract

The invention discloses a multi-screen interaction method based on screen virtualization, which is characterized in that when a system is initialized, a logical screen and application mapping configuration table and a logical screen and physical screen mapping configuration initial table are automatically established; when the system is upgraded, the system configuration updating module automatically updates the two tables; a user establishes a mapping relation between a logical screen and a physical screen through a logical screen and physical screen mapping configurator, the user manually configures the physical screen mapped by each logical screen, and after the configuration is finished, the mapping relation is recorded in a logical screen and physical screen mapping configuration table; the physical screen distributor reads the configuration information of the logical screen and the physical screen mapping configuration table, establishes communication and transmission channels with each physical screen, and projects the teaching content and application thereof corresponding to a certain logical screen onto a specified physical screen. The invention conveniently establishes the flexible association and various configuration combinations between the teaching content and the application thereof and the physical screen through the screen virtualization technology.

Description

A multi-screen interaction method, system and application based on screen virtualization

technical field

The present invention relates to a technology of multi-screen interaction, in particular to a "screen virtualization" (ScreenVirtualization) technology, which automatically configures multiple "logical screens" according to the content and the applications required for playback, and then distributes them to each "physical screen" The multi-screen interaction method, system and application thereof.

Background technique

With the continuous advancement of educational informatization, a large number of new technologies are applied in classrooms, and multimedia classrooms and their devices are constantly developing towards digital and intelligent future classrooms. At present, many school classrooms are equipped with electronic display screens, and many classrooms are equipped with multiple electronic display screens.

Multi-screen interaction refers to the technology of sharing display content on multiple display screens at the same time, that is, after multiple display screens are connected through a dedicated network and equipment, various content (such as text, pictures, etc.) can be simultaneously performed between the display screens. , audio, video, etc.) display and control operations.

Most of the existing multi-screen interactive systems realize the "same screen" display of multiple display screens in the same classroom, that is, the content displayed on the multiple display screens is the same. For example, Apple's AirPlay technology can transmit video mirroring on Apple devices to devices that support AirPlay, and the content on the iPad screen in the teacher's hand can be directly projected to the computer screen and electronic display screen through wireless WiFi, so as to realize the realization of iPad The screen, computer screen, and electronic display screen are three screens on the same screen. This function allows classrooms or students to operate iPads anywhere in the classroom, and share related operations with other people in the classroom.

There are some shortcomings and limitations in the display of multiple electronic displays on the same screen:

1. Most of the existing electronic display screens and the source equipment use direct physical or network connections. The display screen only displays the content of the source equipment screen, and cannot perform intelligent screen projection display according to the teaching content and the applications required for playback. ;

2. The teaching process involves a variety of different types of teaching content. The frequent switching of these content will cause the "intermittent" display of the teaching content, which will cut off the context of the teaching content, and will also affect the teaching effect;

3. Multiple electronic display screens display the same content on the same screen, which results in a waste of hardware resources to a certain extent, and also limits the play of multi-screen interaction capabilities.

SUMMARY OF THE INVENTION

In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a multi-screen interaction method based on screen virtualization, comprising the following steps:

Step 1: When the system is initialized, Table 1 and Table 2 are automatically created, the table 1 is the logical screen and application mapping configuration table, and the table 2 is the initial table for the mapping configuration between the logical screen and the physical screen; the mapping configuration between the logical screen and the physical screen The configuration interface pops up on the device, and the configuration interface automatically reads Table 2 and loads the preset logical screen combination; the Table 2 automatically scans the number of physical screens on site and displays it in the configuration interface;

Step 2: When the system is upgraded, the system configuration update module automatically detects whether there is an update in Table 1 and Table 2, and if so, automatically updates Table 1 and/or Table 2;

Step 3: The user establishes the mapping relationship between the logical screen and the physical screen through the logical screen and physical screen mapping configurator according to the user's own use requirements, combined with the physical screen location information on the deployment site, and the user manually configures the physical screen mapped by each logical screen. , after the configuration is completed, the mapping relationship is recorded in Table 3; the table 3 is the mapping configuration table between the logical screen and the physical screen;

Step 4: The physical screen distributor reads the configuration information in Table 3, establishes a communication and transmission channel with each physical screen, and projects the content and application corresponding to a certain logical screen to a designated physical screen.

In the present invention, the table 1 records the mapping relationship between a group of logical screens and content playback applications, and the table 2 records the mapping relationship between a group of logical screens recommended by the system and their physical screens. In the system initialization stage, the physical screen records The item is a null value, and the table 3 records the mapping relationship between each logical screen and the physical screen after the manual configuration by the user is completed.

In the present invention, the classification of logical screens, the selection of content playback applications, and the establishment of mapping relationships are automatically generated by new system recommendations; the logical screen classifications include file display screens, video playback screens, source code editing screens, and source code running screens; the The selection of content playback applications includes PowerPoint and Keynote for playing files, Windows Media Player and Quick Time for playing videos, Visual Studio and Eclipse for source code editing, and Windows Command Line and Unix Shell for source code running.

In the present invention, each physical screen is set to project a logical screen; let N be a logical screen and M be a physical screen, when N<=M, assign at least one physical screen to each logical screen; when N> When M, some logical screens still need to be switched on the physical screen; after the system is configured for the first time, the user will automatically perform multi-screen distribution and projection of content according to the configuration every time the user casts the screen, or the user manually reconfigures the system.

Based on the above system, the present invention also proposes a multi-screen interactive system based on screen virtualization. The multi-screen interactive system includes: a system configuration update module, a logical screen and physical screen mapping configurator, and a physical screen distributor; wherein,

The system configuration update module automatically updates Table 1 and/or Table 2 when the system is upgraded;

The logical screen and physical screen mapping configurator establishes the mapping relationship between the logical screen and the physical screen, and is recorded in Table 3;

After the physical screen distributor is initialized and configured, the system automatically reads the mapping configuration information in Table 3 every time the user casts a screen, loads the mapping relationship recorded in each row, and establishes a communication and transmission channel with each physical screen. The content of a logical screen and its applications are projected onto the specified physical screen.

The invention also proposes an application of a multi-screen interactive system based on screen virtualization in a teaching scene. A method for automatic distribution and multi-screen interaction of teaching content and applications required for playback based on screen virtualization, and a system for multi-screen interactive teaching through the method.

A technical problem solved by the present invention is how to establish an effective mapping between the teaching content and its display application and multiple physical display screens, so as to facilitate the realization of intelligent screen projection according to the teaching content and its display application, and avoid the "intermittent" caused by frequent switching. Display, strengthen the coherence of teaching content display, at the same time, using the hardware advantages of multiple physical screens, specific types of teaching content and applications can be displayed on specific physical screens according to user needs.

In addition, the technical problems solved include: perception and classification of teaching content and application scenarios, distribution configuration between application scenarios and physical display screens, and implementation of the entire system.

The present invention proposes multi-screen projection, that is, multiple logical screens are mapped to multiple physical screens instead of being on a single physical screen. First, different logical screens are divided according to different types of projected content and different playing applications, and then the logical screens are projected onto different physical screens according to user needs.

The main innovation point of the present invention is "screen virtualization", which mainly includes two layers of mapping, that is, the mapping between the logical screen and the application, and the mapping between the logical screen and the physical screen. Through the mapping between logical screens and applications, the classification of different types of content and different types of applications is realized, and the automatic distribution of content is also realized; in addition, through the mapping between logical screens and physical screens, the flexibility of different projection contents and physical screens is realized. The configuration also meets the various needs of users.

The beneficial effects of the present invention are:

The invention breaks through the traditional way of establishing a direct physical or network connection between the electronic display screen and the source device, and automatically establishes the mapping between the teaching content and its application and the logical screen through the "screen virtualization" technology, and then the user himself Configure the mapping between the logical screen and the physical screen, so as to easily establish the flexible association and various configuration combinations between the teaching content and its application and the physical screen. Even for the same content, when the applications and scenarios required for display are different, different logic screens can be configured for display, which effectively avoids frequent switching and "intermittent" display when displaying content, and enhances the consistency of content display. , improving the teaching effect, and at the same time, it can also make full use of the hardware resources of multiple physical screens.

In addition, through a one-time system configuration, each screen projection will automatically perceive different types of teaching content and applications, automatically assign logical screens, and project them to the designated physical screens according to the preset mapping relationship without manual intervention. Realize automatic distribution and intelligent screen projection of teaching content.

In addition to being applicable to teaching content and the applications required for playback, the screen virtualization technology proposed in the present invention is also applicable to wider application scenarios where there are multiple types of screen projection content or multiple playback applications.

Description of drawings

Fig. 1 is the screen virtualization method and system architecture diagram of the present invention;

Figure 2 is a behavioral model of system configuration and operation;

Fig. 3 is the realization scheme of embodiment 1 of the present invention;

FIG. 4 is an implementation scheme of Embodiment 2 of the present invention.

Detailed ways

The present invention will be further described in detail with reference to the following specific embodiments and accompanying drawings. Except for the content specifically mentioned below, the process, conditions, experimental methods, etc. for implementing the present invention are all common knowledge and common knowledge in the field, and the present invention is not particularly limited.

The present invention proposes a method and system for automatic distribution and multi-screen interaction based on screen virtualization of teaching content and applications required for playback, and proposes the concepts of "screen virtualization" and "logical screen". The mapping relationship of the logical screen, and then the user configures the mapping relationship between the logical screen and the physical screen according to the needs, and finally the system distributes and projects the teaching content and its application to the corresponding physical screen according to the established two-layer mapping relationship.

The architecture diagram of the method and its system, as shown in Figure 1, the system configuration method and specific steps are as follows:

The system of the present invention includes: a system configuration update module, a logical screen and a physical screen mapping configurator, and a physical screen distributor.

The system configuration update module: when the system is upgraded, the system configuration update module will automatically update "Table 1" and/or "Table 2", for example, a new application program is added to a certain logical screen classification, or a new application The combination leads to the need to add new logic screens, etc.

The logical screen and physical screen mapping configurator: establishes the mapping relationship between the logical screen and the physical screen, and records it in "Table 3". The mapping configuration between the logical screen and the physical screen is configured by the user according to the needs, and can be modified repeatedly.

The mapping relationship is mainly realized through the configuration of the mapping table. The details are as follows: When the system is initialized, users (such as system configuration administrators, operators, teachers, etc.) manually configure the mapping relationship between each physical screen and each logical screen through the logical screen and physical screen mapping configurator, and record the mapping relationship between each physical screen and each logical screen. Table 3 "Logical Screen and Physical Screen Mapping Configuration Table".

The physical screen allocator: After initializing the configuration, the system will automatically read the mapping configuration information in Table 3 "Logical Screen and Physical Screen Mapping Configuration Table" every time the user casts a screen, load the mapping relationship of each row of records, and establish and The communication and transmission channels of each physical screen project the teaching content and its application corresponding to a logical screen to the designated physical screen.

The "table one" refers to the configuration table of logical screen and application mapping;

The "Table 2" refers to the initial table of mapping configuration between the logical screen and the physical screen;

The "Table 3" refers to the mapping configuration table between the logical screen and the physical screen.

Based on the above system, the present invention also proposes a method of use, comprising the following steps:

Step 1: When the system is initialized, the "Logical Screen and Physical Screen Mapping Configurator" pops up a configuration interface, which automatically reads Table 2 "Logical Screen and Physical Screen Mapping Configuration Initial Table" and loads the preset logical screen combination; then , Table 2 "Logical Screen and Physical Screen Mapping Configuration Initial Table" automatically scans the number of physical screens on site and displays them in the configuration interface, and automatically creates two tables: Table 1 "Logical Screen and Application Mapping Configuration Table" and Table 2 "Logical Screen and Application Mapping Configuration Table" Screen and Physical Screen Mapping Configuration Initial Table".

"Table 1" records the mapping relationship between a group of logic screens and common teaching content playback applications. The logic screen classification, application selection and mapping relationship establishment are automatically generated by the system recommendation, and users do not need additional configuration.

"Table 2" records the mapping relationship between a group of logical screens recommended by the system and their physical screens. Since the system is currently in the initialization stage, the physical screens have not been sensed and configured, so the record items of the physical screens are null.

The classification of the logic screen includes a courseware display screen, a video playback screen, a source code editing screen, a source code running screen, etc.;

The selection of the application includes PowerPoint, Keynote, etc. for playing courseware, Windows Media Player, Quick Time, etc. for playing video, Visual Studio, Eclipse, etc. for source code editing, Windows command line, Unix, etc. for source code operation. Shell et al.

Step 2: "Table 1" and "Table 2" are automatically generated and loaded by the system in the initialization phase by default. When the system is upgraded, the "System Configuration Update" module will automatically detect whether there are updates in "Table 1" and "Table 2", and if so, it will automatically update "Table 1" and/or "Table 2", for example, if there is a new An application program is added to a certain logical screen category, or a new application combination leads to the need to add a new logical screen, etc.

Step 3: The user establishes the mapping relationship between the logical screen and the physical screen through the "Logical Screen and Physical Screen Mapping Configurator" according to his own use requirements, combined with the physical screen location and other information on the deployment classroom site, and the user manually configures each logical screen After the mapped physical screen is configured, the mapping relationship is recorded in Table 3 "Logical Screen and Physical Screen Mapping Configuration Table". The configuration of "Table 3" is configured by users according to their needs, and can be modified repeatedly. Each physical screen is set to project a logical screen. The projection refers to mapping a logical relationship, and the projection is the action of the logical relationship materialized in the scene. When N<=M, N is a logical screen, M is a physical screen, and at least one physical screen can be allocated to each logical screen, so as to avoid frequent switching and "intermittent" display; when N>M, some logical screens are still Need to switch on the physical screen.

Step 4: The "Physical Screen Distributor" reads the configuration information of "Table 3", establishes a communication and transmission channel with each physical screen, and projects the teaching content and application corresponding to a logical screen to the designated physical screen.

The behavior model of system configuration and operation is shown in Figure 2. After the system is configured for the first time, the user will automatically perform multi-screen distribution and projection of teaching content according to the configuration for each screen projection operation. Users can also reconfigure the system when needed. .

Users can repeatedly screen projection. After an initial configuration, the repeated/repeated screen projection behavior in daily use is a screen projection behavior that does not require manual configuration and automatically loads the set configuration.

Example 1

Taking the teaching of a computer programming course as an example, the teaching content to be displayed includes courseware, video, source code, etc. Here, the courseware may be in .ppt, .pptx, .key and other formats, and different applications, such as PowerPoint or Keynote, may be used to explain the courseware. Video also includes a variety of video formats, such as .avi, .mp4, .mov, etc., and video playback may use different applications, such as Windows Media Player, Apple QuickTime, etc. In addition, the teaching of computer programming courses involves two application scenarios: source code editing and running. Tools such as Visual Studio and Eclipse may be used to edit program source code, while Windows command line, Unix Shell and other tools may be used to run program source code. . The types of teaching content mentioned above are different, and the applications and scenarios involved in the display are also different. If there is only one display screen in the classroom, or multiple display screens display the same content on the same screen, the screen needs to be switched frequently, resulting in "intermittent" display.

According to the method and system of the present invention, as shown in Figure 3, when the system is initialized for the first time, the system configurations of "Table 1" and "Table 2" are loaded, and four logical screens are defined, corresponding to courseware display, video playback, Source code editing and source code running. Among them, the logic screen 1 is used to play the courseware, run applications such as PowerPoint and Keynote, and continue to play the courseware during the class to ensure the continuity of the courseware presentation. Logic screen 2 is used to play video, run Windows Media Player, QuickTime and other applications, when video needs to be played, it will be presented separately, without affecting the display of courseware in "Logic Screen 1". Logic screen 3 is used for source code editing, running Visual Studio, Eclipse and other applications; logic screen 4 is used for source code execution, running Windows command line, Unix Shell and other applications; even if the same set of source code, use "Logic Screen 3" when editing, running When using "Logical Screen 4", they do not affect each other and do not need to be switched.

The above concept of "logic screen" provides a layer of operational logic to classify specific application scenarios. Next, put the "logical screen" on the "physical screen". As shown in Figure 3, assuming there are four corresponding physical screens, the system automatically recommends a set of mappings between logical screens and physical screens. Users can manually configure according to the classroom site conditions and their own needs, and the configuration is completed. Then start casting. Each physical screen will display the content of the assigned logical screen without interfering with each other, effectively avoiding the "intermittent" display problem caused by frequent screen switching.

Example 2

Taking any course teaching that includes video as an example, the teaching content to be displayed is a set of courseware. The main application for playing the courseware is PowerPoint. At the same time, some slides in the courseware may contain videos, and the main application for playing the video is Windows Media Player. , because the full screen will be occupied when playing the video, resulting in a certain degree of "intermittent" display. For this reason, when the system is configured, it is specified that all courseware explanations are placed on "Logical Screen 1" and projected to the classrooms in tandem. On "Physical Screen 1" and "Physical Screen 2", when a video needs to be played during the courseware explanation, the video and its playback application are automatically assigned to "Logical Screen 2" and projected to the classrooms on the left and right of the classroom. "Physical screen 3" and "Physical screen 4", so as to realize the dynamic perception and automatic screen projection of teaching content. In addition, the content of a logical screen is distributed to two physical screens in different positions, and the multi-screen hardware is rationally utilized. At the same time, it also guarantees the screen projection effect of each position in the classroom.

The protection content of the present invention is not limited to the above embodiments. Variations and advantages that can occur to those skilled in the art without departing from the spirit and scope of the inventive concept are included in the present invention, and the appended claims are the scope of protection.

Claims (6)

1. a multi-screen interaction method based on screen virtualization, is characterized in that, comprises the steps: Step 1: When the system is initialized, Table 1 and Table 2 are automatically created, the table 1 is the logical screen and application mapping configuration table, and the table 2 is the initial table for the mapping configuration between the logical screen and the physical screen; the mapping configuration between the logical screen and the physical screen The configuration interface pops up on the device, and the configuration interface automatically reads Table 2 and loads the preset logical screen combination; the Table 2 automatically scans the number of physical screens on site and displays it in the configuration interface; Step 2: When the system is upgraded, the system configuration update module automatically detects whether there is an update in Table 1 and Table 2, and if so, automatically updates Table 1 and/or Table 2; Step 3: The user establishes the mapping relationship between the logical screen and the physical screen through the logical screen and physical screen mapping configurator according to the user's own use requirements, combined with the physical screen location information on the deployment site, and the user manually configures the physical screen mapped by each logical screen. , after the configuration is completed, the mapping relationship is recorded in Table 3; the table 3 is the mapping configuration table between the logical screen and the physical screen; Step 4: The physical screen distributor reads the configuration information in Table 3, establishes a communication and transmission channel with each physical screen, and projects the content and application corresponding to a certain logical screen to a designated physical screen. 2. The multi-screen interaction method according to claim 1, wherein the table 1 records the mapping relationship between a group of logical screens and content playback applications, and the table 2 records a group of logical screens recommended by the system and their physical properties. The mapping relationship of the screen, in the system initialization stage, the record item of the physical screen is a null value, and the table 3 records the mapping relationship between each logical screen and the physical screen after the manual configuration by the user is completed. 3. The multi-screen interaction method according to claim 1, wherein the logical screen classification, the selection of the content playback application and the establishment of the mapping relationship are automatically generated by a new system recommendation; the logical screen classification comprises a file display screen, Video playback screen, source code editing screen, and source code running screen; the selection of the content playback application includes PowerPoint and Keynote for playing files, Windows Media Player and Quick Time for playing videos, and Visual Studio and Eclipse for source editing. , Windows command line, Unix Shell for source code running. 4. The multi-screen interaction method according to claim 1, wherein each physical screen is set to project a logical screen; let N be a logical screen, and M be a physical screen, when N<=M, Assign at least one physical screen to each logical screen; when N>M, some logical screens still need to be switched on the physical screen; after the system is configured for the first time, the user will automatically perform multi-screen distribution and projection of content according to the configuration every time the user performs a screen projection operation , or the user manually reconfigures the system. 5. A multi-screen interaction system based on screen virtualization, wherein the system adopts the multi-screen interaction method according to any one of claims 1-4, and the multi-screen interaction system comprises: a system configuration Update module, logical screen and physical screen mapping configurator, physical screen distributor; among them, The system configuration update module automatically updates Table 1 and/or Table 2 when the system is upgraded; The logical screen and physical screen mapping configurator establishes the mapping relationship between the logical screen and the physical screen, and is recorded in Table 3; After the physical screen distributor is initialized and configured, the system automatically reads the mapping configuration information in Table 3 every time the user casts a screen, loads the mapping relationship recorded in each row, and establishes a communication and transmission channel with each physical screen. The content of a logical screen and its applications are projected onto the specified physical screen. 6. The application of the multi-screen interactive system based on screen virtualization according to claim 5 in a teaching scene.

Images