CN116954809B - Remote control methods, devices, systems and electronic devices based on Android devices - Google Patents

Abstract

The invention relates to the technical field of intelligent equipment control, and provides a remote control method, device and system based on android equipment and electronic equipment, wherein the method comprises the following steps: acquiring a local input event based on interface information of a signal source; determining that the local input event is a right mouse button event; when the right button event of the mouse forms a complete event group, encapsulating the right button event of the mouse into a protocol data packet; and controlling the interface of the signal source according to the protocol data packet. According to the method, the acquired local input events are analyzed and identified, the complete event group comprising the right mouse button pressing event and the right mouse button lifting event is packaged into the protocol data packet, and according to the protocol data packet, the signal source is controlled without special setting of an Android system or modification of an Android system kernel in the process, and can be extracted to the complete and continuous right mouse button event group, so that the integrity and continuity of the right mouse button events are ensured, and the right mouse button control sensitivity is effectively improved.

Description

Remote control methods, devices, systems and electronic devices based on Android devices

Technical field

The present invention relates to the technical field of intelligent device control, and in particular to a remote control method, device, system and electronic device based on Android devices.

Background technique

Currently, Android is mainly targeted at mobile devices, and its support for mice is relatively weak. By default, the right-click event of the external mouse will be responded to by pressing the return key, and the application page will return to the previous page step by step, and finally return to the system desktop. For example, if an external mouse is connected to a tablet mobile device under the Android system, since there is no subsequent mouse lift action, the right-click release (lift) command cannot be sent to the tablet mobile device, which will cause the operating system of the tablet mobile device to right-click to stick (press no action). reaction), the right-click operation is insensitive, which is actually caused by the missing command due to the loss of the lift command.

Therefore, it is very necessary to solve the problem that the existing technology cannot obtain continuous right-click events of the mouse, which causes the right-click to stick and the right-click operation to be insensitive during remote control.

Contents of the invention

The present invention provides a remote control method, device, system and electronic equipment based on Android devices to overcome the defects of the existing technology due to the inability to obtain continuous right-click events of the mouse, resulting in right-click adhesion and insensitive right-click operations during control. Ensure the integrity and continuity of mouse right-click events.

On the one hand, the present invention provides a remote control method based on an Android device, which includes: collecting local input events based on interface information of a signal source; determining that the local input event is a right mouse button event; when the right mouse button event constitutes a complete When forming an event group, the right mouse button event is encapsulated into a protocol data packet; the interface of the signal source is controlled according to the protocol data packet; wherein the complete event group includes a right mouse button press event, a right mouse button up event event and any number of mouse movement events.

Further, the local input event includes a first type of event and a second type of event, the first type of event includes a keyboard event and the right mouse button event, and the second type of event includes a touch point event and a mouse event, so The mouse events include the right mouse button event and other mouse events.

Further, the collection of local input events includes: collecting the first type of events based on the first listening modifier in the Android composable interface; collecting all the events based on the second listening modifier in the Android composable interface. Describe the second type of event.

Further, the collecting local input events previously included: adding a new listening modifier to the Android composable interface, the listening modifier is used to collect the local input events; wherein, the new listening modifier The qualifier includes the first listening modifier and the second listening modifier.

Further, determining that the first type of event is the keyboard event or the right mouse button event includes: when the return key value of the first type of event is KEYCODE_BACK, determining that the first type of event is the KEYCODE_BACK event. The right mouse button event is described; when the return key value of the first type event is not KEYCODE_BACK, it is determined that the first type event is the keyboard event.

Further, determining that the second type of event is the contact event or the mouse event includes: when the pressure value of the second type of event is zero, determining that the second type of event is the Mouse event; when the pressure value of the second type of event is not zero, the second type of event is determined to be the contact event.

Further, determining that the second type of event is the right mouse button event or the other mouse event includes: the MotionEvent.BUTTON_BACK value of the callback variable corresponding to the callback function in the Android composable interface is a preset value. In this case, it is determined that the second type of event is the right mouse button event; in the case where the MotionEvent.BUTTON_BACK value of the callback variable corresponding to the callback function in the Android composable interface is a value other than the preset value, it is determined The second type of event is the other mouse event.

Further, determining that the second type of event is the right mouse button event includes: discarding the right mouse button event.

Further, determining that the first type of event is the right mouse button event includes: collecting the right mouse button press event, and responding to the right mouse button press event on the operating interface of the Android device; Collect the right mouse button lift event, and respond to the right mouse button lift event on the operating interface of the Android device; after responding to the right mouse button press event, the operating interface of the Android device responds to the right mouse button press event. Respond to the above right mouse button lift event.

Further, determining that the second type of event is the other mouse event includes: collecting the mouse movement event, and responding to the mouse movement event on the operating interface of the Android device; the Android device The operation interface of the Android device responds to the mouse move event after responding to the mouse right button press event; the operation interface of the Android device responds to the mouse right button lift after responding to the mouse move event. Respond to events.

In a second aspect, the present invention also provides a remote control device based on an Android device, applying any of the above remote control methods based on an Android device. The device includes: an input event collection module for Interface information, collect local input events; input event analysis module, used to determine that the local input event is a right mouse button event; input event encapsulation module, used to when the right mouse button event constitutes a complete event group, the mouse right button event The right-click event is encapsulated into a protocol data packet; a remote control module is used to control the interface of the signal source according to the protocol data packet; wherein the complete event group includes a right-click mouse button press event and a right mouse button lift-up event. and any number of mouse movement events.

In a third aspect, the present invention also provides a remote control system based on an Android device, including: a keyboard and mouse device, an Android device, a collection control terminal and a signal source; the keyboard and mouse device is connected to the Android device, and the keyboard and mouse device is connected to the Android device. The device is used to input local input events to the Android device; the Android device is connected to the collection control terminal, and during operation, the remote control method based on the Android device as described in any one of the above is used to obtain the protocol data packet; The acquisition control end is connected to the signal source, and the acquisition control end is used to transmit the protocol data packet to the signal source, and transmit the interface information of the signal source to the Android device; the signal source Used to control interface operations according to the received protocol data packet and transmit interface information to the collection control terminal.

In a fourth aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, any one of the above is implemented. The above-mentioned remote control method based on Android devices.

In a fifth aspect, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the remote control method based on an Android device is implemented as described in any of the above. .

The invention provides a remote control method based on an Android device, which collects local input events through interface information based on signal sources, and determines the local input event to be a right mouse button event. When the right mouse button event is enough to constitute a complete event group , encapsulate the right mouse button event into a protocol data packet, and then control the interface of the signal source according to the protocol data packet. This method analyzes and identifies the collected local input events, and encapsulates the complete event group including the right mouse button press event and the right mouse button lift event into a protocol data packet to control the signal source according to the protocol data packet. This process There is no need to specially set up the Android system or modify the Android system kernel. The signal source can extract a complete and continuous group of right mouse button events, which overcomes the inability of the existing technology to obtain continuous right mouse button events, which causes right button sticking during remote control. , the defect of insensitive right-click operation is ensured to ensure the integrity and continuity of the mouse right-click event.

Description of drawings

In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are of the present invention. For some embodiments of the invention, those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of main body interaction of the remote control method based on Android devices provided by the present invention;

Figure 2 is a schematic flow chart of the remote control method based on Android devices provided by the present invention;

Figure 3 is a schematic diagram of local input event categories of the remote control method based on Android devices provided by the present invention;

Figure 4 is a schematic diagram of the overall processing of the remote control method based on Android devices provided by the present invention;

Figure 5 is an overall framework diagram of the remote control method based on Android devices provided by the present invention;

Figure 6 is a schematic structural diagram of a remote control device based on Android devices provided by the present invention;

Figure 7 is a system schematic diagram of the remote control system based on Android devices provided by the present invention;

Figure 8 is a schematic structural diagram of the electronic device provided by the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention more clear, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

It should be noted that currently Android is mainly oriented to mobile devices, and the support for mice is relatively weak. By default, the right-click event of an external mouse will be responded to by pressing the return key, and the application page will return to the previous page step by step. Finally return to the system desktop.

In this regard, in the Android system, the existing technology provides a way to change the right mouse button response event, that is, through system settings or by modifying the Android system kernel and rebuilding the system. However, this method of modifying the right mouse button response event is not universal. Very poor promotion.

In the Android Compose UI (Android Composable Interface Framework) system, developers can monitor and extract mouse events through modifiers, but it cannot take into account all events of the right mouse button. When remotely controlling a PC device through an external mouse of an Android device, since there is no subsequent mouse lift action, the right-click release command cannot be sent to the PC device, which will cause the PC operating system's right-click to stick and make the right-click operation insensitive. In fact, The command is missed due to the loss of the lift command.

Considering this, in view of the problems involved in the aforementioned application scenarios, the present invention proposes a remote control method based on Android devices. This method does not require customized settings for the Android system, nor does it require modifying the Android system kernel or rebuilding the system. , you can achieve the continuity and integrity of mouse right-click events.

Here, before the detailed description of the remote control method based on Android devices provided by the present invention, the execution subjects involved in the method will first be briefly described.

Figure 1 shows a schematic diagram of main body interaction of the remote control method based on Android devices provided by the present invention.

As shown in Figure 1, when executing the remote control method based on an Android device provided by the present invention, four entities need to be involved, namely, a keyboard and mouse device, an Android device, a collection control terminal, and a signal source.

Among them, the connection between the keyboard and mouse device and the Android device can be a physical connection (wired connection) or a wireless connection (such as Bluetooth connection), which is not specifically limited here.

There is a network connection between the Android device and the acquisition control terminal, as well as between the acquisition control terminal and the signal source. It can be understood that the acquisition control terminal plays the role of an intermediate bridge between the Android device and the signal source to achieve an indirect connection and information interaction between the Android device and the signal source. Among them, the collection control terminal can be an independent device or a module integrated into an Android device.

In the actual information interaction process, first, establish the connection between the four and realize the connectivity between the four.

Secondly, when the Android device has control requirements, it will forward the control requirements to the collection control terminal. After receiving the control requirements of the Android device (such as the number and IP address of the signal source), the collection control terminal will obtain the corresponding signal according to the control requirements. Source related information (such as the interface information of the signal source), and forward the obtained related information to the Android device.

Then, after the Android device receives relevant information about the signal source, it can remotely control the signal source through the mouse. At this time, the mouse inputs a local input event, and the Android device collects and analyzes the local input event. After receiving the complete local input After the event group is generated, it is encapsulated into a protocol data packet and forwarded to the collection control end.

Immediately after, after receiving the protocol data packet, the acquisition control terminal forwards it to the corresponding signal source, thereby realizing the control operation of the Android terminal (i.e., the Android device) on the remote PC terminal (i.e., the signal source).

At the same time, the signal source will also transmit the interface information after the control operation to the Android device.

On Android devices, the interface information received can be processed online. For example, the document content of the signal source can be modified, and the modified content can be directly synchronized to the corresponding signal source. For another example, you can take a screenshot of the signal source interface and annotate information on the screenshot to make decisions based on the annotated content. The annotated content may not be synchronized to the corresponding signal source.

It should also be noted that the signal source may be one or multiple, and is not specifically limited here.

Based on the above content, FIG. 2 shows a schematic flow chart of the remote control method based on Android devices provided by the present invention. Specifically, as shown in Figure 2, the method includes:

S210: Collect local input events based on the interface information of the signal source.

It should be noted that the remote control method based on the Android device provided by the present invention uses the Android device as the execution subject.

It can be understood that based on the control requirements of the Android device, the acquisition control terminal obtains the corresponding interface information from the corresponding signal source and forwards it to the Android device.

After receiving the interface information of the signal source, the Android device can determine the control instructions for the interface information of the signal source, thereby controlling the interface information of the signal source.

In this step, to control the interface information of the signal source, you first need to collect local input events. Specifically, this can be collected by adding a new listening modifier in the Compose component of the Android system (Android composable interface).

Generally speaking, remote signal sources can be controlled through an external mouse, a keyboard (physical keyboard or on-screen soft keyboard), or contacts (touch screen, gestures).

Therefore, based on the interface information of the signal source, local input events are collected. The local input events may be mouse events, keyboard events, or contact events.

It should also be noted that the number of signal sources can be one or multiple at the same time, and is not specifically limited here.

In a specific embodiment, there are 9 signal sources, the interface information of the 9 signal sources is received at the same time, and the interface information of the 9 signal sources is controlled.

It should be noted that the interface information of the signal source refers to the screen content of the signal source displayed on the display screen of the PC. The picture content of the signal source can be, for example, a page of a document, a photo, a frame of a video being played, etc.

S220: Determine the local input event to be the right mouse button event.

It can be understood that, on the basis of collecting local input events based on the interface information of the signal source in step S210, it is necessary to further determine whether the collected local input event is a right mouse button event. In the case where it is determined that the local input event is a right mouse button event, , execute the subsequent step S230.

Determining that the local input event is a right mouse button event can be achieved by setting a combination of different modifiers in the Compose component of the Android system.

For example, an IF statement can be used to determine that the local input event is a right mouse button event when a certain attribute of the local input event meets a preset condition.

The right mouse button event includes a right mouse button press event, a right mouse button lift event, and a mouse movement event between the press event and the lift event.

In a specific embodiment, whether the local input event is a right mouse button event is determined by whether the return key value corresponding to the local input event is KEYCODE_BACK (return key value). When the key value corresponding to the local input event is KEYCODE_BACK, it is determined that the local input event is indeed a right mouse button event.

It should be noted that KEYCODE_BACK here can be one or more preset values, for example, set to a value of 1, 3 or 5, which is not specifically limited here.

It should also be noted that when the local input event is of other event types, such as keyboard events and contact events, the existing event processing flow can handle it better, so it will not be elaborated here.

S230: When the right mouse button event constitutes a complete event group, the right mouse button event is encapsulated into a protocol data packet.

It can be understood that, on the basis of determining that the local input event is a right mouse button event in step S220, further, if the collected right mouse button events are enough to constitute a complete event group, the collected right mouse button events are encapsulated and packaged. , get the corresponding protocol data packet.

A complete event group refers to a complete and continuous set of instruction events that trigger the triggering of a local input event and terminate the triggering of the local input event. Among them, the complete event group includes a right mouse button down event, a right mouse button up event, and any number of mouse movement events.

In a specific embodiment, a right mouse button press event is first collected, and then a right mouse button lift event is collected. The two can form a complete event group, and the right mouse button press event and the right mouse button lift event are Encapsulated into a protocol data packet.

In another specific embodiment, a right mouse button press event is first collected, and then five mouse movement events and a right mouse button lift event are collected. At this time, a complete event group can be formed, and a The right mouse button press event, 5 mouse movement events and a right mouse button lift event are encapsulated into a protocol data packet.

That is to say, in the complete event group, the number of right mouse button down events and right mouse button up events is fixed, one each, but the number of mouse movement events is not fixed and can be 0 or more.

Based on the collection time, after the right mouse button press event is collected and before the right mouse button lift event is collected, the mouse movement events that occur in between are included in the complete event group.

S240, according to the protocol data packet, controls the interface of the signal source.

It can be understood that, in step S230, when the right mouse button event constitutes a complete event group, the right mouse button event is encapsulated into a protocol data packet, and further, the protocol data packet is transmitted to one or more signals to be controlled through the network. source, thereby realizing the control operation of the remote signal source by the Android device.

Specifically, the protocol data packet can be transmitted to the corresponding signal source through UDP (User Datagram Protocol) or WebSocket (a protocol for full-duplex communication on a single Transmission Control Protocol connection).

In this embodiment, the local input event is collected through the interface information based on the signal source, and the local input event is determined to be the right mouse button event. When the right mouse button event is enough to form a complete event group, the right mouse button event is encapsulated into a protocol Data packets, and then control the interface of the signal source according to the protocol data packets. This method analyzes and identifies the collected local input events, and encapsulates the complete event group including the right mouse button press event and the right mouse button lift event into a protocol data packet to control the signal source according to the protocol data packet. This process There is no need to specially set up the Android system or modify the Android system kernel. The signal source can extract a complete and continuous group of right mouse button events, which overcomes the inability of the existing technology to obtain continuous right mouse button events, which causes right button sticking during remote control. , the defect of insensitive right-click operation is ensured to ensure the integrity and continuity of the mouse right-click event.

Through the present invention, the Android device can accurately identify keyboard events, mouse right-click events, and contact events. The above types of events can be used interactively according to the user's needs. Users are not limited to a certain control mode. Increase the convenience of interaction between users and Android devices.

Based on the above embodiments, further, FIG. 3 shows a schematic diagram of local input event categories of the remote control method based on Android devices provided by the present invention.

As shown in Figure 3, local input events include the first type of events and the second type of events. The first type of events include keyboard events and right mouse button events. The second type of events include touch events and mouse events. Mouse events include right mouse button events. and other mouse events.

It can be understood that in this embodiment, local input events are divided into two major categories at a time, namely, first type events and second type events.

In the first category of events, analyzing them can be subdivided into keyboard events and right mouse button events; in the second category of events, analyzing them can be subdivided into touch events and mouse events.

Among them, it should be noted that the mouse events in the second type of events include not only right mouse button events, but also other mouse events other than right mouse button events, such as left mouse button events, scrolling events, etc.

In order to avoid multiple responses to the same event when the right mouse button event is collected when the first type of event and the second type of event are collected, only the right mouse button event in the first type of event is responded to, and the right mouse button event in the second type of event is responded to. The right mouse button event (discontinuity exists) is not processed. Specifically, when it is analyzed in the second type of event that the local input event is a right mouse button event, in order to avoid redundant data and discontinuity in the right mouse button event group, the right mouse button event is directly discarded without any processing.

For the above different types of local input events, they can be collected by adding new different listening modifiers in the Compose component.

Expand, collect local input events, including: collecting the first type of events based on the first listening modifier in the Compose component (Android composable interface); collecting the second type of events based on the second listening modifier in the Compose component.

In a specific embodiment, the first listening modifier is Modifier.onKeyEvent modifier, and the second listening modifier is Modifier.pointerInput modifier.

That is to say, for the first type of events, you can directly call the Modifier.onKeyEvent modifier (modifier for monitoring and processing key input events) in the Compose component for collection. Further analysis and identification of the first type of events will be provided in subsequent embodiments. Expand in.

For the second type of events, you can directly call the Modifier.pointerInput modifier (modifier for monitoring and processing pointer input events) in the Compose component to collect. Further analysis and identification of the second type of events will be carried out in subsequent embodiments.

It should be noted that by default, the local input events collected by the Modifier.onKeyEvent modifier in the Compose component are keyboard events, and the local input events collected by the Modifier.pointerInput modifier in the Compose component are contact events.

In other words, these two modifiers do not serve mouse events by default. In this way, you can avoid the situation where the right mouse button press event is responded to the return key by default.

It should also be noted that before collecting local input events, the Android device will add new listening modifiers to the Compose component, such as the first listening modifier and the second listening modifier mentioned above, to take advantage of the newly added listening modifiers to collect Different local input events.

In this embodiment, local input events are divided into first-type events and second-type events. The first-type events can be specifically divided into keyboard events and mouse right-click events. The second-type events can be specifically divided into contact events and Mouse events, and use the Modifier.onKeyEvent modifier and Modifier.pointerInput modifier in the Compose component to collect the first type of events and the second type of events, and then analyze and identify the local input events, including the right mouse button press The complete event group of the down event and the right mouse button up event is encapsulated into a protocol data packet, thereby controlling the signal source according to the protocol data packet. This process does not require special setting of the Android system, nor does it need to modify the Android system kernel. The signal source can be extracted The complete and continuous right-click event group overcomes the shortcomings of the existing technology due to the inability to obtain continuous right-click events, resulting in right-click sticking and insensitive right-click operations during remote control, and ensures the integrity and continuity of right-click events. .

On the basis of the above embodiment, further determining that the first type of event is a keyboard event or a right mouse button event includes: when the return key value of the first type of event is KEYCODE_BACK, determining that the first type of event is a right mouse button event. ; When the return key value of the first type of event is not KEYCODE_BACK, determine the first type of event to be a keyboard event.

It can be understood that local input events are collected based on the interface information of the signal source. On this basis, the event type of the local input event needs to be further determined, so that different event types are processed according to the corresponding event processing flow.

In this embodiment, the main focus is on the analysis and identification of keyboard events and right mouse button events in the first type of events.

Expanding, if the local input event is collected through the Modifier.onKeyEvent modifier in the Compose component, it can be preliminarily determined that the local input event belongs to the first type of event rather than the second type of event.

On the basis of determining that the local input event is the first type of event, the local input event may be a keyboard event or a right mouse button event, so further analysis and identification are required.

Specifically, in the first type of event, each keyboard value corresponds to a corresponding return key value. The return key value corresponding to the right mouse button event is KEYCODE_BACK. Therefore, the local input can be determined through the return key value of the first type of event. Whether the event is a right mouse button event.

That is, when the return key value of the first type of event is KEYCODE_BACK, the type of the local input event is the right mouse button event. Furthermore, the current right mouse button event can be judged to be the right mouse button press by the type attribute value of the first type of event. event (KeyEventType.KeyDown), or the right mouse button lift event (KeyEventType.KeyUp).

When the return key value of the first type of event is not KEYCODE_BACK, it is determined that the type of the local input event is a keyboard event, and the process is processed according to the keyboard event processing flow.

It should be noted that both the first type of event and the second type of event include the right mouse button event. However, when the first type of event and the second type of event both analyze and obtain the right mouse button event, the second type of event analyzes the mouse right button event. The right-click event will be discarded directly without any operation. In other words, the right-click event will only be processed accordingly when it is analyzed in the first type of event (distribute the instruction of the right-click event to the branch of the keyboard event, and handle accordingly).

In this embodiment, the first type of event is subdivided by the return key value of the first type of event. When it is determined that the return key value of the first type of event is KEYCODE_BACK, the local input event is a right mouse button event, and When it is determined that the return key value of the first type of event is not KEYCODE_BACK, the local input event is a keyboard event, so that different processing is performed according to the actual event type of the local input event. In this process, the complete event group including the right mouse button down event and the right mouse button up event is encapsulated into a protocol data packet, thereby controlling the signal source according to the protocol data packet. This process does not require special setting of the Android system, nor does it require Modifying the Android system kernel, the signal source can extract a complete and continuous right-click event group of the mouse, overcoming the shortcomings of the existing technology that cannot obtain continuous right-click events, resulting in right-click sticking and insensitive right-click operations during remote control, ensuring Improved the integrity and continuity of mouse right-click events.

On the basis of the above embodiment, further determining that the second type of event is a touch event or a mouse event includes: when the pressure value of the second type of event is zero, determining that the second type of event is a mouse event; When the pressure value of the second type of event is not zero, the second type of event is determined to be a contact event.

It can be understood that local input events are collected based on the interface information of the signal source. On this basis, the event type of the local input event needs to be further determined, so that different event types are processed according to the corresponding event processing flow.

In this embodiment, the focus is mainly on the analysis and identification of touch events and mouse events in the second type of events.

Expanding, if the local input event is collected through the Modifier.pointerInput modifier in the Compose component, it can be initially determined that the local input event belongs to the second type of event, not the first type of event.

On the basis of determining that the local input event is the second type of event, the local input event may be a contact event, it may be a right mouse button event, or it may be other mouse events other than the right mouse button event, such as a left mouse button event, so it is necessary Perform further analysis and identification on it.

Specifically, in the second type of event, the contact event corresponds to a pressure value, and the pressure value corresponding to the mouse event is zero. Considering this, this implementation uses the pressure value of the second type of event (calling the attribute of the second type of event value isBackPress), determines whether the local input event is a mouse event.

That is, when the pressure value of the second type of event is zero, the type of the local input event is a mouse event. At this time, the mouse event may be a right mouse button event or other mouse events other than the right mouse button event.

Further, determining that the second type of event is a right mouse button event or other mouse event includes: determining that the second type of event is a mouse when the MotionEvent.BUTTON_BACK value of the callback variable corresponding to the callback function in the Compose component is a preset value. Right-click event; when the MotionEvent.BUTTON_BACK value of the callback variable corresponding to the callback function in the Compose component is a value other than the preset value, the second type of event is determined to be other mouse events.

Specifically, the specific type of mouse event is determined by whether the MotionEvent.BUTTON_BACK value of the callback variable (event.buttons.isBackPressed) of the callback function (awaitPointerEventScope) in the Compose component (Android composable interface) is a preset value.

When the MotionEvent.BUTTON_BACK value of the callback variable (event.buttons.isBackPressed) of the callback function (awaitPointerEventScope) in the Compose component is the preset value, the mouse event is specifically a right mouse button event, and the right mouse button event must be a right mouse button Press event.

When the MotionEvent.BUTTON_BACK value of the callback variable (event.buttons.isBackPressed) of the callback function (awaitPointerEventScope) in the Compose component is a value other than the preset value, the mouse event is a mouse event other than the right mouse button event.

Among them, the preset value can be set according to the actual situation, and is not specifically limited here.

For example, in one embodiment, the preset value is 8.

When the pressure value of the second type of event is not zero, it means that the generation of the local input event is accompanied by a pressure value, and the local input event is a contact event.

It should also be noted that when the second type of event analysis finds that the local input event is a right mouse button event, the branch of the second type of event will not process the right mouse button event, but directly discard the event.

In this embodiment, the second type of event is subdivided by the pressure value of the second type of event. When it is determined that the pressure value of the second type of event is zero, the second type of event is a mouse event. When the pressure value of the second type of event is not zero, the second type of event is a contact event, so different processing is performed according to the actual event type of the second type of event. In this process, the complete event group including the right mouse button down event and the right mouse button up event is encapsulated into a protocol data packet, thereby controlling the signal source according to the protocol data packet. This process does not require special setting of the Android system, nor does it require Modifying the Android system kernel, the signal source can extract a complete and continuous right-click event group of the mouse, overcoming the shortcomings of the existing technology that cannot obtain continuous right-click events, resulting in right-click sticking and insensitive right-click operations during remote control, ensuring Improved the integrity and continuity of mouse right-click events.

Based on the above embodiment, it is further determined that the pressure value of the second type of event is zero, and based on the callback function in the Compose component, it is determined that the local input event is a right mouse button event, and then includes: discarding the right mouse button event and not executing Other operations.

It can be understood that the above-mentioned first type of event and second type of event can be understood as two branches, namely the first branch and the second branch.

Among them, the first branch is used to process right mouse button events and keyboard events, while the second branch is used to process other mouse events other than right mouse button events, as well as touch events.

When the local input event is a right mouse button event, the first branch and the second branch will collect the event at the same time. Since the second branch cannot obtain a complete and continuous right mouse button event, it also avoids redundant repeated data transmission. Therefore, when the local input event collected by the second branch is a right mouse button event, no processing is performed on the right mouse button event, but the event is directly discarded in this branch.

According to the above, when the local input event is a right mouse button event, it is only processed by the first branch, and the second branch directly discards it without performing any other operations.

In this embodiment, by analyzing the local input event as a right mouse button event in the second type of event, discarding the right mouse button event in the second type of event and not performing any other operations on it, it is possible to avoid the right mouse button event in the second type of event. The first type of event and the second type of event are repeatedly transmitted and processed, which avoids redundant data repeated transmission, and also avoids the discontinuity of the right mouse button event group in the second type of event.

In addition, FIG. 4 shows an overall processing schematic diagram of the remote control method based on Android devices provided by the present invention.

As shown in Figure 4, first, local input events are collected. During preliminary collection, the local input events may be the first type of events or the second type of events.

In the branch of the first type of event, the local input event can be further determined to be a keyboard event or a right mouse button event in the first type of event through the return key value of the first type of event.

Specifically, when the return key value of the first type of event is KEYCODE_BACK, the local input event is a mouse right-click event; when the return key value of the first type of event is not KEYCODE_BACK, the local input event is a keyboard event. Therefore, according to the mouse The processing flow of right-click events and keyboard events executes subsequent steps.

In the branch of the second type of event, the local input event can be further determined to be a mouse event or a touch event in the second type of event through the pressure value of the second type of event.

Specifically, when the pressure value of the second type of event is zero, the local input event is a mouse event; when the pressure value of the second type of event is not zero, the local input event is a contact event.

It should be noted that in the branch of the second type of event, when the local input event is determined to be the right mouse button, in order to avoid redundant repeated transmission of data and to avoid the discontinuity of the right mouse button event group, it is directly discarded ( The right mouse button event is processed in the branch to which the first type of event belongs), and no processing is performed on it.

For events other than the right mouse button event, the existing technology can handle them better.

For the right mouse button event, when the collected right mouse button events are enough to form a complete event group, the collected right mouse button events will be encapsulated into protocol data packets, and the protocol data packets will be transmitted to the controlled one or Multiple signal sources to control at least one signal source.

Figure 5 shows the overall framework diagram of the remote control method based on Android devices provided by the present invention.

As shown in Figure 5, for local input events, it mainly includes the collection and analysis of local input events. Collection can be achieved through the Modifier.onKeyEvent modifier and Modifier.pointerInput modifier in the Compose component, and analysis can be achieved through local input The return key value and pressure value of the event are implemented.

After analyzing the local input event and obtaining its actual event type, it can be processed according to the corresponding processing flow.

For the right mouse button event, when it is determined that the collected right mouse button event is enough to form a complete event group, the collected right mouse button event is encapsulated in the protocol to obtain a protocol data packet, and then the data is sent according to the network transmission protocol. , transmit the protocol data packet to at least one remote signal source, thereby enabling the Android device to control at least one signal source.

Further, after determining that the first type of event is a right mouse button event, the method provided by the present invention further includes: collecting a right mouse button press event, and responding to the right mouse button press event on the operating interface of the Android device. And, collect the right mouse button lift event, and respond to the right mouse button lift event on the operating interface of the Android device. Also, after the operation interface of the Android device responds to the right mouse button press event, it then responds to the right mouse button lift event. It can be seen that the right mouse button event is triggered based on pressing and terminated based on lifting, and the order of the two is fixed.

Furthermore, after determining that the second type of event is another mouse event, the method provided by the present invention also includes: collecting mouse movement events, and responding to the mouse movement events on the operating interface of the Android device. Then, the operation interface of the Android device responds to the mouse movement event after responding to the mouse right button press event. Finally, after responding to the mouse movement event, the operating interface of the Android device responds to the right mouse button lift event. It can be seen that the right mouse button event is triggered based on pressing and terminated based on lifting. The mouse movement event belongs to the process action between the above-mentioned pressing and lifting, and the order of the three is fixed. It should be noted that the pressing and lifting of the right mouse button are distributed to the keyboard event branch for response processing; while the movement of the mouse is distributed to the contact event branch for response processing, and multiple event types work interactively.

Figure 6 shows a schematic structural diagram of a remote control device based on an Android device provided by the present invention. As shown in Figure 6, the device includes:

The input event collection module 610 is used to collect local input events based on the interface information of the signal source; the input event analysis module 620 is used to determine that the local input event is a mouse right-click event; the input event encapsulation module 630 is used to when the When the right mouse button event constitutes a complete event group, the right mouse button event is encapsulated into a protocol data packet; the remote control module 640 is used to control the interface of the signal source according to the protocol data packet; wherein, the complete event The group includes a right mouse button down event, a right mouse button up event, and any number of mouse move events.

In this embodiment, the input event collection module 610 collects local input events based on the interface information of the signal source. The input event analysis module 620 determines that the local input event is a right mouse button event. The input event encapsulation module 630 determines that the right mouse button event is sufficient to constitute In the case of a complete event group, the right mouse button event is encapsulated into a protocol data packet, and then the remote control module 640 controls the interface of the signal source according to the protocol data packet. The device analyzes and identifies the collected local input events, and encapsulates the complete event group including the right mouse button press event and the right mouse button lift event into a protocol data packet to control the signal source according to the protocol data packet. This process There is no need to specially set up the Android system or modify the Android system kernel. The signal source can extract a complete and continuous group of right mouse button events, which overcomes the inability of the existing technology to obtain continuous right mouse button events, which causes right button sticking during remote control. , the defect of insensitive right-click operation is ensured to ensure the integrity and continuity of the mouse right-click event.

It should be noted that the remote control device based on the Android device provided in this embodiment and the remote control method based on the Android device described above can be referenced correspondingly and will not be described again here.

Figure 7 shows a system schematic diagram of the remote control system based on Android devices provided by the present invention. As shown in Figure 7, the system includes:

Keyboard and mouse device 710, Android device 720, collection control terminal 730 and signal source 740; the keyboard and mouse device 710 is connected to the Android device 720, and the keyboard and mouse device 710 is used to input local input events to the Android device 720; the Android device 720 is connected to the collection The control terminal 730 is connected, and during operation, the remote control method based on the Android device described in any of the above embodiments is used to obtain the protocol data packet; the collection control terminal 730 is connected to the signal source 740, and the collection control terminal is used to transmit the protocol data packet to Signal source 740, and transmit the interface information of the signal source 740 to the Android device 720; the signal source 740 is used to control the interface operation according to the received protocol data packet, and transmit the interface information to the collection control terminal 730.

The keyboard and mouse device 710 includes a mouse, a physical keyboard, an on-screen soft keyboard, etc. The Android device 720 can be an Android terminal, an Android iPad or an Android computer, which is not specifically limited here.

The signal source 740 can be a mobile phone, a computer, a camera, a monitor, etc., and is not specifically limited here.

It should be noted that the number of signal sources 740 in this embodiment is at least one, which is not specifically limited here.

It should also be noted that the remote control system based on the Android device provided in this embodiment can correspond to the remote control method based on the Android device described above, and will not be elaborated here.

Moreover, the remote control system based on Android devices, the remote control method based on Android devices and the remote control device based on Android devices described above can be applied in teaching application scenarios. Specifically, the teacher can view For the Android device 720, the student terminal can be regarded as the signal source 740. The teacher terminal can annotate and explain the homework, reports, etc. of at least one student terminal through an external mouse, and synchronize the comments and explanation content to the student terminal, so that the student terminal Able to modify and adjust according to the teacher's comments and explanations.

In this embodiment, the system analyzes and identifies the collected local input events, and encapsulates the complete event group including the right mouse button press event and the right mouse button lift event into a protocol data packet, so as to use the protocol data packet according to the protocol data packet. Controlling the signal source does not require special settings for the Android system or modification of the Android system kernel. The signal source can extract a complete and continuous group of right mouse button events, overcoming the inability of the existing technology to obtain continuous right mouse button events. The defects of right-click sticking and insensitive right-click operation occur during remote control, ensuring the integrity and continuity of mouse right-click events.

Figure 8 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 8, the electronic device may include: a processor (processor) 810, a communications interface (communications interface) 820, a memory (memory) 830 and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete communication with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute a remote control method based on the Android device. The method includes: collecting local input events based on the interface information of the signal source; determining that the local input event is a right mouse button event; when When the right mouse button event constitutes a complete event group, the right mouse button event is encapsulated into a protocol data packet; the interface of the signal source is controlled according to the protocol data packet; wherein the complete event group includes a right mouse button Press events, a right mouse button up event, and any number of mouse move events.

In addition, the above-mentioned logical instructions in the memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .

On the other hand, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. The computer program is implemented when executed by the processor to execute the remote control method based on the Android device provided by the above methods. The method includes: collecting local input events based on interface information of a signal source; determining that the local input event is a right mouse button event; and when the right mouse button event constitutes a complete event group, encapsulating the right mouse button event into protocol data Package; control the interface of the signal source according to the protocol data package; wherein the complete event group includes a right mouse button press event, a right mouse button lift event and any number of mouse movement events.

The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the part of the above technical solution that essentially contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disc, optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be used Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (12)

1. The remote control method based on the android device is characterized by comprising the following steps of:

acquiring local input events based on interface information of a signal source, wherein the local input events comprise a first type event and a second type event, the first type event comprises a keyboard event and a right mouse event, the second type event comprises a contact event and a mouse event, and the mouse event comprises the right mouse event and other mouse events;

the collecting local input events includes:

collecting the first type of events based on a first monitoring modifier in an android combinable interface;

collecting the second class of events based on a second monitoring modifier in the android combinable interface;

determining that the local input event is a right mouse button event;

when the right mouse button event forms a complete event group, encapsulating the right mouse button event into a protocol data packet;

controlling the interface of the signal source according to the protocol data packet;

wherein the complete event group includes one right mouse button down event, one right mouse button up event, and any number of mouse movement events.

2. The android device-based remote control method of claim 1, wherein the collecting the local input event is preceded by:

Adding a new monitoring modifier to the android combinable interface, wherein the monitoring modifier is used for collecting the local input event;

wherein the new snoop modifier includes the first snoop modifier and the second snoop modifier.

3. The android device-based remote control method of claim 1, wherein determining that the first type of event is the keyboard event or the mouse right button event comprises:

determining that the first type event is the right mouse button event under the condition that the return key value of the first type event is KEYCODE_BACK;

and under the condition that the return key value of the first type event is not KEYCODE_BACK, determining that the first type event is the keyboard event.

4. The android device-based remote control method of claim 3, wherein determining that the second type of event is the contact event or the mouse event comprises:

under the condition that the pressure value of the second class event is zero, determining the second class event as the mouse event;

and under the condition that the voltage value of the second type event is not zero, determining the second type event as the contact event.

5. The android device-based remote control method of claim 4, wherein determining that the second type of event is the right mouse button event or the other mouse event comprises:

determining that the second type event is the right mouse event under the condition that the relation value of the callback variable corresponding to the callback function in the android combinable interface is a preset value;

and under the condition that the relation value of the callback variable corresponding to the callback function in the android combinable interface is other than the preset value, determining that the second type event is the other mouse event.

6. The android device-based remote control method of claim 5, wherein said determining that said second class event is said mouse right click event, then comprises:

discarding the right mouse button event.

7. The android device-based remote control method of claim 5, wherein said determining that said first type of event is said mouse right click event, then comprises:

collecting the right mouse button pressing event, and responding to the right mouse button pressing event on the operation interface of the android device;

Collecting the right mouse button lifting event, and responding to the right mouse button lifting event on the operation interface of the android device;

and the operation interface of the android device responds to the right mouse button lifting event after responding to the right mouse button pressing event.

8. The android device-based remote control method of claim 7, wherein said determining that said second class event is said other mouse event comprises:

collecting the mouse moving event, and responding to the mouse moving event at an operation interface of the android device;

the operation interface of the android device responds to the mouse movement event after responding to the mouse right button pressing event;

and the operation interface of the android device responds to the right mouse button lifting event after responding to the mouse movement event.

9. An android device-based remote control apparatus, characterized in that the android device-based remote control method of any one of claims 1 to 8 is applied, the apparatus comprising:

the input event acquisition module is used for acquiring local input events based on interface information of a signal source, wherein the local input events comprise a first type event and a second type event, the first type event comprises a keyboard event and a right mouse event, the second type event comprises a contact event and a mouse event, and the mouse event comprises the right mouse event and other mouse events;

The collecting local input events includes:

collecting the first type of events based on a first monitoring modifier in an android combinable interface;

collecting the second class of events based on a second monitoring modifier in the android combinable interface;

the input event analysis module is used for determining that the local input event is a right mouse button event;

the input event encapsulation module is used for encapsulating the right mouse button event into a protocol data packet when the right mouse button event forms a complete event group;

the remote control module is used for controlling the interface of the signal source according to the protocol data packet;

wherein the complete event group includes one right mouse button down event, one right mouse button up event, and any number of mouse movement events.

10. An android device-based remote control system, comprising:

the system comprises a mouse device, an android device, an acquisition control end and a signal source;

the mouse device is connected with the android device and is used for inputting a local input event to the android device;

the android device is connected with the acquisition control end, and a protocol data packet is obtained by adopting the remote control method based on the android device according to any one of claims 1-8 during operation;

The acquisition control end is connected with the signal source and is used for transmitting the protocol data packet to the signal source and transmitting interface information of the signal source to the android device;

the signal source is used for controlling interface operation according to the received protocol data packet and transmitting interface information to the acquisition control end.

11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the android device based remote control method of any one of claims 1 to 8 when the program is executed.

12. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the steps of the android device-based remote control method of any of claims 1 to 8.