WO2024114770A1 - Cursor control method and electronic device - Google Patents

Abstract

Provided in the present application are a cursor control method and an electronic device. The method is applied to a first electronic device. The method comprises: receiving a first instruction sent by a second electronic device, and displaying a cursor in a static state according to the first instruction; receiving a second instruction sent by the second electronic device, changing the cursor from the static state to a pointing state according to the second instruction, moving, on a screen of a first electronic device, at least one of the cursor, a focus where the cursor currently resides and a list displayed on the screen of the first electronic device, and displaying the cursor in the static state when the cursor or the list stops moving; and receiving a third instruction sent by the second electronic device, and playing, according to the third instruction, footage corresponding to a target focus where the cursor resides. The present application can solve the problem of it being difficult to accurately control pointing interaction, and prevent cursor jitter caused by misoperation by a user, thereby realizing efficient interaction, and improving the user experience.

Description

Cursor control method and electronic device

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on December 1, 2022, with application number 202211538582.X, and the priority of the Chinese patent application with the invention name "A cursor control method and electronic device", all contents of which are incorporated by reference in this application.

Technical Field

The embodiments of the present application relate to the field of electronic devices, and in particular to a cursor control method and electronic device.

Background technique

With the continuous development of smart screens, more and more companies are migrating mobile applications to smart screens to expand the large-screen ecosystem. Currently, the industry's commonly used method is active pointing, where users can hold the remote control and actively point to the large screen for operation. Although this active pointing method is simple and easy to learn, the user experience is poor during actual operation and efficient interaction cannot be achieved.

Specifically, when the user's mobile phone remote control actively points to the large screen for operation, many non-pointing behaviors will also cause the remote control posture to change, resulting in the cursor moving. For example, when the user chooses to play video 1 and clicks or slides on the remote control, the remote control posture changes, causing the cursor that should be displayed on video 1 to move to other positions on the large screen, thereby mistakenly opening the focus of the position where the cursor finally stops, reducing the user experience.

Summary of the invention

The embodiments of the present application provide a cursor control method and an electronic device, which can solve the problem that pointing interaction is difficult to accurately control, achieve efficient interaction, and improve user experience.

In a first aspect, a cursor control method is provided, the method being applied to a first electronic device, the method comprising:

Receive a first instruction sent by the second electronic device, and display the cursor in a static state according to the first instruction; receive a second instruction sent by the second electronic device, change the cursor from the static state to a pointing state according to the second instruction, and move the cursor, the focus on which the cursor is currently resting, and at least one of the list displayed on the screen of the first electronic device on the screen of the first electronic device, and display the cursor in the static state when the cursor or the list stops moving, wherein, when the cursor is displayed in the pointing state, the cursor, the focus on which the cursor is currently resting, and at least one of the list is movable, and when the cursor is displayed in the static state, the cursor, the focus on which the cursor is currently resting, and the list are not movable; receive a third instruction sent by the second electronic device, and play the picture corresponding to the target focus on which the cursor is resting according to the third instruction.

According to the solution provided by the embodiment of the present application, the first electronic device can display the cursor as a static state according to the first instruction sent by the second electronic device, and when the cursor is in the static state, the first electronic device can change the cursor to a pointing state according to the second instruction sent by the second electronic device, and move the cursor, the focus where the cursor currently stays, and at least one of the list displayed on the screen of the first electronic device on the screen of the first electronic device. When the movement stops, the cursor is displayed as a static state. Further, the first electronic device can play the picture corresponding to the target focus where the cursor stops when the movement stops according to the received third instruction. According to the solution provided by the embodiment of the present application, by distinguishing the state of the cursor and when the cursor is in the pointing state, the first electronic device can move the cursor, the focus where the cursor currently stays, and at least one of the list according to the received instruction. When the cursor is in the static state, the cursor, the focus where the cursor currently stays, and the list cannot be moved, which can solve the problem that pointing interaction is difficult to control accurately, and prevent the cursor from jittering due to user misoperation, thereby achieving efficient interaction and improving user experience.

In combination with the first aspect, in some possible implementations, the second instruction is used to instruct the cursor to move in a first direction and a first distance; the cursor is changed from the static state to the pointing state according to the second instruction, and the cursor, the focus where the cursor is currently resting, and at least one of the list displayed on the screen of the first electronic device are moved on the screen of the first electronic device, including: the cursor is changed from the static state to the pointing state according to the second instruction, and the cursor is moved in the first direction on the screen of the first electronic device. The first distance.

In the solution provided by the embodiment of the present application, the second instruction instructs the cursor to move in a first direction and a first distance. At this time, the first electronic device can move the cursor in the first direction and the first distance on the screen of the first electronic device according to the second instruction, that is, the first electronic device can accurately move the cursor according to the second instruction, which can solve the problem that the pointing interaction is difficult to accurately control, thereby further realizing efficient Interaction improves user experience.

In combination with the first aspect, in some possible implementations, the cursor in the pointing state presents a first state, and the first state represents a direction in which the cursor moves.

In the solution provided by the embodiment of the present application, the cursor in the pointing state presents a first state, and the first state represents the direction of movement of the cursor. Through such a design, the interaction state can be better fed back to the user, thereby improving the user experience.

In combination with the first aspect, in some possible implementations, the second instruction is used to instruct the list displayed on the screen of the first electronic device to move in a second direction and a second distance; changing the cursor from the static state to the pointing state according to the second instruction, and moving at least one of the cursor, the focus where the cursor is currently resting, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, includes: changing the cursor from the static state to the pointing state according to the second instruction, and moving the list on the screen of the first electronic device in the second direction and the second distance.

In the solution provided in an embodiment of the present application, the second instruction instructs the list displayed on the screen of the first electronic device to move in a second direction and a second distance. At this time, the first electronic device can move the second distance in the second direction on the screen of the first electronic device according to the second instruction list, that is, the first electronic device can accurately move the list displayed on the screen of the first electronic device according to the second instruction, which can further achieve efficient interaction and improve user experience.

In combination with the first aspect, in some possible implementations, the cursor in the pointing state presents a second state, and the second state represents the direction of movement of the list and the amplitude of the cursor deformation.

The solution provided in the embodiment of the present application is that the cursor in the pointing state presents a second state, and the second state represents the direction of movement of the list displayed on the screen of the first electronic device and the amplitude of the cursor deformation. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

In combination with the first aspect, in some possible implementations, the second instruction includes the cursor moving in a third direction and moving a third distance; changing the cursor from the static state to the pointing state according to the second instruction, and moving the cursor, the focus where the cursor is currently resting, and at least one of the list displayed on the screen of the first electronic device on the screen of the first electronic device, includes: changing the cursor from the static state to the pointing state according to the second instruction, and moving the cursor and the focus where the cursor is currently resting on the screen of the first electronic device in the third direction by the third distance.

In the solution provided in an embodiment of the present application, the second instruction instructs the cursor to move in a third direction and a third distance. At this time, the first electronic device can move the cursor and the focus where the cursor is currently located on the screen of the first electronic device in the third direction and a third distance according to the second instruction. That is, the first electronic device can accurately move the cursor and the focus where the cursor is currently located according to the second instruction, which can further achieve efficient interaction and enhance user experience.

In combination with the first aspect, in some possible implementations, the cursor presents a third state, and the third state represents the direction in which the cursor and the focus where the cursor rests move.

In the solution provided by the embodiment of the present application, the cursor in the pointing state presents a third state, and the third state represents the direction of movement of the cursor and the focus where the cursor rests. Through such a design, the interaction state can be better fed back to the user, thereby improving the user experience.

In combination with the first aspect, in some possible implementations, the method further includes: receiving a fourth instruction sent by the second electronic device, and displaying a function box according to the fourth instruction, wherein the function box includes function keys for controlling the screen currently displayed by the first electronic device.

According to the solution provided by the embodiment of the present application, the first electronic device can receive the fourth instruction sent by the second electronic device, and display a function box according to the fourth instruction, wherein the function box includes function keys for controlling the screen currently displayed by the first electronic device. The embodiment of the present application provides a method for displaying the function box, which can be distinguished from the pointing operation or the sliding operation through such a design, and the currently displayed screen can be adjusted through the function keys.

In combination with the first aspect, in some possible implementations, if the cursor is moved out of the screen of the first electronic device according to the second instruction, the cursor in the pointing state displays a fourth state, and the fourth state represents the direction in which the cursor moves out of the screen of the first electronic device and the amplitude by which the cursor moves out of the screen of the second electronic device; the method also includes: receiving a fifth instruction sent by the second electronic device, and displaying the cursor on the screen of the first electronic device according to the fifth instruction.

According to the solution provided in the embodiment of the present application, the first electronic device can receive the fifth instruction sent by the second electronic device and redisplay the cursor on the screen of the first electronic device according to the fifth instruction. In the process of the first electronic device redisplaying the cursor on the screen of the first electronic device according to the fifth instruction, the shape of the cursor will change accordingly. The degree of cursor deformation can represent the angle at which the posture of the second electronic device deviates from the first electronic device. Through such a dynamic effect design, the interaction status can be better fed back to the user, thereby improving the user experience. In particular, when the first electronic device moves the cursor out of the screen of the first electronic device according to the second instruction, the solution provided in the embodiment of the present application can make The cursor is displayed again on the screen of the first electronic device, and the angle at which the posture of the second electronic device deviates from the first electronic device can be understood according to the degree of deformation of the cursor, thereby improving user experience.

In a second aspect, a cursor control method is provided, which is applied to a second electronic device, and the method includes: generating a first instruction in response to a first operation of a user, the first instruction being used to instruct a cursor displayed on the first electronic device to display a static state; sending the first instruction to the first electronic device; generating a second instruction in response to a second operation of the user, the second instruction being used to instruct the cursor to change from the static state to a pointing state, and the direction and/or distance in which the cursor, the focus on which the cursor currently resides, and at least one of the list displayed on the screen of the first electronic device move on the screen of the first electronic device when the cursor is displayed in the pointing state, and the cursor being displayed in the static state when the cursor or the list stops moving, wherein when the cursor is displayed in the pointing state, the cursor, the focus on which the cursor currently resides, and at least one of the list are movable, and when the cursor is displayed in the static state, the cursor, the focus on which the cursor currently resides, and the list are immovable; sending the second instruction to the first electronic device; generating a third instruction in response to a third operation of the user, the third instruction being used to instruct the playback of a picture corresponding to the target focus on which the cursor resides; and sending the third instruction to the first electronic device.

According to the solution provided by the embodiment of the present application, in response to a first operation of a user, the second electronic device can generate a first instruction and send the first instruction to the first electronic device, so that the first electronic device can display the cursor in a static state according to the first instruction; in response to a second operation of the user, the second electronic device can generate a second instruction and send the second instruction to the first electronic device, so that the first electronic device can change the cursor to a pointing state according to the second instruction, and move the cursor, the focus where the cursor currently stays, and at least one of the list displayed on the screen of the first electronic device on the screen of the first electronic device, and when the movement stops, the cursor is displayed in a static state; in response to a third operation of the user, the second electronic device generates a third instruction and sends the third instruction to the first electronic device, so that the first electronic device can play the picture corresponding to the target focus where the cursor stops moving according to the received third instruction. According to the solution provided in the embodiment of the present application, the second electronic device can generate different instructions in response to different operations of the user, so that the first electronic device can perform corresponding operations according to the received instructions. Specifically, when the cursor is in a pointing state, the cursor, the focus where the cursor is currently resting, and at least one of the list are moved; when the cursor is in a stationary state, the cursor, the focus where the cursor is currently resting, and the list cannot be moved. This can solve the problem that pointing interaction is difficult to control accurately, prevent cursor jitter caused by user misoperation, thereby achieving efficient interaction and improving user experience.

In combination with the second aspect, in some possible implementations, generating a second instruction in response to a second operation of the user includes: determining an operation type of the second operation in response to the second operation, the operation type including a pointing operation and a sliding operation; and generating the second instruction according to the operation type of the second operation.

According to the solution provided in the embodiment of the present application, in response to the second operation, the second electronic device can first determine the operation type of the second operation, and more accurately generate a second instruction according to the operation type of the second operation and send it to the first electronic device, so that the first electronic device can perform a corresponding action according to the second instruction. Since the second instruction is generated based on the operation type of the second operation, in other words, the content included in the second instruction can reflect the user's operation on the second electronic device, so that when the first electronic device performs the corresponding operation according to the second instruction, it can be synchronized with the user's second operation. For example, the second operation is a pointing operation, and the first electronic device can move the cursor and/or the focus where the cursor rests according to the pointing operation without sliding the list, which can further achieve efficient interaction and enhance user experience.

In combination with the second aspect, in some possible implementations, determining the operation type of the second operation in response to the second operation includes: determining, in response to the second operation, the magnitude of a numerical value corresponding to the movement trajectory and a first threshold, or determining the magnitude of an angle difference and a second threshold, wherein the movement trajectory is the movement trajectory of a finger on the screen of the second electronic device included in the second operation, and the angle difference is the difference in the change of the posture angle of the second electronic device caused by the second operation; if the numerical value corresponding to the movement trajectory is greater than the first threshold, determining that the second operation is the sliding operation; if the angle difference is greater than the second threshold, determining that the second operation is the pointing operation.

The solution provided by the embodiment of the present application is that if the value corresponding to the movement trajectory of the finger acting on the screen of the second electronic device included in the second operation is greater than the first threshold, the second electronic device can determine that the second operation is a sliding operation; if the difference in the change of the posture angle of the second electronic device caused by the second operation is greater than the second threshold, the second electronic device can determine that the second operation is a pointing operation. In other words, the second electronic device can determine whether the second operation is a sliding operation or a pointing operation based on the difference in the movement trajectory and the change in the posture angle, and generate a corresponding instruction based on the operation type and send it to the first electronic device, so that the first electronic device can perform a corresponding action according to the received instruction, which can prevent the inaccuracy of the content indicated by the generated second instruction due to the uncertainty of the operation type, resulting in an error in the execution of the first electronic device. Therefore, the embodiment of the present application can achieve precise interaction and improve user experience.

In conjunction with the second aspect, in some possible implementations, if the operation type of the second operation is the pointing operation, the second instruction is used to indicate the direction and/or distance in which the cursor and/or the focus on which the cursor currently resides moves on the screen of the first electronic device; if the operation type of the second operation is the sliding operation, the second instruction is used to indicate the column displayed on the screen of the first electronic device. The table shows the direction and/or distance moved on the screen of the first electronic device.

In the solution provided by the embodiment of the present application, if the operation type of the second operation is a pointing operation, the second instruction is used to indicate the direction and/or distance that the cursor and/or the focus where the cursor currently stays move on the screen of the first electronic device; if the operation type of the second operation is a sliding operation, the second instruction is used to indicate the direction and/or distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device. So that the first electronic device can accurately move at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device according to the second instruction, which can further achieve efficient interaction and improve user experience.

In combination with the second aspect, in some possible implementations, the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's wrist holding the second electronic device moving in a first direction; the second instruction is used to instruct the cursor to move in the first direction and a first distance.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's wrist holding the second electronic device moving in a first direction, then the second instruction can instruct the cursor to move in the first direction and move a first distance, so that the first electronic device can move the cursor on the screen of the first electronic device in the first direction and by the first distance according to the second instruction, that is, the first electronic device can accurately move the cursor according to the second instruction, which can solve the problem that pointing interaction is difficult to accurately control, thereby further realizing efficient interaction and improving user experience.

In combination with the second aspect, in some possible implementations, the cursor in the pointing state presents a first state, and the first state represents a direction in which the cursor moves.

In the solution provided in the embodiment of the present application, the second instruction is also used to instruct the cursor in the pointing state to present a first state, and the first state represents the direction of movement of the cursor. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

In combination with the second aspect, in some possible implementations, the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's finger sliding the screen of the second electronic device along a second direction; the second instruction is used to instruct the list displayed on the screen of the first electronic device to move in the second direction and a second distance.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's finger sliding the screen of the second electronic device along the second direction; then the second instruction can instruct the list displayed on the screen of the first electronic device to move in the second direction and a second distance, so that the first electronic device can move the second distance in the second direction on the screen of the first electronic device according to the second instruction list, that is, the first electronic device can accurately move the list displayed on the screen of the first electronic device according to the second instruction, which can further achieve efficient interaction and improve user experience.

In combination with the second aspect, in some possible implementations, the cursor in the pointing state presents a second state, and the second state represents the direction of movement of the list and the amplitude of the cursor deformation.

According to the solution provided in the application embodiment, the second instruction is also used to instruct the cursor in the pointing state to present a second state, and the second state represents the direction of movement of the list displayed on the screen of the first electronic device and the amplitude of the cursor deformation. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

In combination with the second aspect, in some possible implementations, the first operation is that the user's finger clicks on the screen of the second electronic device, lifts up the finger, and then clicks again after lifting up the finger, and the second operation is that the wrist of the user holding the second electronic device moves in a third direction; the second instruction is used to instruct the cursor to move in the third direction and a third distance.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is that the user's finger clicks on the screen of the second electronic device and then lifts up and clicks again after lifting up, and the second operation is that the wrist of the user holding the second electronic device moves to a third direction; then the second instruction can instruct the cursor to move in the third direction and move a third distance, so that the first electronic device can move the cursor and the focus where the cursor is currently located on the screen of the first electronic device by a third distance in the third direction according to the second instruction, that is, the first electronic device can accurately move the cursor and the focus where the cursor is currently located according to the second instruction, which can further achieve efficient interaction and improve user experience.

In combination with the second aspect, in some possible implementations, the cursor in the pointing state presents a third state, and the third state represents the direction in which the cursor and the focus where the cursor rests move.

In the solution provided in the embodiment of the present application, the second instruction also includes the cursor in the pointing state presenting a third state, and the third state represents the direction of movement of the cursor and the focus where the cursor rests. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

In conjunction with the second aspect, in some possible implementations, when the user's finger clicks on the screen of the second electronic device and then lifts up and then clicks again after lifting up, and the duration of the second click is greater than a preset threshold, the method further includes: generating a fourth instruction, the fourth instruction being used to instruct the first electronic device to display a function box, the function box including a function for controlling the first electronic device to sending the fourth instruction to the first electronic device.

According to the solution provided by the embodiment of the present application, when the user's finger clicks on the screen of the second electronic device and then lifts it up, and then clicks again after lifting it up, and the duration of the second click is greater than a preset threshold, the second electronic device can generate a fourth instruction and send it to the first electronic device, so that the first electronic device can display a function box according to the fourth instruction, and the function box includes function keys for controlling the screen currently displayed by the first electronic device. The embodiment of the present application provides a method for displaying a function box, which can be distinguished from a pointing operation or a sliding operation through such a design, and the currently displayed screen can be adjusted through the function keys.

In combination with the second aspect, in some possible implementations, the method also includes: in response to a fourth operation of the user, generating a fifth instruction, the fifth instruction being used to indicate a direction in which the cursor moves so that the cursor is displayed on the screen of the first electronic device; and sending the fifth instruction to the first electronic device.

According to the solution provided by the embodiment of the present application, the second electronic device can generate a fifth instruction in response to the fourth operation of the user and send it to the first electronic device, so that the first electronic device can redisplay the cursor on the screen of the first electronic device according to the fifth instruction. In particular, when the first electronic device moves the cursor out of the screen of the first electronic device according to the second instruction, the solution provided by the embodiment of the present application can make the cursor redisplay on the screen of the first electronic device.

In combination with the second aspect, in some possible implementations, the distance that the cursor and the focus where the cursor currently rests move on the screen of the first electronic device is related to the speed at which the second electronic device rotates, and the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device.

In the solution provided by the embodiment of the present application, the distance that the cursor and the focus where the cursor currently rests move on the screen of the first electronic device is related to the rotation speed of the second electronic device, and the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device. When the rotation speed of the second electronic device is a fixed value, or when the sliding speed of the user on the screen of the second electronic device is fixed, the movement distance of the cursor or the list can be adjusted by adjusting the display-control ratio, thereby improving the user experience.

In combination with the second aspect, in some possible implementations, the faster the second electronic device rotates, the larger the display-control ratio, and the greater the distance the cursor and/or the focus where the cursor is currently located moves on the screen of the first electronic device; the slower the second electronic device rotates, the smaller the display-control ratio, and the smaller the distance the cursor and/or the focus where the cursor is currently located moves on the screen of the first electronic device.

In the solution provided by the embodiment of the present application, the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device is related to the rotation speed of the second electronic device, that is, the faster the second electronic device rotates and the larger the display-control ratio, the greater the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device; the slower the second electronic device rotates and the smaller the display-control ratio, the smaller the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. By adjusting the display control ratio according to the rotation speed of the second electronic device and further adjusting the distance that the cursor and/or the focus where the cursor is currently resting moves, to a certain extent, the speed and accuracy of the movement can be taken into account. Specifically, the faster the second electronic device rotates, the greater the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. It can be understood that when the speed of the second electronic device rotates faster, the efficiency of the movement of the cursor and/or the focus where the cursor is currently resting is given priority, that is, the cursor and/or the focus where the cursor is currently resting is quickly moved to the corresponding position; the slower the speed of the second electronic device rotates, the smaller the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. It can be understood that when the speed of the second electronic device rotates slower, the accuracy of the movement of the cursor and/or the focus where the cursor is currently resting is given priority, that is, the cursor and/or the focus where the cursor is currently resting is slowly moved to the precise position.

In combination with the second aspect, in some possible implementations, the faster the user slides on the screen of the second electronic device, the larger the display-to-control ratio, and the greater the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device; the slower the user slides on the screen of the second electronic device, the smaller the display-to-control ratio, and the smaller the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device.

In the solution provided by the embodiment of the present application, the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device, that is, the faster the user slides on the screen of the second electronic device and the larger the display-control ratio, the larger the distance that the list moves on the screen of the first electronic device; the slower the user slides on the screen of the second electronic device and the smaller the display-control ratio, the smaller the distance that the list moves on the screen of the first electronic device. By adjusting the display-control ratio according to the speed at which the user slides on the screen of the second electronic device and further adjusting the distance that the list moves, to a certain extent, the speed and accuracy of movement can be taken into account. Specifically, the faster the user slides on the screen of the second electronic device, the larger the distance that the list eventually moves on the screen of the first electronic device. It can be understood that when the user slides on the screen of the second electronic device faster, the efficiency of the list movement is given priority, that is, the list is quickly moved to the corresponding position; the slower the user slides on the screen of the second electronic device, the smaller the display-control ratio is. The smaller the distance moved on the screen of the second electronic device, it can be understood that when the user slides slower on the screen of the second electronic device, the accuracy of the list movement is given priority, that is, the list is slowly moved to a precise position.

In a third aspect, a device is provided, which is included in an electronic device, and has the function of implementing the electronic device behavior in the above aspects and possible implementations of the above aspects. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

In a fourth aspect, an electronic device is provided, comprising: one or more processors; a memory; one or more applications; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the method in any possible implementation of the first aspect or the second aspect.

In a fifth aspect, a chip system is provided, comprising at least one processor, wherein when program instructions are executed in the at least one processor, the function of the method in any possible implementation of the first aspect or the second aspect mentioned above is implemented on an electronic device.

In a sixth aspect, a computer storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a method in any possible implementation of the first aspect or the second aspect.

In a seventh aspect, a computer program product is provided. When the computer program product runs on an electronic device, the electronic device executes a method in any possible design of the first aspect or the second aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

FIG. 2 is a schematic diagram of a software structure of an electronic device provided in an embodiment of the present application.

FIG3 is a schematic diagram of a set of GUIs provided in an embodiment of the present application.

FIG. 4 is a schematic diagram of another set of GUIs provided in an embodiment of the present application.

FIG. 5 is a schematic diagram of another set of GUIs provided in an embodiment of the present application.

FIG. 6 is a schematic diagram of another set of GUIs provided in an embodiment of the present application.

FIG. 7 is a schematic diagram of a method for controlling cursor display provided in an embodiment of the present application.

FIG. 8 is a schematic diagram of a cursor display provided in an embodiment of the present application.

FIG9 is a schematic diagram of a user pointing based on body orientation provided in an embodiment of the present application.

FIG. 10 is a schematic diagram of another method of pointing based on body orientation of a user provided in an embodiment of the present application.

FIG. 11 is a schematic diagram of a plane coordinate system established based on a mobile phone screen according to an embodiment of the present application.

FIG12 is a schematic diagram of a three-dimensional rectangular coordinate system established based on a mobile phone screen according to an embodiment of the present application.

FIG. 13 is a schematic diagram of a display-control ratio curve provided in an embodiment of the present application.

FIG. 14 is a schematic diagram of a light control method provided in an embodiment of the present application.

FIG15 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

FIG16 is a schematic block diagram of another electronic device provided in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

The solution provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

For example, FIG1 shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power supply Management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (SIM) interface, and/or a universal serial bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or a data signal. In an embodiment, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The mobile communication module 150 can provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) and the like applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and/or a satellite based augmentation system (SBAS).

The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture static images or videos. The video codec is used to compress or decompress digital videos.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

FIG2 is a software structure diagram of the electronic device 100 of an embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

As shown in FIG. 2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

As shown in FIG. 2 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

In order to facilitate understanding of the solution of the present application, the following first introduces the technology related to the present application.

With the continuous development of smart screens, more and more companies are migrating mobile applications to smart screens to expand the large-screen ecosystem. Currently, the industry's commonly used method is active pointing, where users can hold the remote control and actively point to the large screen for operation. Although this active pointing method is simple and easy to learn, the user experience is poor during actual operation and efficient interaction cannot be achieved.

Specifically, when the user's mobile phone remote control actively points to the large screen for operation, many non-pointing behaviors will also cause the remote control posture to change, resulting in the cursor moving. For example, when the user chooses to play video 1 and clicks or slides on the remote control, the remote control posture changes, causing the cursor that should be displayed on video 1 to move to other positions on the large screen, thereby mistakenly opening the focus of the position where the cursor finally stops, reducing the user experience.

The embodiment of the present application provides a cursor control method, which can solve the problem that pointing interaction is difficult to accurately control, achieve efficient interaction, and improve user experience.

The following embodiments of the present application will take an electronic device having the structure shown in FIG. 1 and FIG. 2 as an example, and will specifically describe the cursor control method provided in the embodiments of the present application in combination with the accompanying drawings and application scenarios.

Figure 3 shows a set of graphical user interfaces (GUI) for a TV, wherein Figure 3 (a) to Figure 3 (j) show a method for a user to control a cursor displayed on the TV interface through a mobile phone.

Referring to the GUI shown in (a) of FIG. 3 , the TV in the GUI displays the main interface, and a tab bar is displayed above the main interface: My, Children, Featured, VIP, etc., and multiple videos are also displayed on the main interface. The current cursor state is ●, that is, the cursor is in a static state, and the cursor is located in the focus of video 1. The user can control the state and position of the cursor on the TV screen through the mobile phone. When the TV detects that the user's finger is pressing the screen 301 of the mobile phone, the TV can display the GUI shown in (b) of FIG. 3 .

Referring to the GUI shown in (b) of FIG. 3 , it can be seen that the state of the cursor displayed on the TV screen changes from ● to That is, the cursor changes from the normal state to the static state. The user can point the mobile phone at the TV and turn his wrist to change the posture of the mobile phone. When the TV detects that the posture of the mobile phone rotates from left to right, the TV can display the GUI shown in (c) of Figure 3.

It should be noted that the states presented by the cursor may include normal state, static state and pointing state. When the cursor is in the pointing state, when the user turns his wrist to change the posture of the mobile phone, the cursor moves with the change of the mobile phone posture; when the cursor is in the static state or normal state, when the user turns his wrist to change the posture of the mobile phone, the cursor will not move.

Referring to the GUI shown in (c) of FIG. 3 , it can be seen that when the posture of the mobile phone rotates from left to right, the state of the cursor changes from Transformed to That is, the cursor changes from a static state to a pointing state, where Indicates that the cursor is moving to the right at this time. The user can stop rotating the wrist at any time. When the cursor stays in the focus that the user wants to play, the user can stop rotating the wrist, and the TV can display the GUI shown in (d) in Figure 3.

Referring to the GUI shown in (d) of FIG. 3 , it can be seen that the cursor displayed on the TV screen is located in the focus of video 2 at this time, and the state of the cursor is That is, the cursor returns to the static state from the pointing state. When the television detects that the user's finger clicks the screen 302 of the mobile phone again, the television can display the GUI as shown in (e) in FIG. 3 .

Referring to the GUI shown in (e) in FIG3 , the TV screen displays information about Video 2. The user can press and hold the screen of the mobile phone with his finger, and at the same time, the user rotates his wrist to move the cursor to the position of the "Play" function control. Then, after clicking on the screen of the mobile phone, the TV can play Video 2.

In some embodiments, when the user turns his wrist, the cursor may move to the edge of the TV screen or even out of the edge of the TV screen due to excessive wrist rotation. An embodiment of the present application provides a cursor display method that can dynamically inform the user of the angle of the cursor deviation, so that the user can turn his wrist in the opposite direction to compensate for the deviation.

As shown in the GUI in (c) of FIG. 3 above, when the mobile phone is rotated from left to right, the cursor changes from Transformed to When the user stops rotating his wrist, the cursor is located in the focus of video 2. If the user rotates his wrist too much, causing the cursor to move to the edge of the TV screen, the TV can display the GUI shown in (f) in FIG. 3 .

Referring to the GUI shown in (f) of FIG. 3 , it can be seen that at this time, the state presented by the cursor is still And the cursor is located at the edge of the right side of the screen. If the user's wrist continues to turn right, the mobile phone also continues to turn right, and the TV can display the GUI shown in (g) in Figure 3.

Referring to the GUI shown in (g) of FIG. 3 , it can be seen that the cursor has completely moved out of the screen and the icon is displayed on the right edge of the TV screen. It indicates that the cursor has been adsorbed to the right edge at this time. If the user's wrist continues to turn to the right, the TV can display the GUI shown in (h) in Figure 3.

Referring to the GUI shown in (h) of FIG. 3 , it can be seen that the edge on the right side of the TV screen displays icons And the icon displayed on the right edge of (h) in Figure 3 The icon shown on the right edge of (g) in Figure 3 The height of is higher, indicating that the adsorption effect of the cursor is further enhanced. At this time, the user can rotate the wrist in the opposite direction, that is, the user can rotate the wrist from right to left, and the mobile phone also starts to rotate to the left, and the TV can display the GUI shown in (i) in Figure 3.

Referring to the GUI shown in (i) of FIG. 3 , it can be seen that the icon And the icon displayed on the right edge of (i) in Figure 3 Compared to the icon shown on the right edge of (h) in Figure 3 The height of is lower, indicating that the adsorption effect of the cursor is weakened. At this time, the user can further rotate the wrist in the opposite direction, that is, the user continues to rotate the wrist to the left, and the TV can display the GUI shown in (j) in Figure 3.

Referring to the GUI shown in (j) of FIG. 3 , it can be seen that the cursor is displayed on the right edge of the TV screen. This means that the angle of the phone has been compensated by the user's reverse wrist rotation, and the user can continue to rotate the wrist to control the movement of the cursor. In addition, it should be noted that since the user rotates the wrist from right to left during the reverse wrist rotation, when the cursor reappears from the right edge of the TV screen, the cursor The white circle in the middle is located to the left of the entire cursor.

The solution provided in the embodiment of the present application distinguishes the state of the cursor. If the cursor is in a normal or static state, when the user turns his wrist to change the posture of the mobile phone used to control the cursor, the cursor will not move, that is, the position of the cursor will not change with the change of the posture of the mobile phone; If the cursor is in a pointing state, when the user turns his wrist to change the posture of the mobile phone used to control the cursor, the cursor moves with the movement of the mobile phone, that is, the position of the cursor will change with the change of the posture of the mobile phone. Compared with the solution in the prior art that the cursor moves due to the change of the posture of the mobile phone caused by many non-pointing behaviors of the user, the embodiment of the present application can solve the problem that the pointing interaction is difficult to control accurately by distinguishing the state of the cursor and when the cursor is in a pointing state, the position of the cursor changes with the change of the posture of the mobile phone used to control the cursor, thereby achieving efficient interaction and improving the user experience. In addition, the embodiment of the present application also provides a visual prompt method for feedback of the cursor state, that is, when the cursor moves out of the edge of the TV screen, the shape of the cursor will change, and the degree of cursor deformation can represent the degree of deviation of the mobile phone posture from the TV. At this time, the user can compensate for the angle of deviation of the mobile phone posture by moving or rotating the mobile phone in the opposite direction. When the angle of deviation of the mobile phone is compensated by reverse rotation, the cursor reappears from the edge of the TV screen. Through such a dynamic effect design, the interaction status can be better fed back to the user, thereby improving the user experience.

FIG. 4 shows another set of GUIs for a TV, wherein (a) to (e) in FIG. 4 show a method for a user to control a cursor displayed on a TV interface via a mobile phone.

Referring to the GUI shown in (a) of FIG. 4 , the TV in the GUI displays the main interface, and a tab bar is displayed above the main interface: My, Children, Featured, VIP, etc., and multiple videos are also displayed on the main interface. The current cursor status is ●, and the cursor is located in the focus of video 2. The user can control the status and position of the cursor on the TV screen through the mobile phone. When the TV detects that the user's finger is pressing the screen 401 of the mobile phone, the TV can display the GUI shown in (b) of FIG. 4 .

Referring to the GUI shown in (b) of FIG. 4 , it can be seen that the state of the cursor displayed on the TV screen changes from ● to That is, the cursor enters the static state from the normal state, and the user's finger sliding on the screen of the mobile phone can scroll the list of the TV interface. When the TV detects that the user's finger slides upward 402 on the screen of the mobile phone, the TV can display the GUI as shown in (c) in Figure 4.

Referring to the GUI shown in (c) of FIG. 4 , it can be seen that when the user slides his finger upward on the mobile phone screen, the state of the cursor changes to That is, the cursor changes from a static state to a pointing state, where It indicates that the list on the TV interface (including video 1, video 2, video 3, popular recommendations, my apps, etc. displayed on the TV screen) is moving upward as a whole. When the cursor happens to stay in the focus that the user wants to play, at this time, the user's finger can remain still on the mobile phone screen, and the TV can display the GUI as shown in (d) in Figure 4.

It should be noted that in the embodiment of the present application, the amplitude of the cursor deformation is related to the distance the user slides on the mobile phone screen. The longer the distance the user slides on the mobile phone screen, the greater the amplitude of the cursor deformation; the shorter the distance the user slides on the mobile phone screen, the smaller the amplitude of the cursor deformation. However, it is worth noting that there is a maximum upper limit value for the amplitude of the cursor deformation, and the maximum upper limit value can be a preset value.

Referring to the GUI shown in (d) of FIG. 4 , it can be seen that the cursor displayed on the TV screen is located in the focus of video 4 at this time, and the state of the cursor is That is, the cursor returns to the static state from the pointing state, and when the TV detects that the user's finger leaves the TV, the TV may display the GUI as shown in (e) in FIG. 4 .

Referring to the GUI shown in (e) of FIG. 4 , it can be seen that the cursor is still in the focus of video 4 and the current state of the cursor is ●. At this time, the user can play video 4 in the manner shown in FIG. 3 above.

The solution provided by the embodiment of the present application allows the user to scroll the video displayed on the TV screen by sliding on the mobile phone screen, which allows the user to position the cursor to the focus of the user's desired playback more quickly. Specifically, when the user's finger slides on the mobile phone screen, the state of the cursor will change, and the magnitude of the change is related to the distance the user's mobile phone slides on the mobile phone screen, that is, the longer the distance the user slides on the mobile phone screen, the greater the magnitude of the cursor change; the shorter the distance the user slides on the mobile phone screen, the smaller the magnitude of the cursor change. Such a motion effect design can better feedback the interaction status to the user, thereby improving the user experience.

FIG. 5 shows another set of GUIs for a TV, wherein FIG. 5( a ) to FIG. 5( d ) show a method for a user to control a cursor displayed on a TV interface via a mobile phone.

Referring to the GUI shown in (a) of FIG. 5 , the TV in the GUI displays the main interface, and a tab bar is displayed above the main interface: My, Children, Featured, VIP, etc., and multiple videos are also displayed on the main interface. The current cursor status is ●, and the cursor is located in the focus of video 2. The user can control the status and position of the cursor on the TV screen through the mobile phone. When the TV detects that the user's finger clicks on the screen of the mobile phone and then lifts it up and then clicks 501 again after lifting it up, the TV can display the GUI shown in (b) of FIG. 5 .

Referring to the GUI shown in (b) of FIG. 5 , it can be seen that the state of the cursor displayed on the TV screen changes from ● to That is, the cursor has entered the static state from the normal state, where Indicates that the user can drag the focus selected by the cursor. The user can change the posture of the mobile phone by rotating the wrist, so as to drag the position of the focus. When the user's wrist rotates from left to right, the posture of the mobile phone also rotates from left to right, and the TV can display the GUI shown in (c) of Figure 5.

Referring to the GUI shown in (c) of FIG. 5 , it can be seen that when the posture of the mobile phone rotates from left to right, the state of the cursor changes from Transformed to That is, the cursor changes from a static state to a pointing state, where Indicates that the cursor and video 2 are currently in focus. When the cursor and the focus of video 2 are dragged to the position the user wants to display, the user's finger can leave the screen of the mobile phone, and the TV can display the GUI shown in (d) in Figure 5.

Referring to the GUI shown in (d) of FIG. 5 , it can be seen that the cursor status at this time returns to And the focus of video 2 is now located at the far right of the TV interface, and video 3, which was located at the far right of the TV interface before dragging, is now located at the position of video 2 before dragging, which is equivalent to swapping the positions of video 3 and video 2.

The solution provided by the embodiment of the present application allows the user to drag the focus on the TV screen. Specifically, when the TV detects that the user's finger clicks on the screen of the mobile phone, lifts it up, and then clicks again after lifting it up, the state of the cursor will change. When the state of the cursor changes to In this state, the user can drag the focus on the TV screen to the position where the user wants the focus to be located. Through such a design, the focus on the TV screen can be dragged, thereby improving the user experience.

FIG. 6 shows another set of GUIs for a TV, wherein FIG. 6 (a) to FIG. 6 (c) show a method for a user to control a cursor displayed on a TV interface via a mobile phone.

Referring to the GUI shown in (a) of FIG6 , the TV in the GUI displays the main interface, and a tab bar is displayed above the main interface: My, Children, Featured, VIP, etc., and multiple videos are also displayed on the main interface. The current cursor status is ●, and the cursor is located in the focus of video 2. The user can control the status and position of the cursor on the TV screen through the mobile phone. When the user plays video 2 according to the method shown in FIG3 above, the TV can display the GUI shown in (b) of FIG6 .

Referring to the GUI shown in (b) of FIG6 , the interface displays the screen of video 2. When the TV detects that the user's finger clicks on the screen of the mobile phone and then lifts it up and clicks again after lifting it up, and the click duration exceeds a threshold, a long press operation is triggered and the TV can display the GUI shown in (c) of FIG6 .

Referring to the GUI shown in (c) of FIG6 , it can be seen that a function box pops up on the interface at this time, and the function box includes function keys for controlling the playback screen of the video 2. The user can control the playback of the video 2 by clicking the corresponding function keys.

FIG7 is a schematic diagram of a method for controlling cursor display provided in an embodiment of the present application. The embodiment of the present application takes a mobile phone and a television as examples. The method may include steps S710 to S730.

S710, finger pressed on the phone screen.

S712: Whether the mobile phone can identify the operation type.

If yes, execute step S714; if no, execute step S716.

In an embodiment of the present application, the mobile phone in step S712 can determine the operation type based on the user's operation on the mobile phone. For example, when the user presses his finger on the mobile phone screen, if the mobile phone detects that the user is holding the mobile phone and the mobile phone's own posture is changing, the mobile phone can determine that the operation type is a pointing operation; if the mobile phone detects that the touch coordinates of the user's finger on the mobile phone screen change, the mobile phone can determine that the operation type is a sliding operation.

S714: Whether it is a sliding operation.

If yes, execute step S718; if no, execute step S720.

S718, scrolling the list on the TV screen based on the change in the touch coordinates.

S720, based on the change of the mobile phone posture, control the cursor to move on the TV screen.

In the embodiment of the present application, when the mobile phone determines that the operation type is a sliding operation, the list on the TV screen scrolls as the touch coordinates of the user on the mobile phone screen change. As shown in FIG. 4 above, when the user slides his finger upward on the mobile phone screen, the list on the TV screen also scrolls upward; similarly, when the user slides his finger downward on the mobile phone screen, the list on the TV screen also scrolls downward.

When the operation type is determined to be a pointing operation, the cursor displayed on the TV screen will also move as the posture of the mobile phone changes. As shown in Figure 3 above, when the user holds the mobile phone and points the mobile phone at the TV and turns the wrist to the right, the cursor displayed on the TV screen will also move to the right. When the user holds the mobile phone and points the mobile phone at the TV and turns the wrist to the left, the cursor displayed on the TV screen will also move to the left. When the user holds the mobile phone and points the mobile phone at the TV and moves the wrist upward, the cursor displayed on the TV screen will also move upward. When the user holds the mobile phone and points the mobile phone at the TV and moves the wrist downward, the cursor displayed on the TV screen will also move upward.

In some embodiments, when the operation type is determined to be a pointing operation, as the posture of the mobile phone changes, in addition to the movement of the cursor displayed on the TV screen, the focus where the cursor is located will also move simultaneously. The mobile phone can distinguish by the movement of the user's finger on the mobile phone screen before the user turns or moves the wrist. If the user clicks on the mobile phone screen and then lifts up the finger before turning or moving the wrist and clicks again after lifting it up, when the user turns or moves the wrist, the focus where the cursor is located moves with the cursor. As shown in Figure 5 above, when the user's wrist turns from left to right, the posture of the mobile phone also turns from left to right, and the cursor and the video 2 where the cursor is located move together. This focus uniformly moves to the right.

As shown in Figure 8, it is a schematic diagram of a cursor display provided by an embodiment of the present application. Among them, Figure 8 (a) is the normal state of the cursor; Figure 8 (b) is the movement state of the cursor when the user's finger clicks on the screen of the mobile phone and the user turns or moves the wrist; Figure 8 (c) is the movement state of the cursor when the user's finger slides on the screen of the mobile phone; Figure 8 (d) is the movement state of the cursor when the user's finger clicks on the screen of the mobile phone, lifts up, and clicks again after lifting up, and the user turns or moves the wrist.

In addition, with the solution provided in the embodiments of the present application, the user can point within a suitable wrist movement range according to his or her own orientation.

As shown in FIG. 9 and FIG. 10 , they are schematic diagrams of a user pointing based on body orientation provided in an embodiment of the present application.

As shown in Figure 9, when the user is facing the TV screen, the user can point toward the TV screen based on the activity range shown in Figure 9 (angle α shown in the figure); as shown in Figure 10, when the user is located on the right side of the TV screen, the user can point toward the left side of the TV screen based on the right side of the TV screen and the activity range shown in Figure 10 (angle β shown in the figure).

S716: Is this the first frame in which the finger just touches the screen?

If yes, execute step S722; if no, execute step S724.

S722, record the initial touch coordinates and the initial posture of the mobile phone.

In an embodiment of the present application, in the case where the type of operation cannot be determined, the mobile phone can determine whether the operation of pressing the finger on the mobile phone screen is the first frame in which the finger has just touched the mobile phone screen. If so, the initial touch coordinates of the finger touching the mobile phone screen and the initial posture of the mobile phone can be recorded to facilitate the calculation of the finger movement trajectory of subsequent sliding operations and the change in the posture of the mobile phone during rotation operations.

S724: Whether the value of the finger movement trajectory exceeds a first threshold.

If yes, execute step S726; if no, execute step S728.

S726, enter the sliding operation state.

In an embodiment of the present application, for the calculation of the numerical value of the finger movement trajectory: a plane coordinate system can be established based on the center of the mobile phone screen, and the numerical value of the finger movement trajectory can be calculated according to the change of the coordinates touched by the finger.

As shown in FIG. 11 , it is a schematic diagram of a plane coordinate system established based on a mobile phone screen according to an embodiment of the present application. Assuming that the coordinate value of the initial touch position A of the user's finger is (a1, a2), when the user's finger slides to position B and stops, the coordinate value of B is (b1, b2). If the user's finger movement trajectory is a straight line, the value of the user's finger movement trajectory can be expressed as

In an embodiment of the present application, after obtaining the numerical value of the finger movement trajectory, the size of the numerical value and the first threshold can be judged. If the numerical value is greater than the first threshold, it can be considered that the user has slid on the mobile phone screen, thereby entering the sliding operation state; if the numerical value is less than or equal to the first threshold, the size of the mobile phone posture change value and the second threshold can be further judged.

When the value of the finger movement trajectory is greater than the first threshold, the operation of the user's finger on the mobile phone screen is reflected on the TV, and the list on the TV interface will scroll up or down. Specifically, if the user slides his finger upward on the mobile phone screen, the list on the TV interface will also scroll up, and if the user slides his finger downward on the mobile phone screen, the list on the TV interface will also scroll down.

S728: Whether the mobile phone posture change value exceeds a second threshold.

If yes, execute step S730, if no, end.

S730, entering the pointing operation state.

In an embodiment of the present application, for the change value of the mobile phone posture, a three-dimensional rectangular coordinate system can be established based on the center of the mobile phone screen, and the rotation angle of the mobile phone, that is, the change value of the mobile phone posture, can be calculated according to the change of the mobile phone posture.

As shown in FIG. 12 , it is a schematic diagram of a three-dimensional rectangular coordinate system established based on a mobile phone screen according to an embodiment of the present application.

Referring to (a) in FIG12 , the center of the mobile phone screen is taken as the coordinate origin, and the right, inward, and upward are the x-axis, y-axis, and z-axis respectively. When the mobile phone is rotated to the right by a certain angle, the coordinate axis will also rotate with the rotation of the mobile phone. At this time, the coordinate axis will change accordingly, as shown in (b) in FIG12 .

Referring to (b) in FIG. 12 , it can be seen that when the mobile phone is rotated to the right by a certain angle from the posture shown in (a) in FIG. 12 , the posture of the mobile phone changes. Since the coordinate axis rotates with the rotation of the mobile phone, the coordinate axis presented at this time deviates from the coordinate axis presented in (a) in FIG. 12 .

In order to compensate for this deviation, when the mobile phone is rotated to a certain angle, the posture of the mobile phone can be calculated according to the new coordinate system (i.e., the x' axis, z' axis and y axis), see (c) in Figure 12. The new coordinate system can be understood to be consistent with the coordinate system shown in (a) in Figure 12.

Specifically, the calculation can be performed according to the following formula:

[q ₀ ,q ₁ ,q ₂ ,q ₃ ] = currentAttitudeRotationQuaternion*(relativeRotation*currentAttitudeRotationQuaternion_inverse)

relativeRotation＝lastAttitude _inverse *currentAttitude

Among them, R represents the relative rotation angle of the device between two adjacent frames, [x, y, z] is the relative rotation angle R expressed in the form of Euler angles, [q ₀ ,q ₁ ,q ₂ ,q ₃ ] is the relative rotation angle R expressed in the form of quaternion, currentAttitudeRotationQuaternion is the rotation quaternion of the roll angle, relativeRotation is the relative rotation of the coordinate systems of two adjacent frames, currentAttitudeRotationQuaternion_inverse is the inverse of the rotation quaternion of the roll angle, roll is the roll angle of the device relative to the earth coordinate system after rotation, lastAttitude is the attitude of the device relative to the earth coordinate system before rotation (quaternion form), lastAttitude _inverse is the inverse of lastAttitude, and currentaAttitude is the attitude of the device relative to the earth coordinate system (quaternion form) after rotation.

In an embodiment of the present application, after obtaining the mobile phone posture change value, the size of the mobile phone posture change value and the second threshold can be judged. If the posture change value is greater than the second threshold, it can be considered that the user has rotated the mobile phone, thereby entering the pointing operation state; if the posture change value is less than or equal to the second threshold, the judgment process is terminated.

When the posture change value is greater than the second threshold, the user's operation of rotating or moving the wrist while holding the mobile phone is reflected on the TV, and the cursor displayed on the TV will move horizontally or vertically. Specifically, if the user rotates the wrist to the right while holding the mobile phone, the cursor displayed on the TV will move to the right; if the user rotates the wrist to the left while holding the mobile phone, the cursor displayed on the TV will move to the left; if the user moves the wrist upward while holding the mobile phone, the cursor displayed on the TV will move upward; if the user moves the wrist downward while holding the mobile phone, the cursor displayed on the TV will move downward.

Of course, in some possible implementations, the two judgment processes of judging the trajectory of the finger movement and the size of the first threshold and judging the change value of the mobile phone posture and the size of the second threshold are not prioritized. In other words, the trajectory of the finger movement and the size of the first threshold can be judged first, and then the change value of the mobile phone posture and the size of the second threshold can be judged; or, the change value of the mobile phone posture and the size of the second threshold can be judged first, and then the trajectory of the finger movement and the size of the first threshold can be judged; there is no limitation.

It should be noted that although the above embodiments show that when the value of the finger movement trajectory exceeds the first threshold, it can be considered that the user has slid on the mobile phone screen, in some embodiments, when the value of the finger movement trajectory is equal to or greater than the first threshold, it can also be considered that the user has slid on the mobile phone screen; similarly, when the mobile phone posture change value is equal to the second threshold, it can also be considered that the user has rotated the mobile phone.

In addition, the embodiment of the present application can adjust the display-control ratio by the rotation speed of the wrist to control the accuracy and speed of the cursor movement. Specifically, if the rotation speed of the wrist is low, such as the rotation speed of the wrist is less than the preset value, the accuracy of the cursor display can be given priority, that is, as the user's wrist rotates, the cursor can move slowly and eventually stay accurately at the position where the user wants the cursor to be displayed; if the rotation speed of the wrist is high, such as the rotation speed of the wrist is greater than the preset value, the efficiency of the cursor display can be given priority, that is, as the user's wrist rotates, the cursor can move quickly and eventually stay near the position where the user wants the cursor to be displayed.

As shown in Figure 13, it is a schematic diagram of a display-control ratio curve provided in an embodiment of the present application. It can be seen from the figure that for any curve, the faster the user moves the mobile phone or the faster the user's finger moves, the larger the display-control ratio, and the longer the cursor displayed on the TV screen and/or the focus where the cursor stays moves; the slower the user moves the mobile phone or the slower the user's finger moves, the smaller the display-control ratio, and the shorter the cursor displayed on the TV screen and/or the focus where the cursor stays moves.

It is worth noting that the faster the user moves the phone, the greater the distance the cursor and/or the focus where the cursor is currently resting moves on the TV screen. It can be understood that when the phone moves faster, the efficiency of the movement of the cursor and/or the focus where the cursor is currently resting is prioritized, that is, the cursor and/or the focus where the cursor is currently resting is quickly moved to the corresponding position; when the user moves the phone slower, the smaller the distance the cursor and/or the focus where the cursor is currently resting moves on the TV screen. It can be understood that when the phone moves slower, the accuracy of the movement of the cursor and/or the focus where the cursor is currently resting is prioritized, that is, the cursor and/or the focus where the cursor is currently resting is slowly moved to the precise position.

In addition, the figure also shows multiple curves, which are mainly related to the sensitivity of the cursor. The more sensitive the cursor is, the faster the mobile phone or finger moves, the greater the display-control ratio. Therefore, the cursor sensitivity corresponding to curve 1 shown in the figure is the greatest, followed by curve 2, and so on.

Case 1: The user rotates the phone with his wrist

The display-control ratio can be calculated according to the following formula:
Gain＝loopup(R)
R＝max(| _Rz |,| _Ry |)+min(| _Rz |,| _Ry |)/2

Wherein, Gain is the display-control ratio, R is the total rotation angle of two adjacent frames of the mobile phone relative to each other, _Rz is the rotation angle of two adjacent frames of the mobile phone in the horizontal direction, and _Ry is the rotation angle of two adjacent frames of the mobile phone in the vertical direction.

The mobile phone rotates by an angle R, which is reflected on the TV. The displacement of the cursor displayed on the TV is represented by d:

d _x = R _z *Gain
_dy ＝ _Ry *Gain

Where _dx is the lateral displacement of the cursor, and _dy is the longitudinal displacement of the cursor.

It should be noted that in the embodiment of the present application, the same wrist movement has the same pointing effect for different user's hand-holding postures of the mobile phone. In other words, the posture change value of the mobile phone has nothing to do with the user's hand-holding posture of the mobile phone, but only with the rotation direction of the wrist of the hand holding the mobile phone. When the wrist rotates horizontally, the cursor displayed on the TV moves horizontally, and when the wrist rotates vertically, the cursor displayed on the TV moves vertically.

Case 2: The user slides his finger on the screen

The displacement of the user's finger sliding on the mobile phone screen is k, and the displacement of the cursor displayed on the TV is d:

If the user slides his finger up and down, then d = Gain*k _y

If the user slides his finger left or right, then d = Gain*k _x

Where, Gain is the display-control ratio, ky _is the longitudinal displacement of the finger, and _{kx is} the lateral displacement of the finger.

The solution provided by the embodiment of the present application is that the mobile phone determines the user operation type by combining the operation of the user's finger and the change of the mobile phone's posture, and enters the corresponding operation state according to the operation type. Specifically, when the mobile phone can determine the operation type based on the operation of the user's finger, the list on the TV screen can be scrolled based on the change of the coordinates of the finger touching the mobile phone screen, or the cursor can be controlled to move on the TV screen based on the change of the mobile phone's posture; when the mobile phone cannot determine the operation type based on the operation of the user's finger, the initial touch coordinates of the user's finger on the mobile phone screen can be recorded first, and then the value of the movement trajectory of the user's finger can be determined with the size of the first threshold, or the value of the mobile phone posture change value with the size of the second threshold, so as to enter the corresponding operation state more accurately. The above design can solve the problem that pointing interaction is difficult to control accurately, thereby achieving efficient interaction and improving user experience.

As shown in Fig. 14, it is a schematic diagram of a light control method provided by an embodiment of the present application. The method can be executed by a first electronic device and a second electronic device, and the method can include steps 1410-1426.

1410. In response to a first operation of a user, the second electronic device generates a first instruction, where the first instruction is used to instruct a cursor displayed on the first electronic device to display a static state.

1412, the second electronic device sends the first instruction to the first electronic device;

1414. The first electronic device displays the cursor in a static state according to the first instruction.

The first electronic device in the embodiment of the present application may be the television in the above embodiment, and the second electronic device may be the mobile phone in the above embodiment. When the first electronic device receives the first instruction sent by the second electronic device, the cursor displayed on the screen of the first electronic device may be adjusted to a static state according to the first instruction.

1416. In response to a second operation of the user, the second electronic device generates a second instruction, wherein the second instruction is used to instruct the cursor to change from the static state to the pointing state, and when the cursor is displayed in the pointing state, the direction and/or distance in which at least one of the cursor, the focus on which the cursor currently resides, and the list displayed on the screen of the first electronic device moves on the screen of the first electronic device, and when the cursor or the list stops moving, the cursor is displayed in the static state, wherein when the cursor is displayed in the pointing state, at least one of the cursor, the focus on which the cursor currently resides, and the list is movable, and when the cursor is displayed in the static state, the cursor, the focus on which the cursor currently resides, and the list are not movable;

1418. The second electronic device sends the second instruction to the first electronic device.

1420, the first electronic device displays the cursor as changing from the static state to the pointing state according to the second instruction, and moves at least one of the cursor, the focus on which the cursor currently rests, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, and displays the cursor in the static state when the cursor or the list stops moving.

In the embodiment of the present application, when the cursor is displayed in a static state, when the first electronic device receives the second electronic device sent by the second electronic device, Instruction, a corresponding operation can be performed according to the second instruction, for example, the cursor can be changed to a pointing state according to the second instruction, and at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device can be moved on the screen of the first electronic device. The specific operations performed by the first electronic device are related to the content specifically indicated by the second instruction, and the specific content indicated by the second instruction is related to the second operation of the user. For specific situations, please refer to the following situations 1, 2, and 3.

1422. In response to a third operation of the user, the second electronic device generates a third instruction, where the third instruction is used to instruct to play a picture corresponding to the target focus where the cursor is located;

1424. The second electronic device sends the third instruction to the first electronic device.

1426, the first electronic device receives the third instruction sent by the second electronic device, and plays the picture corresponding to the target focus where the cursor is located according to the third instruction.

According to the solution provided by the embodiment of the present application, the first electronic device can display the cursor as a static state according to the first instruction sent by the second electronic device, and when the cursor is in the static state, the first electronic device can change the cursor to a pointing state according to the second instruction sent by the second electronic device, and move at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device on the screen of the first electronic device. When the movement stops, the cursor is displayed in a static state. At this time, the second electronic device can generate a third instruction according to the third operation of the user, and the first electronic device can play the picture corresponding to the target focus where the cursor stops when the movement stops according to the received third instruction. According to the solution provided by the embodiment of the present application, by distinguishing the state of the cursor and when the cursor is in the pointing state, the first electronic device can move at least one of the cursor, the focus where the cursor currently stays, and the list according to the received instruction. When the cursor is in the static state, the cursor, the focus where the cursor currently stays, and the list cannot be moved, which can solve the problem that it is difficult to accurately control the pointing interaction, and prevent the cursor from jittering due to user misoperation, thereby achieving efficient interaction and improving user experience.

Optionally, in some embodiments, generating a second instruction in response to a second operation of the user includes:

In response to the second operation, determining an operation type of the second operation, where the operation type includes a pointing operation and a sliding operation;

The second instruction is generated according to the operation type of the second operation.

According to the solution provided in the embodiment of the present application, in response to the second operation, the second electronic device can first determine the operation type of the second operation, and more accurately generate a second instruction according to the operation type of the second operation and send it to the first electronic device, so that the first electronic device can perform a corresponding action according to the second instruction. Since the second instruction is generated based on the operation type of the second operation, in other words, the content included in the second instruction can reflect the user's operation on the second electronic device, so that when the first electronic device performs the corresponding operation according to the second instruction, it can be synchronized with the user's second operation. For example, the second operation is a pointing operation, and the first electronic device can move the cursor and/or the focus where the cursor rests according to the pointing operation without sliding the list, which can further achieve efficient interaction and enhance user experience.

Optionally, in some embodiments, in response to the second operation, determining the operation type of the second operation includes:

In response to the second operation, determining the magnitude of a numerical value corresponding to the movement trajectory and a first threshold, or determining the magnitude of an angle difference and a second threshold, wherein the movement trajectory is a movement trajectory of a finger acting on a screen of the second electronic device included in the second operation, and the angle difference is a difference in a change in a posture angle of the second electronic device caused by the second operation;

If the value corresponding to the movement trajectory is greater than the first threshold, determining that the second operation is the sliding operation;

If the angle difference is greater than the second threshold, it is determined that the second operation is the pointing operation.

The solution provided by the embodiment of the present application is that if the value corresponding to the movement trajectory of the finger acting on the screen of the second electronic device included in the second operation is greater than the first threshold, the second electronic device can determine that the second operation is a sliding operation; if the difference in the change of the posture angle of the second electronic device caused by the second operation is greater than the second threshold, the second electronic device can determine that the second operation is a pointing operation. In other words, the second electronic device can determine whether the second operation is a sliding operation or a pointing operation based on the difference in the movement trajectory and the change in the posture angle, and generate a corresponding instruction based on the operation type and send it to the first electronic device, so that the first electronic device can perform a corresponding action according to the received instruction, which can prevent the inaccuracy of the content indicated by the generated second instruction due to the uncertainty of the operation type, resulting in an error in the execution of the first electronic device. Therefore, the embodiment of the present application can achieve precise interaction and improve user experience.

Optionally, in some embodiments, if the operation type of the second operation is the pointing operation, the second instruction is used to indicate the direction and/or distance in which the cursor and/or the focus where the cursor currently resides moves on the screen of the first electronic device;

If the operation type of the second operation is the sliding operation, the second instruction is used to indicate the direction and/or distance in which the list displayed on the screen of the first electronic device moves on the screen of the first electronic device.

In the solution provided by the embodiment of the present application, if the operation type of the second operation is a pointing operation, the second instruction is used to indicate the direction and/or distance that the cursor and/or the focus where the cursor currently stays moves on the screen of the first electronic device; if the operation type of the second operation is a sliding operation, the second instruction is used to indicate the direction and/or distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device. Thus, the first electronic device can accurately move the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device according to the second instruction. At least one movement can further achieve efficient interaction and enhance user experience.

As mentioned above, in response to the user's first operation, the second electronic device generates a first instruction; in response to the user's second operation, the second electronic device generates a second instruction; wherein the specific content indicated by the second instruction is related to the first operation and the second operation, please refer to the following for details.

Case 1:

The first operation is that the user presses the screen of the second electronic device with a finger, and the second operation is that the user moves the wrist of the user holding the second electronic device in a first direction;

The second instruction is used to instruct the cursor to move in the first direction and over a first distance.

The step of displaying the cursor in a pointing state according to the second instruction and moving at least one of the cursor, a focus on which the cursor currently resides, and a list displayed on the screen of the first electronic device on the screen of the first electronic device includes:

The cursor is displayed in a pointing state according to the second instruction, and the cursor is moved by the first distance in the first direction on the screen of the first electronic device.

For details about the first situation, please refer to the content shown in FIG. 3 above, which will not be described in detail.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's wrist holding the second electronic device moving in a first direction, then the second instruction can instruct the cursor to move in the first direction and move a first distance. At this time, the first electronic device can move the cursor on the screen of the first electronic device in the first direction and by the first distance according to the second instruction, that is, the first electronic device can accurately move the cursor according to the second instruction, which can solve the problem that pointing interaction is difficult to accurately control, thereby further realizing efficient interaction and improving user experience.

Optionally, in some embodiments, the second instruction is further used to instruct the cursor in the pointing state to present a first state, wherein the first state represents a direction in which the cursor moves.

In the solution provided in the embodiment of the present application, the second instruction is also used to instruct the cursor in the pointing state to present a first state, and the first state represents the direction of movement of the cursor. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

Case 2:

The first operation is that the user's finger presses the screen of the second electronic device, and the second operation is that the user's finger slides the screen of the second electronic device along a second direction;

The second instruction is used to instruct the list displayed on the screen of the first electronic device to move in a second direction and a second distance.

The step of displaying the cursor in a pointing state according to the second instruction and moving at least one of the cursor, a focus on which the cursor currently resides, and a list displayed on the screen of the first electronic device on the screen of the first electronic device includes:

The cursor is displayed in a pointing state according to the second instruction, and the list is moved by the second distance in the second direction on the screen of the first electronic device.

For details about the second situation, please refer to the content shown in FIG. 4 above, which will not be described in detail.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is the user's finger pressing the screen of the second electronic device, and the second operation is the user's finger sliding the screen of the second electronic device along the second direction; then the second instruction can instruct the list displayed on the screen of the first electronic device to move in the second direction and a second distance. At this time, the first electronic device can move the second distance in the second direction on the screen of the first electronic device according to the second instruction list, that is, the first electronic device can accurately move the list displayed on the screen of the first electronic device according to the second instruction, which can further achieve efficient interaction and improve user experience.

Optionally, in some embodiments, the second instruction is further used to instruct the cursor in the pointing state to present a second state, and the second state represents the direction of movement of the list and the amplitude of the cursor deformation.

In the solution provided in an embodiment of the present application, the second instruction is also used to instruct the cursor in the pointing state to present a second state, and the second state represents the direction of movement of the list displayed on the screen of the first electronic device and the amplitude of the cursor deformation. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

Case 3:

The first operation is that the user's finger clicks on the screen of the second electronic device, lifts up, and then clicks again after being lifted up, and the second operation is that the user's wrist holding the second electronic device moves in a third direction;

The second instruction is used to instruct the cursor to move along the third direction and a third distance.

The step of displaying the cursor in a pointing state according to the second instruction and moving at least one of the cursor, a focus on which the cursor currently resides, and a list displayed on the screen of the first electronic device on the screen of the first electronic device includes:

According to the second instruction, the cursor is displayed as a pointing state, and the cursor and the focus currently on which the cursor is located are placed on the first The screen of the electronic device moves the third distance along the third direction.

For details about situation three, please refer to the content shown in FIG. 5 above, which will not be described in detail.

In the solution provided in the embodiment of the present application, the specific content indicated by the second instruction is related to the first operation and the second operation. If the first operation is that the user's finger clicks on the screen of the second electronic device and then lifts up and clicks again after lifting up, and the second operation is that the wrist of the user holding the second electronic device moves to a third direction; then the second instruction can instruct the cursor to move in the third direction and move a third distance. At this time, the first electronic device can move the cursor and the focus where the cursor is currently located on the screen of the first electronic device by a third distance in the third direction according to the second instruction, that is, the first electronic device can accurately move the cursor and the focus where the cursor is currently located according to the second instruction, which can further achieve efficient interaction and improve user experience.

Optionally, in some embodiments, the second instruction is further used to instruct the cursor in the pointing state to present a third state, and the third state represents the direction in which the cursor and the focus where the cursor rests move.

In the solution provided in the embodiment of the present application, the second instruction also includes the cursor in the pointing state presenting a third state, and the third state represents the direction of movement of the cursor and the focus where the cursor rests. Through such a design, the interaction status can be better fed back to the user, thereby improving the user experience.

Optionally, in some embodiments, when the user's finger clicks on the screen of the second electronic device and then lifts up and clicks again after lifting up, and the duration of the second click is greater than a preset threshold, the method further includes:

The second electronic device generates a fourth instruction, where the fourth instruction is used to instruct the first electronic device to display a function box, where the function box includes function keys for controlling a screen currently displayed by the first electronic device;

The second electronic device sends the fourth instruction to the first electronic device.

The first electronic device receives the fourth instruction sent by the second electronic device, and displays a function box according to the fourth instruction, wherein the function box includes function keys for controlling the screen currently displayed by the first electronic device.

According to the solution provided by the embodiment of the present application, when the user's finger clicks on the screen of the second electronic device and then lifts it, and then clicks again after lifting it, and the duration of the second click is greater than a preset threshold, the second electronic device can generate a fourth instruction and send it to the first electronic device. The first electronic device can display a function box according to the fourth instruction, and the function box includes function keys for controlling the screen currently displayed by the first electronic device. The embodiment of the present application provides a method for displaying a function box, which can be distinguished from a pointing operation or a sliding operation through such a design, and the currently displayed screen can be adjusted through the function keys.

Optionally, in some embodiments, if the cursor is moved out of the screen of the first electronic device according to the second instruction, the cursor in the pointing state displays a fourth state, and the fourth state represents the direction in which the cursor moves out of the screen of the first electronic device and the amplitude by which the cursor moves out of the screen of the second electronic device;

The method further comprises:

In response to a fourth operation of the user, the second electronic device generates a fifth instruction, wherein the fifth instruction is used to indicate a direction in which the cursor moves so that the cursor is displayed on the screen of the first electronic device;

The second electronic device sends the fifth instruction to the first electronic device.

The first electronic device receives the fifth instruction sent by the second electronic device, and displays the cursor on the screen of the first electronic device according to the fifth instruction.

According to the solution provided by the embodiment of the present application, if the first electronic device moves the cursor out of the screen of the first electronic device according to the second instruction, the cursor in the pointing state displays the fourth state, and the fourth state represents the direction in which the cursor moves out of the screen of the first electronic device and the amplitude of the cursor moving out of the screen of the second electronic device; the second electronic device can generate a fifth instruction in response to the user's fourth operation and send it to the first electronic device, so that the first electronic device can redisplay the cursor on the screen of the first electronic device according to the fifth instruction. In the process of the first electronic device redisplaying the cursor on the screen of the first electronic device according to the fifth instruction, the shape of the cursor will change accordingly, and the degree of the cursor deformation can represent the angle of the posture of the second electronic device deviating from the first electronic device. Through such a dynamic effect design, the interaction status can be better fed back to the user, thereby improving the user experience.

Optionally, in some embodiments, the distance that the cursor and the focus where the cursor currently rests move on the screen of the first electronic device is related to the speed at which the second electronic device rotates, and the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device.

In the solution provided by the embodiment of the present application, the distance that the cursor and the focus where the cursor currently rests move on the screen of the first electronic device is related to the rotation speed of the second electronic device, and the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device. When the rotation speed of the second electronic device is a fixed value, or when the sliding speed of the user on the screen of the second electronic device is fixed, the movement distance of the cursor or the list can be adjusted by adjusting the display-control ratio, thereby improving the user experience.

Optionally, in some embodiments, the faster the second electronic device rotates, the greater the display-control ratio is, and the greater the distance that the cursor and/or the focus where the cursor currently stays moves on the screen of the first electronic device is;

The slower the rotation speed of the second electronic device is, the smaller the display-control ratio is, and the smaller the distance that the cursor and/or the focus where the cursor currently stays moves on the screen of the first electronic device is.

In the solution provided by the embodiment of the present application, the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device is related to the rotation speed of the second electronic device, that is, the faster the second electronic device rotates and the larger the display-control ratio, the greater the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device; the slower the second electronic device rotates and the smaller the display-control ratio, the smaller the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. By adjusting the display control ratio according to the rotation speed of the second electronic device and further adjusting the distance that the cursor and/or the focus where the cursor is currently resting moves, to a certain extent, the speed and accuracy of the movement can be taken into account. Specifically, the faster the second electronic device rotates, the greater the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. It can be understood that when the second electronic device rotates faster, the efficiency of the movement of the cursor and/or the focus where the cursor is currently resting is given priority, that is, the cursor and/or the focus where the cursor is currently resting is quickly moved to the corresponding position; the slower the second electronic device rotates, the smaller the distance that the cursor and/or the focus where the cursor is currently resting moves on the screen of the first electronic device. It can be understood that when the second electronic device rotates slower, the accuracy of the movement of the cursor and/or the focus where the cursor is currently resting is given priority, that is, the cursor and/or the focus where the cursor is currently resting is slowly moved to the precise position.

Optionally, in some embodiments, the faster the user slides on the screen of the second electronic device, the larger the display-control ratio is, and the greater the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device;

The slower the user slides on the screen of the second electronic device, the smaller the display-to-control ratio is, and the smaller the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device.

In the solution provided by the embodiment of the present application, the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device, that is, the faster the user slides on the screen of the second electronic device, the larger the display-control ratio, and the larger the distance that the list moves on the screen of the first electronic device, and the slower the user slides on the screen of the second electronic device, the smaller the display-control ratio, and the smaller the distance that the list moves on the screen of the first electronic device. By adjusting the display-control ratio according to the speed at which the user slides on the screen of the second electronic device, the distance that the list moves on the screen of the first electronic device is further adjusted. To a certain extent, the speed and accuracy of movement can be taken into account. Specifically, the faster the user slides on the screen of the second electronic device, the larger the distance that the final list moves on the screen of the first electronic device. It can be understood that when the user slides on the screen of the second electronic device faster, the efficiency of the list movement is given priority, that is, the list is quickly moved to the corresponding position; the slower the user slides on the screen of the second electronic device, the smaller the distance that the final list moves on the screen of the first electronic device. It can be understood that when the user slides on the screen of the second electronic device slower, the accuracy of the list movement is given priority, that is, the list is slowly moved to a precise position.

It is understandable that, in order to realize the above functions, the electronic device includes hardware and/or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

In the case of dividing each functional module according to each function, FIG15 shows a possible composition diagram of the electronic device 1500 involved in the above embodiment. As shown in FIG15 , the electronic device 1500 may include: a transceiver module 1510 and a processing module 1520 .

When the electronic device 1500 is used to implement the function of the first electronic device in the method embodiment described in Figure 14, the processing module 1520 can be used to support the electronic device 1500 to execute the above steps 1414, 1420, 1426, etc., and/or other processes for the technology described in this article.

When the electronic device 1500 is used to implement the function of the second electronic device in the method embodiment described in Figure 14, the transceiver module 1510 can be used to support the electronic device 1500 to execute the above steps 1412, 1418, 1424, etc., and/or other processes for the technology described in this article.

The processing module 1520 may be used to support the electronic device 1500 in executing the above steps 1410 , 1416 , 1422 , etc., and/or other processes for the technology described herein.

It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

The electronic device provided in this embodiment is used to execute the method of the present application described above, and thus can achieve the same effect as the above-mentioned implementation method.

In the case of using integrated units, the electronic device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the above-mentioned various units. The storage module can be used to support the electronic device to execute stored program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a device having the structure shown in FIG. 1 .

Figure 16 shows another possible composition diagram of the electronic device 800 involved in the above embodiment. As shown in Figure 16, the electronic device 800 may include a communication unit 810, an input unit 820, a processing unit 830, an output unit (or also called a display unit) 840, a peripheral interface 850, a storage unit 860, a power supply 870, a video decoder 880 and an audio decoder 890.

The communication unit 810 is used to establish a communication channel so that the electronic device 800 can connect to a remote server through the communication channel and download media data from the remote server. The communication unit 810 may include communication modules such as a WLAN module, a Bluetooth module, an NFC module, a baseband module, and a radio frequency (RF) circuit corresponding to the communication module, which is used to perform wireless local area network communication, Bluetooth communication, NFC communication, infrared communication and/or cellular communication system communication, such as wideband code division multiple access (W-CDMA) and/or high speed downlink packet access (HSDPA). The communication module 810 is used to control the communication of various components in the electronic device and can support direct memory access.

The input unit 820 can be used to implement user interaction with the electronic device and/or input information into the electronic device. In the specific implementation of the present application, the input unit can be a touch panel, or other human-computer interaction interfaces, such as physical input keys, microphones, etc., or other external information acquisition devices, such as cameras, etc.

The processing unit 830 is the control center of the electronic device, and can use various interfaces and lines to connect various parts of the entire electronic device, and execute or execute software programs and/or modules stored in the storage unit, and call data stored in the storage unit to perform various functions of the electronic device and/or process data. For example, the above steps 1410, 1416, 1422, or steps 1414, 1420, 1426, etc. can be implemented by the processing unit 1430.

The output unit 840 includes but is not limited to an image output unit and a sound output unit. The image output unit is used to output text, pictures and/or videos. In a specific embodiment of the present application, the touch panel used by the above-mentioned input unit 820 can also be used as a display panel of the output unit 840 at the same time. For example, when the touch panel detects a touch or proximity gesture operation on it, it is transmitted to the processing unit to determine the type of touch event, and then the processing unit provides a corresponding visual output on the display panel according to the type of touch event. Although in Figure 16, the input unit 820 and the output unit 840 are used as two independent components to realize the input and output functions of the electronic device, in some embodiments, the touch panel and the display panel can be integrated to realize the input and output functions of the electronic device. For example, the image output unit can display various graphical user interfaces as virtual control components, including but not limited to windows, scroll bars, icons and scrapbooks, for users to operate by touch.

The storage unit 860 may be used to store software programs and modules. The processing unit executes various functional applications of the electronic device and realizes data processing by running the software programs and modules stored in the storage unit.

This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

Through the description of the above implementation modes, those skilled in the art can understand that for the convenience and simplicity of description, only the above functions are used. The division of energy modules is illustrated by way of example. In practical applications, the above functions can be distributed to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (26)

A cursor control method, characterized in that the method is applied to a first electronic device, and the method comprises: receiving a first instruction sent by the second electronic device, and displaying the cursor in a static state according to the first instruction; receiving a second instruction sent by the second electronic device, changing the cursor from the static state to the pointing state according to the second instruction, and moving at least one of the cursor, the focus on which the cursor currently stays, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, and displaying the cursor in the static state when the cursor or the list stops moving, wherein when the cursor is displayed in the pointing state, at least one of the cursor, the focus on which the cursor currently stays, and the list is movable, and when the cursor is displayed in the static state, the cursor, the focus on which the cursor currently stays, and the list are not movable; A third instruction sent by the second electronic device is received, and a picture corresponding to the target focus where the cursor is located is played according to the third instruction. The method according to claim 1, wherein the second instruction is used to instruct the cursor to move in a first direction and a first distance; The step of changing the cursor from the static state to the pointing state according to the second instruction, and moving at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, comprises: The cursor is changed from the static state to the pointing state according to the second instruction, and the cursor is moved by the first distance in the first direction on the screen of the first electronic device. The method according to claim 2 is characterized in that the cursor in the pointing state presents a first state, and the first state represents the direction in which the cursor moves. The method according to claim 1, wherein the second instruction is used to instruct the list displayed on the screen of the first electronic device to move in a second direction and a second distance; The step of changing the cursor from the static state to the pointing state according to the second instruction, and moving at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, comprises: The cursor is changed from the static state to the pointing state according to the second instruction, and the list is moved on the screen of the first electronic device by the second distance in the second direction. The method according to claim 4 is characterized in that the cursor in the pointing state presents a second state, and the second state represents the direction of movement of the list and the amplitude of the cursor deformation. The method according to claim 1, wherein the second instruction comprises the cursor moving in a third direction and a third distance; The step of changing the cursor from the static state to the pointing state according to the second instruction, and moving at least one of the cursor, the focus where the cursor currently stays, and the list displayed on the screen of the first electronic device on the screen of the first electronic device, comprises: The cursor is changed from the static state to the pointing state according to the second instruction, and the cursor and the focus where the cursor currently stays are moved along the third direction by the third distance on the screen of the first electronic device. The method according to claim 6 is characterized in that the cursor presents a third state, and the third state represents the direction of movement of the cursor and the focus where the cursor rests. The method according to claim 6 or 7, characterized in that the method further comprises: A fourth instruction sent by the second electronic device is received, and a function box is displayed according to the fourth instruction, wherein the function box includes function keys for controlling a screen currently displayed by the first electronic device. The method according to any one of claims 1 to 8, characterized in that, if the cursor is moved out of the screen of the first electronic device according to the second instruction, the cursor in the pointing state displays a fourth state, and the fourth state represents the direction in which the cursor moves out of the screen of the first electronic device and the amplitude by which the cursor moves out of the screen of the second electronic device; The method further comprises: A fifth instruction sent by the second electronic device is received, and the cursor is displayed on the screen of the first electronic device according to the fifth instruction. A cursor control method, characterized in that the method is applied to a second electronic device, and the method comprises: In response to a first operation of the user, generating a first instruction, wherein the first instruction is used to instruct a cursor displayed on the first electronic device to display a static state; Sending the first instruction to a first electronic device; In response to a second operation of the user, a second instruction is generated, wherein the second instruction is used to instruct the cursor to change from the static state to the pointing state, and when the cursor is displayed in the pointing state, the direction and/or distance in which at least one of the cursor, the focus on which the cursor currently stays, and the list displayed on the screen of the first electronic device moves on the screen of the first electronic device, and when the cursor or the list stops moving, the cursor is displayed in the static state, wherein when the cursor is displayed in the pointing state, at least one of the cursor, the focus on which the cursor currently stays, and the list is movable, and when the cursor is displayed in the static state, the cursor, the focus on which the cursor currently stays, and the list are not movable; sending the second instruction to the first electronic device; In response to a third operation of the user, generating a third instruction, wherein the third instruction is used to instruct to play a picture corresponding to the target focus where the cursor is located; The third instruction is sent to the first electronic device. The method according to claim 10, wherein generating a second instruction in response to a second operation of the user comprises: In response to the second operation, determining an operation type of the second operation, where the operation type includes a pointing operation and a sliding operation; The second instruction is generated according to the operation type of the second operation. The method according to claim 11, characterized in that, in response to the second operation, determining the operation type of the second operation comprises: In response to the second operation, determining the magnitude of a numerical value corresponding to the movement trajectory and a first threshold, or determining the magnitude of an angle difference and a second threshold, wherein the movement trajectory is a movement trajectory of a finger acting on a screen of the second electronic device included in the second operation, and the angle difference is a difference in a change in a posture angle of the second electronic device caused by the second operation; If the value corresponding to the movement trajectory is greater than the first threshold, determining that the second operation is the sliding operation; If the angle difference is greater than the second threshold, it is determined that the second operation is the pointing operation. The method according to claim 11 or 12, characterized in that, if the operation type of the second operation is the pointing operation, the second instruction is used to indicate the direction and/or distance in which the cursor and/or the focus where the cursor currently resides moves on the screen of the first electronic device; If the operation type of the second operation is the sliding operation, the second instruction is used to indicate the direction and/or distance in which the list displayed on the screen of the first electronic device moves on the screen of the first electronic device. The method according to any one of claims 10 to 13, characterized in that the first operation is that the user's finger presses the screen of the second electronic device, and the second operation is that the user holding the second electronic device moves the wrist in a first direction; The second instruction is used to instruct the cursor to move in the first direction and over a first distance. The method according to claim 14 is characterized in that the cursor in the pointing state presents a first state, and the first state represents the direction in which the cursor moves. The method according to any one of claims 10 to 13, characterized in that the first operation is that the user's finger presses the screen of the second electronic device, and the second operation is that the user's finger slides the screen of the second electronic device along a second direction; The second instruction is used to instruct the list displayed on the screen of the first electronic device to move in a second direction and a second distance. The method according to claim 16 is characterized in that the cursor in the pointing state presents a second state, and the second state represents the direction of movement of the list and the amplitude of the cursor deformation. The method according to any one of claims 10 to 13, characterized in that the first operation is that the user's finger clicks on the screen of the second electronic device and then lifts it and then clicks again after lifting it, and the second operation is that the wrist of the user holding the second electronic device moves in a third direction; The second instruction is used to instruct the cursor to move along the third direction and a third distance. The method according to claim 18 is characterized in that the cursor in the pointing state presents a third state, and the third state represents the direction of movement of the cursor and the focus where the cursor rests. The method according to claim 18 or 19 is characterized in that, when the user's finger clicks on the screen of the second electronic device and then lifts up and clicks again after lifting up, and the duration of the second click is greater than a preset threshold, the method further comprises: Generate a fourth instruction, the fourth instruction is used to instruct the first electronic device to display a function box, the function box includes a function for controlling The function keys of the screen currently displayed by the first electronic device; The fourth instruction is sent to the first electronic device. The method according to any one of claims 10 to 20, characterized in that the method further comprises: In response to a fourth operation of the user, generating a fifth instruction, wherein the fifth instruction is used to indicate a direction in which the cursor moves so that the cursor is displayed on the screen of the first electronic device; The fifth instruction is sent to the first electronic device. The method according to any one of claims 10 to 21 is characterized in that the distance that the cursor and the focus where the cursor currently rests move on the screen of the first electronic device is related to the speed at which the second electronic device rotates, and the distance that the list displayed on the screen of the first electronic device moves on the screen of the first electronic device is related to the speed at which the user slides on the screen of the second electronic device. The method according to claim 22, characterized in that the faster the second electronic device rotates, the greater the display-control ratio, and the greater the distance that the cursor and/or the focus where the cursor currently resides moves on the screen of the first electronic device; The slower the rotation speed of the second electronic device is, the smaller the display-control ratio is, and the smaller the distance that the cursor and/or the focus where the cursor currently stays moves on the screen of the first electronic device is. The method according to claim 22, characterized in that the faster the user slides on the screen of the second electronic device, the larger the display-control ratio is, and the greater the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device; The slower the user slides on the screen of the second electronic device, the smaller the display-to-control ratio is, and the smaller the distance the list displayed on the screen of the first electronic device moves on the screen of the first electronic device. An electronic device, comprising: one or more processors; one or more memories; The one or more memories store one or more computer programs, which include instructions. When the instructions are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1 to 9 or 10 to 24. A computer storage medium, characterized in that it includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method as described in any one of claims 1 to 9 or 10 to 24.