CN111625297A - Application program display method, terminal and computer readable storage medium - Google Patents

Abstract

The embodiments of the present application disclose an application display method, a terminal, and a computer-readable storage medium. The terminal captures movement information of a touch object; the touch object is used to touch an application on a display screen of the terminal to realize The operation of the application; based on the movement information, the motion trajectory of the touch object is obtained; according to the motion trajectory, the target touch position of the touch object on the display screen is determined; the process of the application to be displayed corresponding to the target touch position is started, and the target touch position is loaded. Displaying the resources of the application; if receiving a start command of the application to be displayed by the touch object, responding to the start command, through the process of the application to be displayed, the resource of the application to be displayed is called to display the application to be displayed.

Description

Application display method, terminal and computer-readable storage medium

technical field

The present application relates to the field of computer technology, and in particular, to a method for displaying application programs.

Background technique

At present, the application is opened by clicking the application icon, so that the application is displayed on the display screen, usually after the application icon is triggered, the process of the application is started first, then the resources of the application are loaded, and the display is displayed after the resource is loaded. The user interface of the application. That is, after the application icon is triggered, there is a delay between displaying the operation interface of the application, which affects the display speed of the application.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method for displaying an application program, a terminal and a computer-readable storage medium, which improve the display speed of the application program.

The technical solution of the present application is realized as follows:

An embodiment of the present application provides a method for displaying an application, the method comprising: capturing movement information of a touch object; the touch object is used to touch an application on a display screen of the terminal to realize the operation of the application; obtaining the motion track of the touch object based on the movement information; determining the target touch position of the touch object on the display screen according to the motion track; enabling the target touch The process of the application to be displayed corresponding to the location, and the resources of the application to be displayed are loaded; if the start command of the application to be displayed by the touch object is received, the application to be displayed is responded to the start command by the application to be displayed. The process calls the resource of the application to be displayed, and displays the application to be displayed.

An embodiment of the present application provides a terminal, including:

a capture module for capturing movement information of a touch object; the touch object is used to touch an application on the display screen of the terminal to operate the application;

an acquisition module, configured to acquire the motion trajectory of the touch object based on the movement information;

a determining module, configured to determine the target touch position of the touch object on the display screen according to the motion trajectory;

a startup module, configured to start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed;

A display module, configured to respond to the start-up instruction, call the resource of the application to be displayed through the process of the application to be displayed, and display the Describe the application to be displayed.

An embodiment of the present application provides a terminal, where the terminal includes: a processor and a memory for storing a computer program that can run on the processor,

Wherein, the processor is configured to execute the display method of the above application program when running the computer program.

An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the processor executes the above-mentioned display method of the application program.

A method for displaying an application program, a terminal, and a computer-readable storage medium provided by the embodiments of the present application. The terminal captures movement information of a touch object, and the touch object is used to touch an application on a display screen of the terminal to realize Operation on the application; based on the movement information, obtain the motion trajectory of the touch object, determine the target touch position of the touch object on the display screen according to the motion trajectory, start the process of the application to be displayed corresponding to the target touch position, and load it The resource of the application to be displayed, if it receives the start command of the application to be displayed by the touch object, it responds to the start command, calls the resource of the application to be displayed through the process of the application to be displayed, and displays the application to be displayed, that is, the terminal is receiving Before the start command of the application to be displayed by the touch object, after the application to be displayed that the user wishes to start is predicted according to the motion trajectory of the touch object, the application to be displayed is started and the resources are loaded, which reduces the time for the terminal to receive the application program. The delay time between the launch command and the display of the interface of the application increases the display speed of the application.

Description of drawings

FIG. 1 is a system structure diagram of a display application program provided by an embodiment of the present application;

FIG. 2 is a schematic diagram 1 of a display flow of an application provided by an embodiment of the present application;

FIG. 3 is a second schematic diagram of a display flow of an application provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a finger touch operation on a mobile phone according to an embodiment of the present application;

FIG. 5 is a schematic diagram 3 of a display flow of an application provided by an embodiment of the present application;

FIG. 6 is a fourth schematic diagram of a display flow of an application provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of an exemplary predicted touch position of a touch object according to an embodiment of the present application;

FIG. 8 is a schematic diagram 5 of a display flow of an application provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of an exemplary determination of an application to be displayed provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram 1 of a terminal according to an embodiment of the present application;

FIG. 11 is a second schematic structural diagram of a terminal according to an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The existing application program is started based on the touch operation of the touch object on the application icon on the display screen of the terminal. Here, the display screen of the terminal is a touch display screen. After the terminal receives the click of the touch object on the application icon, it starts the process of the application, loads the resources of the application, and then displays the operation interface of the application on the display screen of the terminal. That is to say, the user needs to open the application after clicking. After the icon is displayed, the operation interface of the application cannot be displayed immediately, which affects the response speed of the application.

The present application provides a system structure diagram of a terminal displaying an application program. As shown in FIG. 1 , a terminal 100 is connected to a server 200, and the terminal 100 is connected through the server 200, and the terminal 100 is used to receive an application startup instruction to start the application. Application program, the server 200 is used to provide application services for the terminal 100 .

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

An embodiment of the present application provides a method for displaying an application, as shown in FIG. 2 , the method includes:

S201. Capture movement information of a touch object; the touch object is used to touch an application on a display screen of a terminal to implement an operation on the application;

In the embodiment of the present application, the touch operation on the display screen is only possible when the touch object moves. Therefore, the terminal can capture the movement information of the touch object, determine the movement trajectory of the touch object based on the movement information, and then confirm the touch The control object will perform the touch operation on the application on the display screen of the terminal.

The touch object may be a stylus, or any object supported by the terminal, such as a finger, and can be used to perform touch operations on the display screen of the terminal.

It should be noted that when the interface displayed on the display screen of the terminal is a desktop application interface, the display screen includes application icons of applications supported by the terminal, such as WeChat, Notepad, mailbox, news APP, etc., the embodiment of the present application No restrictions apply.

In this embodiment of the present application, if the touch object moves, there is a landing point on the display screen, and there is a corresponding application icon on the landing point, the terminal can determine that the purpose of the touch operation through the touch object is to start this application.

In the embodiment of the present application, the movement information of the touch object is information used to represent the change of the position of the touch object. The movement information includes: video frame sequence or sensing information of the touch object, wherein the video frame sequence of the touch object includes the change of the image of the touch object over time in the video frame, and the sensor information includes the difference between the touch object and the touch object. Relative position information between displays.

In this embodiment of the present application, the video frame sequence of the touch object can be obtained from a video recorded by a camera, wherein the camera can capture the process of moving the touch object to the display screen when the touch object needs to perform a touch operation on the display screen. Cameras, such as the front camera of a mobile phone, an external camera of a terminal, etc.; the sensing information can be obtained by sensing the sensor arranged on the touch object.

In some embodiments, the terminal may detect whether there is an application icon on the interface currently displayed on the display screen, and start video recording through the camera only if the currently displayed interface has an application icon.

Exemplarily, when the currently displayed interface is a desktop application interface, if the terminal detects that the current desktop application interface contains an application icon, video recording is performed through the camera; or, when the currently displayed interface is the operation interface of the application, if When the terminal detects that the operation interface of the application contains the application icon of the applet in the application, it records the video through the camera.

It is understandable that if the currently displayed interface does not have an application icon, the touch object cannot start the application through the touch operation. In this case, the camera may not be turned on for video recording to reduce unnecessary consumption of terminal resources.

In this embodiment of the present application, the sensor may be a Micro-Electro-Mechanical System (MEMS, Micro-Electro-Mechanical System) sensor, a three-dimensional acceleration sensor, or a vector sensor or any other device that can obtain the relative position of the touch object and the display screen. sensor, this is not limited in this embodiment of the present application.

S202, based on the movement information, obtain the motion trajectory of the touch object;

In this embodiment of the present application, the movement trajectory of the touch object acquired by the terminal represents the movement path of the touch object, and based on the movement information, the terminal can determine the movement trajectory of the touch object.

In this embodiment of the present application, the terminal may identify or reconstruct the motion trajectory of the touch object according to the movement information; for example, when the movement information includes: the video frame sequence of the touch object, the terminal may reconstruct the 3D model based on the video frame sequence of the object , simulate the scene where the touch object moves near the display screen, so as to obtain the motion trajectory of the touch object in the 3D model; when the movement information includes: sensing information, the terminal can The relative position information between the display screens is used to reconstruct the 3D model to determine the motion trajectory.

Here, the method of 3D reconstruction may be a method based on a variational model, or a method based on SFM (Structure from Motion), or a method based on optical flow-scene flow, etc. This is not the case in the embodiments of this application. limit.

In this embodiment of the present application, the motion trajectory obtained by the terminal may adopt different representation forms, for example, the terminal may generate a motion trajectory line or a motion map, and the like.

It should be noted that the moving path of the touch object may be a straight line or a curved line, wherein the straight line may include various motion directions, and the curve may also include various curvatures. The form of expression is not limited.

For example, when the terminal is a mobile phone and the user touches the display screen of the mobile phone with his fingers, if the user holds the mobile phone with his right hand and performs the touch operation with the right thumb, the right thumb clicks on the screen that is far away from the palm. When the display screen is in the position, the motion trajectory of the thumb of the right hand is a curve, and the farther the distance is, the smaller the curvature of the curve.

S203, determining the target touch position of the touch object on the display screen according to the motion trajectory;

In the embodiment of the present application, since the terminal obtains the movement track of the touch object, it can know the movement trend or movement direction of the touch object, and based on the movement direction and other information, it can know that the touch object is far from the display screen of the terminal. Still close to the terminal's display. When the terminal determines that the touch object is close to the display screen, the terminal can determine the landing point of the touch object on the display screen based on the motion trajectory, and the landing point is the target touch position of the touch object on the display screen.

In some embodiments, the terminal can predict the target touch position based on deep learning, and after multiple touch operations of the touch object on the display screen, the terminal can obtain the historical motion trajectory of the touch object and the corresponding historical motion trajectory the actual historical touch position. Based on the historical motion trajectory and the actual historical touch position, the training of the touch position prediction model is performed. In this way, the terminal can predict the target touch position according to the touch position prediction model after acquiring the motion trajectory.

In some embodiments, the terminal may also predict the subsequent motion path of the touch object according to the motion trajectory, and based on the predicted subsequent motion path, determine that the touch object moves according to the predicted subsequent motion path, and finally falls in The target touch location on the display.

It should be noted that, after the terminal moves, its moving direction may point to the display screen, or it may be far away from the display screen. If the moving direction of the touch object points to the display screen, the touch object may touch the display screen, and the display screen will not be affected. To perform a touch operation, that is to say, only based on the motion trajectory pointing to the direction of the display screen, there is a determinable target touch position.

In the embodiment of the present application, if it is determined that the touch object does not have a landing point on the display screen based on the motion trajectory, the movement information of the touch object can be recaptured, and the target touch position can be determined based on the new movement information.

S204. Start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed;

In this embodiment of the present application, after determining the target touch position of the touch object, the terminal directly starts the process of the application to be displayed corresponding to the target touch position, and loads the result of the application to be displayed. When the startup instruction of the application to be displayed is not received, the process of the application to be displayed is started first and the resources of the application to be displayed are loaded.

The resources of the application to be displayed may include resources of the running environment required by the application, such as an instruction set, and may also include page resources to be displayed, such as page code, text, and images.

It should be noted that, after determining the target touch position, the terminal can determine the application on the target touch position as the application to be displayed, and can also set a corresponding display area for the target touch position, and display the corresponding display area. The applications in the screen area are all applications to be displayed, and the manner of determining the applications to be displayed is not limited in this embodiment of the present application.

In this embodiment of the present application, if there is no application to be displayed corresponding to the target touch position, for example, there is no application icon on the target touch position, or there is no application icon near the target touch position, the terminal can be based on the movement trajectory of the touch object. Re-target the touch position, or re-acquire the movement information of the captured touch object to determine a new motion trajectory, and then predict the virtual motion trajectory according to the new motion trajectory, and then re-determine the target touch position.

S205. If a start instruction of the application to be displayed by the touch object is received, in response to the start instruction, through the process of the application to be displayed, the resource of the application to be displayed is called to display the application to be displayed.

In this embodiment of the present application, since the terminal has already started the process of the application to be displayed and loaded the resources of the application to be displayed, after that, if it receives an instruction to start the application to be displayed, the terminal only needs to continue to complete the process of the application to be displayed Startup and resource loading. After the resource loading is completed, the application to be displayed is displayed based on the loaded resources; if the terminal has completed the process startup and resource loading of the application to be displayed when receiving the startup instruction of the application to be displayed, it can be displayed directly. Application to be displayed.

Here, the startup instruction of the application to be displayed may be clicking the icon of the application program, double-clicking the icon of the application program, or long-pressing the icon of the application program, etc., which is not limited in this embodiment of the present application.

In an application display method provided by an embodiment of the present application, a terminal captures movement information of a touch object, and the touch object is used to touch an application on a display screen of the terminal to implement an operation on the application; based on the movement information , obtain the motion trajectory of the touch object, determine the target touch position of the touch object on the display screen according to the motion trajectory, start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed. To the start command of the application to be displayed by the touch object, it responds to the start command, and through the process of the application to be displayed, calls the resources of the application to be displayed, and displays the application to be displayed. Before starting the instruction, after predicting the application to be displayed that the user wants to start according to the motion trajectory of the touch object, the application to be displayed is started and the resources are loaded, which reduces the time from the terminal receiving the application starting instruction to displaying the application. The delay time between interfaces, improving the speed at which the application is displayed.

In some embodiments of the present application, the movement information includes a video frame sequence of the touch object, and the implementation of capturing the movement information of the touch object in S201 , as shown in FIG. 3 , the method may include: S301 - S302 . as follows:

S301. Perform video recording through a set camera to obtain an original video frame sequence; the position of the camera relative to the display screen is relatively fixed;

In the embodiment of the present application, after the terminal turns on the camera for video recording, and the obtained video frame sequence is the original video frame sequence, the terminal can identify the object that touches the original video frame sequence, and obtain a video that identifies the touch object. The frame sequence is the video frame sequence of the touch object, so as to determine the motion trajectory of the touch object.

It should be noted that after the terminal starts recording video, it may only touch the display screen in a certain period of time. Therefore, only some video frames in the original video frame sequence include touch objects, and the terminal needs to recognize the algorithm according to the image. Each frame in the original video frame sequence is identified in turn to determine whether the identified video frame includes a touch object, and movement information of the touch object can be determined based on the video frame including the touch object.

Here, the image recognition algorithm can use the deep learning method, and the deep learning method can include R-FCN (Object Detection via Region-based Fully Convolutional Networks), Region-based Convolutional Neural Network (R-CNN, Region-based Convolutional Neural Networks) Networks), Spatial Pyramid Pooling Layer Net (SPP Net, Spatial Pyramid Pooling Layer Net) and other algorithms, which are not limited in this embodiment of the present application.

The position of the camera relative to the display screen is relatively fixed, and when the position of the touch object in the video frame changes, it can indicate that the touch object has moved and may be moving toward the display screen for touch operation.

In some embodiments, the touch operation of the touch object on the display screen may only be reflected in a preset area in the video frame, where the preset area is related to the relative position between the camera and the display screen. Based on this, the terminal Only a preset area in the original video frame can be identified for the touch object, and it is determined whether the preset area includes the touch object, thereby reducing the workload of the terminal for identifying the touch object and reducing the resource consumption of the terminal.

Here, the camera may be a camera built-in to the terminal, or may be an external camera of the terminal, which is not limited in this embodiment of the present application. Figure 4 shows a schematic diagram of the touch operation of the mobile phone through the finger. As shown in Figure 4, the position of the front camera that comes with the mobile phone is fixed for the display screen, and the mobile phone can obtain the video of the finger through the front camera frame sequence, and then determine that the movement trajectory of the finger is pointing from the left to the display screen.

S302. When the touch object is identified from the ith video frame of the original video frame sequence, use the ith video frame to the i+M th video frame as the video frame sequence of the touch object; i is greater than or equal to 1 An integer of ; M is an integer greater than or equal to 1.

In this embodiment of the present application, when the terminal sequentially identifies each frame in the original video frame sequence, once a touch object is identified in a certain frame, it can be determined that some video frames following this video frame must contain touch Therefore, when a touch object is identified in the ith video frame, starting from the ith video frame, a preset number of video frames can be selected as the video frame sequence of the touch object, where the preset number can be used M represents that M is an integer greater than or equal to 1.

It should be noted that when the camera starts recording video, the terminal starts to acquire video frames, and recognizes the video frames in sequence, so as to determine the motion trajectory of the touch object, and then predict the target touch position.

In some embodiments of the present application, the realization that the terminal obtains the motion trajectory of the touch object based on the movement information may include: based on the video frame sequence of the touch object, taking the center of the display screen as the origin, performing a three-dimensional image of the touch object The model is reconstructed to obtain the motion trajectory of the touch object.

In the embodiment of the present application, the terminal establishes an x-axis and a y-axis on the plane where the display screen is located, takes the center of the display screen as the origin, and uses the axis perpendicular to the display screen as the z-axis to establish a three-dimensional space coordinate system. The video frame sequence of the object, the three-dimensional reconstruction of the touch object in the three-dimensional space coordinate system, and the three-dimensional model of the touch object in the three-dimensional space coordinate system is obtained. According to the movement process of the touch object in the three-dimensional space coordinate system, the touch In this case, the motion trajectory of the touch object may be embodied as a sequence of coordinate values of the touch object in the three-dimensional space coordinate system.

In some embodiments of the present application, in S202, based on the movement information, the movement trajectory of the touch object is obtained. As shown in FIG. 5, the method includes: S501-S503. as follows:

S501, based on the identification of each video frame in the video frame sequence, determine the image information of the touch object in each video frame;

S502, determining the relative position of the touch object and the display screen based on the image information;

In the embodiment of the present application, the terminal identifies each video frame in the video frame sequence according to an image recognition algorithm, determines whether each video frame includes a touch object, and extracts the touch object from the video frame including the touch object. Image information, according to the change rule of the image information in the video frame sequence, to determine the relative position of the touch object and the display screen.

In a possible implementation manner, the image information is the number of pixels occupied by the touch object in the video frame. In the video frame sequence, if the touch object occupies more and more pixels in the video frame, the touch object will The relative position between the control object and the display screen is getting closer and closer, that is, the touch object moves towards the display screen. The relative position is getting farther and farther, that is, the touch object is far away from the display screen.

It should be noted that when the terminal recognizes the touch object in the i-th frame, it will determine the pixels occupied by the touch-sensitive object in the i-th frame and the i+1-th frame, and compare the pixels. The pixels occupied by the touch object are higher than the pixels occupied by the touch object in the ith frame, continue to determine the pixels occupied by the touch object in the i+2th frame, and when the pixels occupied by the touch object continue to increase, the When the number of video frames exceeds the preset number, it means that the touch object moves towards the display screen.

In another possible implementation manner, the image information may include depth of field information of the touch object, the terminal determines the distance between the touch object and the camera according to the depth of field information, and then determines the touch according to the relative position between the camera and the display screen. The distance between the object and the display screen. For example, the terminal is a mobile phone, and the mobile phone records video through the front camera. In the video frame sequence, if the depth of field of the touch object increases, it means that the distance between the touch object and the camera increases. If the depth of field of the touch object is getting smaller and smaller, it means that the distance between the touch object and the camera is getting closer and closer, that is, the touch object moves towards the display screen.

S503 , when the relative position indicates that the touch object moves toward the display screen, based on the video frame sequence, taking the center of the display screen as the origin, reconstruct the three-dimensional model of the touch object, and obtain the motion trajectory of the touch object.

In this embodiment of the present application, after the terminal determines the relative position based on the image information, if the phase position indicates that the touch object moves toward the display screen, it indicates that the terminal may touch the display screen. The frame sequence is used to reconstruct the 3D model of the touch object, so as to obtain the motion trajectory of the touch object.

It should be noted that if the touch object moves, but the direction is away from the display screen, the touch object will not touch the display screen. At this time, the terminal may not obtain the motion trajectory to reduce terminal resource consumption. , the terminal will continue to identify video frames and obtain image information, until it is determined that the touch object moves toward the display screen, reconstruct the 3D model of the touch object, and obtain the motion trajectory of the touch object.

In some embodiments of the present application, the movement information includes: sensing information, and the realization that the terminal captures the movement information of the touch object includes: based on a sensing device set on the touch object, acquiring the information between the touch object and the terminal. Sensing information; the sensing device communicates with the terminal.

In this embodiment of the present application, the terminal acquires sensing information through communication with the sensing device, and the sensing information is obtained by sensing between the touch object and the terminal by the sensing device, and is used to represent the touch The positional relationship between the control object and the display screen.

Here, the sensing device may be a sensor directly implanted in the touch object, or may be a wearable device disposed on the touch object by wearing, which is not limited in this embodiment of the present application.

In some embodiments of the present application, the realization that the terminal obtains the motion trajectory of the touch object based on the movement information includes: determining the distance sequence between the touch object and the display screen of the terminal based on the sensing information; based on the distance sequence, Taking the center of the display screen as the origin, the three-dimensional model of the touch object is reconstructed, and the motion trajectory of the touch object is obtained.

In the embodiment of the present application, the terminal can determine the distance value between the touch object and the display screen of the terminal based on the sensing information, and the terminal can obtain the distance sequence after continuously acquiring the sensing information. When it changes, it means that the touch object is not moving. When the distance value in the distance sequence is getting larger and larger, it means that the touch object is moving away from the display screen. In the above two cases, the terminal does not further process the distance sequence. At this time, the terminal needs to continue to obtain sensing information to determine a new distance sequence; when the distance value in the distance sequence is getting smaller and smaller, it means that the touch object is moving towards the display screen. The center is the origin, and the 3D model is reconstructed to obtain the motion trajectory of the touch object.

In some embodiments of the present application, in S203, the target touch position of the touch object on the display screen is determined according to the motion trajectory. As shown in FIG. 6, the method may include: S601-S602. as follows:

S601. Predict the virtual motion trajectory of the touch object according to the motion trajectory; the virtual motion trajectory represents the motion path predicted by the touch object;

In the embodiment of the present application, after the terminal has determined the motion trajectory of the touch object, the direction of the motion trajectory can be determined as the direction in which the touch object is about to move, and the motion path of the touch object can be predicted along the direction of the motion trajectory.

It should be noted that the method for the terminal to predict the virtual motion trajectory may be to extend the motion trajectory of the touch object along the direction of the motion trajectory, and the extended motion trajectory is the predicted virtual motion trajectory of the touch object; Mathematical statistics methods, such as Markov model, Naive Bayes model, Gaussian process model, etc., can also be machine learning methods, such as neural network, Support Vector Machine (SVM), etc. The application examples are not limited.

S602: Determine the touch position of the virtual motion track on the display screen, and use the touch position as the target touch position.

In this embodiment of the present application, after predicting the virtual motion trajectory, the terminal may determine the touch position on the display screen when the touch object moves along the virtual motion trajectory, and use the touch position as the target touch position.

In some embodiments of the present application, the realization that the terminal starts the process of the application to be displayed corresponding to the target touch position, and loads the resources of the application to be displayed, may include: when there is an application icon at the target touch position, starting the application with the application The process of the application to be displayed corresponding to the icon, and the resources of the application to be displayed are loaded.

Fig. 7 is an exemplary schematic diagram of predicting the touch position of the touch object. As shown in Fig. 7, the center of the display screen is taken as the origin, and the x-axis and the y-axis are established on the plane where the display screen of the mobile phone is located. The z-axis is established in the direction perpendicular to the plane where it is located, and three-dimensional reconstruction is performed to predict the virtual motion trajectory of the finger as shown by the dotted line 71 in the figure, and then determine the target touch position of the finger on the display screen. The application icon on the target touch position is The application icon of the application to be displayed.

In some embodiments of the present application, in S204, the process of starting the application to be displayed corresponding to the target touch position, and loading the resources of the application to be displayed is realized. As shown in FIG. 8, the method may include: S801-S803. as follows:

S801. Determine a corresponding target area on the display screen according to the target touch position;

S802, determining the application in the target area as the application to be displayed;

In this embodiment of the present application, the terminal may set a preset rule to associate the target touch position with the target area, so that after determining the target touch position, the terminal can use the application in the target area as the application to be displayed.

In this embodiment of the present application, after determining the target touch position, the terminal may take the target touch position as the center of the circle, and take the application within the circular area of the preset radius as the application to be displayed, or take the target touch position as the center , the application program within the square area of the preset side length is the application to be displayed. The terminal can also divide the display screen into multiple display areas in advance. After the target touch position is determined, the target touch position is located in the display area The application program is determined as the application to be displayed, which is not limited in this embodiment of the present application.

For example, as shown in Figure 9, the display screen of the mobile phone is divided into 6 display areas on average. After the mobile phone records the video according to the front camera, it predicts the virtual motion trajectory of the finger, and then determines that the target touch position is in the fourth position. In a display area, then, the K application and the L application in the fourth display area are used as the applications to be displayed.

It can be understood that since the target touch position is determined according to the predicted virtual motion trajectory, all application icons in the display area corresponding to the target touch position are determined as applications to be displayed, and these applications are started and loaded in advance. When any one of the display applications receives the startup instruction, the application to be displayed can be displayed immediately, which further improves the response speed of the application.

S803. Start the process of the application to be displayed, and load the resources of the application to be displayed.

In the embodiment of the present application, after the terminal determines the application to be displayed, it will start the process of the application to be displayed, and load the resources of the application to be displayed, thereby reducing the process of starting the application to be displayed by the terminal after the application program receives the start instruction, and The time to load the resources of the application to be displayed increases the display speed of the application.

In some embodiments of the present application, after the terminal starts the process of the application to be displayed corresponding to the target touch position and loads the resources of the application to be displayed, if the terminal does not receive the touch object within the preset time period, the application to be displayed is not received The startup command ends the process of the application to be displayed and releases the resources of the application to be displayed.

In this embodiment of the present application, within a preset time period, the situation that the terminal does not receive a start instruction of the application to be displayed by the touch object may include: all applications on the display screen have not received the start instruction of the touch object, or The application that receives the start instruction of the touch object is not the application to be displayed. Based on this, it can be determined that the application to be displayed is not the application that the user wants to open, and it does not need to be displayed. Therefore, it is necessary to end the process of the application to be displayed and release the application to be displayed. Display the resources of the application to reduce unnecessary resource consumption of the terminal.

An embodiment of the present application provides a terminal. As shown in FIG. 10 , the terminal 10 includes:

The capture module 1001 is used to capture the movement information of the touch object; the touch object is used to touch the application on the display screen of the terminal to realize the operation of the application;

an obtaining module 1002, configured to obtain the motion trajectory of the touch object based on the movement information;

A determination module 1003, configured to determine the target touch position of the touch object on the display screen according to the motion track;

A startup module 1004, configured to start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed;

The display module 1005 is configured to respond to the start-up instruction and call the resources of the application to be displayed through the process of the application to be displayed in response to the start instruction of the application to be displayed by the touch object, and display the application to be displayed.

In some embodiments, the capture module 1001 is specifically configured to perform video recording through a set camera to obtain an original video frame sequence; the position of the camera relative to the display screen is relatively fixed; when the original video frame sequence is obtained from the camera When the touch object is identified in the i-th video frame of , the i-th video frame to the i+M-th video frame is used as the video frame sequence of the touch-control object; the i is greater than An integer equal to 1; the M is an integer greater than or equal to 1.

In some embodiments, the acquiring module 1002 is specifically configured to reconstruct a 3D model of the touch object based on the video frame sequence of the touch object, taking the center of the display screen as the origin, to obtain the touch object The trajectory of the object's motion.

In some embodiments, the acquiring module 1002 is specifically configured to determine the image information of the touch object in each video frame based on the identification of each video frame in the video frame sequence; based on the image information, Determine the relative position of the touch object and the display screen; when the relative position represents the movement of the touch object to the display screen, based on the video frame sequence, take the center of the display screen as the origin , reconstructing a three-dimensional model of the touch object to obtain a motion trajectory of the touch object.

In some embodiments, the capture module 1001 is further configured to acquire sensing information between the touch object and the terminal based on the sensing device set on the touch object; the sensing device is connected to the terminal. terminal communication.

In some embodiments, the acquiring module 1002 is specifically configured to determine, based on the sensing information, a sequence of distances between the touch object and the display screen of the terminal; The center of the touch object is the origin, and the three-dimensional model of the touch object is reconstructed to obtain the motion trajectory of the touch object.

In some embodiments, the determining module 1003 is specifically configured to predict the virtual motion trajectory of the touch object according to the motion trajectory; the virtual motion trajectory represents the motion path predicted by the touch object; The touch position of the virtual motion track on the display screen, and the touch position is used as the target touch position.

In some embodiments, the starting module 1004 is specifically configured to start the process of the application to be displayed corresponding to the application icon when there is an application icon at the target touch position, and load the resources of the application to be displayed.

In some embodiments, the startup module 1004 is specifically configured to determine the corresponding target area on the display screen according to the target touch position; determine the application in the target area as the application to be displayed; start the application to be displayed The process of the application is displayed, and the resources of the application to be displayed are loaded.

In some embodiments, after starting the process of the application to be displayed corresponding to the target touch position and loading the resources of the application to be displayed, the display module 1005 is further configured to: The start instruction of the application to be displayed by the touch object ends the process of the application to be displayed, and releases the resources of the application to be displayed.

An embodiment of the present application provides a terminal. As shown in FIG. 11 , the terminal 11 includes:

The processor 1101, the memory 1102 and the communication bus 1103. The memory 1102 communicates with the processor 1101 through the communication bus 1103. The memory 1102 stores one or more programs executable by the processor 1101. The processor 1101 executes any one of the application display methods in the foregoing embodiments.

It can be understood that the memory in this embodiment may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory, Optical disc, or Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage can be a magnetic disk storage or a magnetic tape storage. Volatile memory may be Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (SRAM), Memory (DynamicRandom Access Memory, DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (DDRSDRAM) Random Access Memory (Enhanced Synchronous Dynamic Random Access Memory, ESDRAM), Synchronous Link Dynamic Random Access Memory (SyncLink Dynamic Random Access Memory, SLDRAM), Direct Rambus Random Access Memory (Direct Rambus Random Access Memory, DRRAM). The memories described in the embodiments of the present application are intended to include, but not be limited to, these and any other suitable types of memories.

The methods disclosed in the above embodiments of the present application may be applied to a processor, or implemented by a processor. A processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above-mentioned method can be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software. The above-mentioned processors may be general-purpose processors, DSPs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The processor may implement or execute the methods, steps, and logical block diagrams disclosed in the embodiments of this application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, the storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps of the foregoing method in combination with its hardware.

Embodiments of the present application provide a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors 1101, the processors 1101 execute the application program display method of the embodiments of the present application.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the protection scope of the present application.

Claims (13)

1. a display method of an application program, is characterized in that, comprises: Capturing the movement information of the touch object; the touch object is used to touch the application on the display screen of the terminal to realize the operation of the application; Based on the movement information, obtain the motion trajectory of the touch object; determining the target touch position of the touch object on the display screen according to the motion track; Start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed; If an activation instruction of the application to be displayed by the touch object is received, in response to the activation instruction, the resource of the application to be displayed is invoked through the process of the application to be displayed, and the application to be displayed is displayed. 2 . The method according to claim 1 , wherein the movement information comprises: a video frame sequence of the touch object; and the capturing movement information of the touch object comprises: 2 . Perform video recording through the set camera to obtain the original video frame sequence; the position of the camera relative to the display screen is relatively fixed; When the touch object is identified from the i-th video frame in the original video frame sequence, the i-th video frame to the i+M-th video frame are used as the touch object's Video frame sequence; the i is an integer greater than or equal to 1; the M is an integer greater than or equal to 1. 3 . The method according to claim 2 , wherein the acquiring the motion trajectory of the touch object based on the movement information comprises: 3 . Based on the video frame sequence of the touch object, taking the center of the display screen as the origin, a three-dimensional model of the touch object is reconstructed to obtain a motion trajectory of the touch object. 4 . The method according to claim 3 , wherein the acquiring the motion trajectory of the touch object based on the movement information comprises: 5 . Determine the image information of the touch object in each video frame based on the identification of each video frame in the video frame sequence; determining the relative position of the touch object and the display screen based on the image information; When the relative position indicates that the touch object moves toward the display screen, based on the video frame sequence, taking the center of the display screen as the origin, reconstruct the three-dimensional model of the touch object, and obtain the The movement track of the touch object. 5. The method according to claim 2, wherein the movement information comprises: sensing information; and the capturing movement information of the touch object comprises: Based on the sensing device set on the touch object, the sensing information between the touch object and the terminal is acquired; the sensing device communicates with the terminal. 6 . The method according to claim 5 , wherein the obtaining the motion trajectory of the touch object based on the movement information comprises: 6 . determining a sequence of distances between the touch object and the display screen of the terminal based on the sensing information; Based on the distance sequence, taking the center of the display screen as the origin, the three-dimensional model of the touch object is reconstructed, and the motion trajectory of the touch object is obtained. 7. The method according to any one of claims 1 to 6, wherein the determining a target touch position of the touch object on the display screen according to the motion trajectory comprises: According to the motion trajectory, a virtual motion trajectory of the touch object is predicted; the virtual motion trajectory represents the motion path predicted by the touch object; A touch position of the virtual motion track on the display screen is determined, and the touch position is used as the target touch position. 8. The method according to any one of claims 1 to 7, wherein the starting a process of the application to be displayed corresponding to the target touch position, and loading the resources of the application to be displayed, comprises: When an application icon exists at the target touch position, the process of the application to be displayed corresponding to the application icon is started, and the resources of the application to be displayed are loaded. 9 . The method according to claim 1 , wherein the starting a process of the application to be displayed corresponding to the target touch position and loading the resources of the application to be displayed comprises: 10 . Determine the corresponding target area on the display screen according to the target touch position; determining the application in the target area as the application to be displayed; The process of the application to be displayed is started, and the resources of the application to be displayed are loaded. 10 . The method according to claim 1 , wherein after starting the process of the application to be displayed corresponding to the target touch position and loading the resources of the application to be displayed, the method further comprises: 10 . If the start instruction of the application to be displayed by the touch object is not received within a preset time period, the process of the application to be displayed is terminated, and the resources of the application to be displayed are released. 11. A terminal, characterized in that, comprising: a capture module for capturing movement information of a touch object; the touch object is used to touch an application on the display screen of the terminal to operate the application; an acquisition module, configured to acquire the motion trajectory of the touch object based on the movement information; a determining module, configured to determine the target touch position of the touch object on the display screen according to the motion trajectory; a startup module, configured to start the process of the application to be displayed corresponding to the target touch position, and load the resources of the application to be displayed; A display module, configured to respond to the start-up instruction, call the resource of the application to be displayed through the process of the application to be displayed, and display the Describe the application to be displayed. 12. A terminal, characterized in that the terminal comprises: a processor and a memory for storing a computer program that can be run on the processor, Wherein, the processor is configured to execute the steps of the method of any one of claims 1 to 10 when running the computer program. 13. A computer-readable storage medium, wherein a computer program is stored, and when the computer program is executed by one or more processors, the processor executes any one of the claims 1 to 10 the steps of the method described in item.

Images