WO2016095689A1 - Recognition and searching method and system based on repeated touch-control operations on terminal interface - Google Patents

Abstract

A recognition and searching method and system based on repeated touch-control operations on a terminal interface, which relate to the technical fields of searching. Repeated touch-control operations of a user on a mask are received. An interface area to be searched is selected on the mask according to an overlap degree between the repeated touch-control operations. Elements contained in the interface area to be searched are recognised. Searching is performed according to the recognised elements. Not only is the process of troublesomely inputting keywords in a search box omitted, searching is more convenient, redundant elements in the interface area to be searched are avoided, and the searching precision is increased.

Description

Method and system for performing recognition search based on multiple touch operations of terminal interface Technical field

The present invention relates to the field of search technologies, and in particular, to a method and system for performing a search and search based on a multi-touch operation of a terminal interface.

Background technique

The search service on the terminal (for example, a smart phone) is input based on the search box, and the user needs to input keywords into the search box cumbersomely, so that the search result related to the keyword can be displayed on the terminal display interface by the search engine. , but the search method is very inefficient and the search experience is very poor.

In particular, when the user is using the terminal, when there is an immediate search requirement for characters, images, and the like on the terminal display interface, the search application needs to be opened first, and after the application pops up in the search box, the search box is further Entering keywords is very inconvenient.

Summary of the invention

In view of the above problems, the present invention has been made in order to provide a method for performing a recognition search based on a terminal interface multiple touch operation that overcomes the above problems or at least partially solves the above problems, and a corresponding multi-touch based on a terminal interface. A system that performs a search search.

According to an aspect of the present invention, a method for performing a recognition search based on a plurality of touch operations of a terminal interface is provided, and the method includes:

When the user triggers the interface touch search request, a mask covering the display area of the terminal interface is generated in the current interface display area of the terminal;

Receiving a plurality of touch operations of the user on the mask, and selecting an interface area to be searched on the mask according to the degree of coincidence between the multiple touch operations;

Identifying elements included in the to-be-searched interface area, and searching according to the identified elements.

According to an aspect of the present invention, a system for performing a recognition search based on a plurality of touch operations of a terminal interface is provided, and the system includes:

a mask generator, configured to generate a mask covering the display area of the terminal interface in the current interface display area of the terminal when the user triggers the interface touch search request;

The touch receiver is adapted to receive a plurality of touch operations of the user on the mask, and select an interface area to be searched on the mask according to a degree of coincidence between the multiple touch operations;

The recognition searcher is adapted to identify an element included in the interface area to be searched, and perform a search according to the identified element.

According to another aspect of the present invention, a computer program is presented comprising computer readable code that, when executed on a computing device, causes the computing device to perform the terminal based interface described above A method of recognizing a search by a touch operation.

According to still another aspect of the present invention, a computer readable medium is proposed, wherein the computer program described above is stored.

The invention selects a to-be-searched interface area on the mask according to the degree of coincidence between the multiple touch operations by receiving a plurality of touch operations of the user on the mask, and identifies that the interface to be searched includes The element searches for the identified element, which not only omits the process of the user inputting the keyword into the search box cumbersomely, makes the search more convenient, and avoids the unnecessary elements in the interface area to be searched, thereby improving the search. Precision.

The above description is only an overview of the technical solutions of the present invention, and the above-described and other objects, features and advantages of the present invention can be more clearly understood. Specific embodiments of the invention are set forth below.

DRAWINGS

Various other advantages and benefits will become apparent to those skilled in the art from a The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting. Throughout the drawings, the same reference numerals are used to refer to the same parts. In the drawing:

1 is a flow chart showing the steps of a method for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention;

2 is a flow chart showing the steps of a method for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention;

3 is a diagram showing an example of selecting an interface region to be searched on the mask on an embodiment of the present invention;

4 is a diagram showing an example of selecting an interface region to be searched on the mask according to an embodiment of the present invention;

FIG. 5 is a diagram showing an example of selecting an interface region to be searched on the mask according to an embodiment of the present invention; FIG.

6 is a flow chart showing the steps of a method for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention;

7 is a structural block diagram of a system for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention;

FIG. 8 is a structural block diagram of a system for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention; FIG.

Figure 9 shows schematically a block diagram of a computing device for performing the method according to the invention;

Fig. 10 schematically shows a storage unit for holding or carrying program code implementing the method according to the invention.

Specific embodiment

The specific embodiments of the present invention are further described in detail below with reference to the drawings and embodiments. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.

1 is a flow chart showing the steps of a method for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention; referring to FIG. 1, the method includes:

S101: When the user triggers the interface touch search request, generating a mask covering the display area of the terminal interface in the current interface display area of the terminal;

It should be noted that the user can trigger the interface touch search request in multiple ways, for example, the user can trigger the search floating window on the screen, or trigger the interface touch search request through shortcuts such as buttons or gestures. This example does not limit this.

S102: Receive a plurality of touch operations of the user on the mask, and select an interface region to be searched on the mask according to the degree of coincidence between the multiple touch operations;

The touch operation of the user on the mask, for example, the user touches the touch point controlled by the screen through a finger or a sensor pen, and the selection manner can be selected by sliding, or circleing on the screen with a finger/stylus, etc., thereby selecting The interface area to be searched and the like.

S103: Identify an element included in the to-be-searched interface area, and perform a search according to the identified element.

It should be noted that, in the normal search situation, the interface to be searched only contains one type of element, such as a piece of text, or a picture, and of course, may also contain multiple types of elements, such as containing both text and The embodiment of the present invention does not limit this, and the types of the elements include, but are not limited to, characters, pictures, and symbols.

It is understood that when the user performs a touch operation on the mask, the user receives multiple touch operations on the mask, and selects a search interface on the mask according to the multiple touch operations. An area that identifies an element included in the area of the interface to be searched. There are many ways to identify them. For example, OCR optical character recognition technology can be used to identify text elements. You can also recognize text elements by calling controls supported by the operating system, such as the Uiautomater control. You can also identify the search by face recognition technology. The elements contained in the interface area are not image faces, and other pictures can be identified by similar image recognition technology on the Internet. Of course, as well as other recognition technologies, multiple recognition technologies can be combined to improve the recognition. degree. In this way, the identified element directly jumps to the search interface, thereby omitting the process of the user inputting the keyword into the search box cumbersomely, or the user needs to copy the page content first, and then open the tedious process of the search client pasteing into the search box. Make the entire search interaction more convenient.

Of course, when the interface area to be searched is selected on the mask according to the multiple touch operation, the general processing manner is to use multiple touch operations on the mask as a whole, but the selection method will be In the embodiment, the user touches multiple touch operations on the mask, and the touch operation is performed according to the multiple touch operations. Selecting an interface area to be searched on the mask, identifying an element included in the interface area to be searched, searching according to the identified element, and avoiding unnecessary elements in the interface area to be searched. Improve the accuracy of the search.

2 is a flow chart showing the steps of a method for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention; referring to FIG. 2, the method includes:

S201: When the user triggers the interface touch search request, the current interface display area of the terminal is screened to obtain a screenshot of the current interface display area of the terminal, and the screenshot is used as a mask covering the display area of the terminal interface;

Understandably, the current interface display area of the terminal is first screened, and then the screenshot is taken as an overlay. The mask of the terminal interface display area can prevent the application or program running in the background of the terminal from being affected, and can return to the original browsing interface after the entire search interaction ends, thereby improving the user experience to a certain extent.

Of course, if the screenshot is only used as a pure transparent mask covering the display area of the terminal interface, the content may be misidentified by the user as the terminal does not generate a response because the mask is consistent with the content displayed on the display interface of the terminal interface. In order to avoid this problem, in the embodiment, the mask is set to be semi-transparent, and the user can be effectively presented.

S202: detecting a plurality of touch trajectories generated by multiple touch operations of the user on the mask, and displaying, on the mask, a plurality of interface regions to be searched corresponding to the plurality of touch trajectories respectively. The presentation manner of the search interface area is different from the other areas on the mask; if the coincidence degree between the two search interface areas is not less than the coincidence threshold, the two search interface areas are merged into one to be Searching the interface area, otherwise the two to-be-searched interface areas are used as independent two to-be-searched interface areas;

It should be noted that, when the interface to be searched is selected on the mask, if the presentation manner of the interface to be searched is not distinguished from other regions on the mask, it may be difficult for the user to determine the to-be-searched. In order to avoid the problem, in the embodiment, the presentation manner of the interface area to be searched is distinguished from other areas on the mask.

Certainly, in order to realize the difference of the presentation manner, a plurality of presentation manners may be adopted. In this embodiment, the display manner of the interface region to be searched may be displayed in a highlighted or fully transparent form, and the other on the mask. The area may still be displayed in a semi-transparent form, or may be displayed in a grayscale or fuzzy form, and other areas on the mask may be displayed in a grayscale form, and the interface area to be searched may be in a fully transparent form. It is to be noted that, of course, the difference in the manner of presentation may be implemented in other ways, which is not limited by the embodiment of the present invention.

It can be understood that, when the interface area to be searched is selected on the mask according to the degree of coincidence between the multiple touch operations, a plurality of manners may be used. In this embodiment, two interface areas to be searched are determined. Whether the coincidence degree is less than the coincidence threshold, if not less than, the two to-be-searched interface areas are merged into one to-be-searched interface area, otherwise the two to-be-searched interface areas are regarded as independent two to-be-searched interface areas, Further avoiding the inclusion of redundant elements in the interface area to be searched provides the accuracy of the search.

Therefore, step S202 is substantially a preferred embodiment of step S102 in the embodiment shown in FIG. 1. Way of application.

In this embodiment, the coincidence threshold value is 30%. Referring to FIG. 3, the interface areas A and B to be searched corresponding to the touch track of the user are displayed on the mask, because the interface areas A and B to be searched are The degree of coincidence is less than 30%, so the interface areas A and B to be searched are regarded as independent two interface areas to be searched;

Referring to FIGS. 4 and 5, the interface areas A and B to be searched corresponding to the user's touch trajectory are displayed on the mask. Since the degree of coincidence between the interface areas A and B to be searched is higher than 30%, the interface area to be searched is to be searched. A and B are merged into one interface area to be searched. It can be understood that the merged interface area to be searched is the same as the interface area B to be searched.

Of course, the coincidence threshold can be adjusted accordingly according to needs, and set to other values.

S203: Identify an element included in the interface area to be searched, remove the mask, and push the element to a search box located in the display area of the terminal interface; if the user clicks on the display area of the terminal interface The first icon is searched based on the element.

It should be noted that after the elements included in the interface to be searched are identified, the elements need to be pushed to the search box located in the display area of the terminal interface, and if the mask is still retained, it is difficult to use. The elements that are pushed into the search box are seen, so the mask needs to be removed after identifying the elements contained in the interface area to be searched.

Therefore, step S203 is substantially a preferred embodiment of step S103 in the embodiment shown in FIG. 1.

It can be understood that, since the element is to be pushed to the search box located in the display area of the terminal interface, the search box needs to be popped up in the display area of the terminal interface.

In order to facilitate the user to perform the search after confirming the element, after the user clicks the first icon that triggers the search, the search is performed according to the element.

It should be noted that, in step S202, the interface area to be searched after the selection according to the degree of coincidence may have a merged interface area to be searched, so as to avoid identifying redundant elements as much as possible, in this embodiment, The board selects the smallest rectangular frame in which each interface area to be searched is located, and identifies the manner of the elements contained in the minimum rectangular frame.

When the elements included in the minimum rectangular frame are identified, the boundary of the minimum rectangular frame may be first determined, and then the left and right coordinate points of the minimum rectangular frame may be determined, and finally determined by the four coordinate points. The element contained within the smallest rectangle.

FIG. 6 is a flow chart showing the steps of a method for performing a search and a search based on a plurality of touch operations on a terminal interface according to an embodiment of the present invention; the method in this embodiment includes:

S301: When the user triggers the interface touch search request, generating a mask covering the display area of the terminal interface in the current interface display area of the terminal;

S302: Receive a plurality of touch operations of the user on the mask, and select an interface area to be searched on the mask according to the degree of coincidence between the multiple touch operations;

S303: Identify an element included in the to-be-searched interface area, and perform a search according to the identified element;

S304: Display a search result item related to the element in the terminal interface display area.

It should be noted that steps S301 to S303 are substantially the same as S101 to S103 of the embodiment shown in FIG. 1, and thus are not described herein again.

It should be noted that the search result item may be displayed in various forms. In this embodiment, the element related to the element is displayed in the form of a combination of an application card, a text item, and a graphic in the terminal interface display area. The search result item, of course, can also be displayed in other forms, which is not limited by the embodiment of the present invention.

It can be understood that, in order to facilitate the user to confirm the search interface area, the elements included in the search interface area are identified, optionally, based on the embodiments shown in FIG. 1 and FIG. Before identifying the element included in the to-be-searched interface area, the method further includes: if the user clicks on the second icon disposed in the mask, triggering to identify an element included in the to-be-searched interface area.

Certainly, other icons may be set in the mask, for example, a repaint icon for reselecting the to-be-searched interface area, a return icon for exiting, etc., may be added as needed, and the embodiment of the present invention is This is not limited.

It is to be understood that the main body of the method of the embodiment shown in FIG. 1 and FIG. 2 is a terminal, and the terminal may be a desktop computer with a touch screen, a notebook, a mobile phone, a tablet computer, etc., in the embodiment of the present invention. There is no restriction on this.

For the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the embodiments of the present invention are not limited by the described action sequence, because the embodiment according to the present invention Some steps can be performed in other orders or at the same time. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

FIG. 7 is a structural block diagram of a system for performing a recognition search based on a multiple touch operation of a terminal interface according to an embodiment of the present invention; referring to FIG. 7, the system includes:

a mask generator, configured to generate a mask covering the display area of the terminal interface in the current interface display area of the terminal when the user triggers the interface touch search request;

The touch receiver is adapted to receive a plurality of touch operations of the user on the mask, and select an interface area to be searched on the mask according to a degree of coincidence between the multiple touch operations;

The recognition searcher is adapted to identify an element included in the interface area to be searched, and perform a search according to the identified element.

In an optional embodiment of the present invention, the touch receiver is further configured to detect a plurality of touch trajectories generated based on a plurality of touch operations of the user on the mask, in the mask A plurality of to-be-searched interface regions respectively corresponding to the plurality of touch trajectories are displayed, and the display manner of the to-be-searched interface regions is different from other regions on the mask; if two interface regions are to be searched If the coincidence degree is not less than the coincidence threshold, the two to-be-searched interface areas are merged into one to-be-searched interface area, otherwise the two to-be-searched interface areas are regarded as independent two to-be-searched interface areas.

In an optional embodiment of the present invention, the identification searcher is further adapted to select, on the mask, a minimum rectangular frame in which each interface region to be searched is located, and identify an element included in the minimum rectangular frame. .

In an optional embodiment of the present invention, the mask generator is further adapted to: when the user triggers the interface touch search request, screen the current interface display area of the terminal to obtain the current interface display area of the terminal. The screenshot is taken as a mask covering the display area of the terminal interface.

In an optional embodiment of the present invention, the mask generator is further adapted to use the screenshot as a mask in a translucent state covering the display interface of the terminal interface.

In an optional embodiment of the present invention, the display manner of the interface area to be searched is displayed in a highlighted or fully transparent form;

and / or,

Other areas on the mask are displayed in grayscale, blurred or translucent form.

In an optional embodiment of the present invention, the identification searcher is further adapted to remove the mask and push the element to a search box located in a display area of the terminal interface;

When the user clicks on the first icon located in the display area of the terminal interface, the identification search And is also adapted to search based on the elements.

In an optional embodiment of the present invention, when the user clicks on the second icon disposed in the mask, the identification searcher is further adapted to trigger execution of searching for a webpage related to the information according to the identified information. .

FIG. 8 is a structural block diagram of a system for performing a recognition search based on a plurality of touch operations of a terminal interface according to an embodiment of the present invention; referring to FIG. 8, the system includes:

a mask generator, configured to generate a mask covering the display area of the terminal interface in the current interface display area of the terminal when the user triggers the interface touch search request;

The touch receiver is adapted to receive a plurality of touch operations of the user on the mask, and select an interface area to be searched on the mask according to a degree of coincidence between the multiple touch operations;

The recognition searcher is adapted to identify an element included in the interface area to be searched, and perform a search according to the identified element.

A search presenter adapted to display search result items related to the element in the terminal interface display area.

In an optional embodiment of the present invention, the search presenter is further adapted to display search results related to the element in a combination of an application card, a text item, and a graphic in the terminal interface display area. item.

In an optional embodiment of the invention, the element comprises at least one of a text, a picture, and a symbol.

For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

It should be noted that in the various components of the system of the present invention, the components therein are logically divided according to the functions to be implemented, but the present invention is not limited thereto, and the components may be re-divided as needed. Alternatively, for example, some components may be combined into a single component, or some components may be further broken down into more subcomponents.

The various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the systems for identifying searches based on terminal interface multiple touch operations in accordance with embodiments of the present invention. Some or all of the features of the part. The invention may also be implemented to perform the description herein Some or all of the methods or device programs (eg, computer programs and computer program products) of the methods described. Such a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

For example, Figure 9 schematically illustrates a block diagram of a computing device for performing the method in accordance with the present invention. The computing device conventionally includes a processor 910 and a computer program product or computer readable medium in the form of a memory 920. The memory 920 may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), an EPROM, a hard disk, or a ROM. Memory 920 has a memory space 930 for program code 931 for performing any of the method steps described above. For example, storage space 930 for program code may include various program code 931 for implementing various steps in the above methods, respectively. The program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such a computer program product is typically a portable or fixed storage unit as described with reference to FIG. The storage unit may have storage segments, storage spaces, and the like that are similarly arranged to memory 920 in the computing device of FIG. The program code can be compressed, for example, in an appropriate form. Typically, the storage unit comprises computer readable code 931' for performing the steps of the method according to the invention, ie code that can be read by a processor, such as 910, which, when run by the computing device, causes the calculation The device performs the various steps in the methods described above.

"an embodiment," or "an embodiment," or "an embodiment," In addition, it is noted that the phrase "in one embodiment" is not necessarily referring to the same embodiment.

In the description provided herein, numerous specific details are set forth. However, it is understood that the embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of the description.

It is to be noted that the above-described embodiments are illustrative of the invention and are not intended to be limiting, and that the invention may be devised without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as a limitation. The word "comprising" does not exclude the presence of the elements or steps that are not recited in the claims. Bit The word "a" or "an" The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by the same hardware item. The use of the words first, second, and third does not indicate any order. These words can be interpreted as names.

In addition, it should be noted that the language used in the specification has been selected for the purpose of readability and teaching, and is not intended to be construed or limited. Therefore, many modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The disclosure of the present invention is intended to be illustrative, and not restrictive, and the scope of the invention is defined by the appended claims.

The above embodiments are merely illustrative of the present invention and are not intended to be limiting of the invention, and various modifications and changes can be made without departing from the spirit and scope of the invention. Equivalent technical solutions are also within the scope of the invention, and the scope of the invention is defined by the claims.

Claims (24)

A method for performing a recognition search based on a plurality of touch operations of a terminal interface, wherein the method includes: When the user triggers the interface touch search request, a mask covering the display area of the terminal interface is generated in the current interface display area of the terminal; Receiving a plurality of touch operations of the user on the mask, and selecting an interface area to be searched on the mask according to the degree of coincidence between the multiple touch operations; Identifying elements included in the to-be-searched interface area, and searching according to the identified elements. The method of claim 1, wherein the receiving a user's multiple touch operations on the mask and selecting on the mask according to the touch coincidence between the multiple touch operations The area to be searched for the interface further includes: Detecting a plurality of touch trajectories generated by the user's multiple touch operations on the mask, and displaying a plurality of to-be-searched interface regions corresponding to the plurality of touch trajectories respectively on the mask The display interface area is displayed differently from other areas on the mask; If the coincidence degree between the two to-be-searched interface areas is not less than the coincidence threshold, the two to-be-searched interface areas are merged into one to-be-searched interface area, otherwise the two to-be-searched interface areas are regarded as independent two to be searched. Interface area. The method of any of claims 1-2, wherein the identifying the elements included in the to-be-searched interface region comprises: Selecting a minimum rectangular frame in which each interface region to be searched is located on the mask, and identifying elements included in the minimum rectangular frame. The method according to any one of claims 1-3, wherein when the user triggers an interface touch search request, a mask covering the display area of the terminal interface is generated in the current interface display area of the terminal, and further includes : When the user triggers the interface touch search request, the current interface display area of the terminal is screened to obtain a screenshot of the current interface display area of the terminal, and the screenshot is used as a mask covering the display area of the terminal interface. The method according to any one of claims 1-4, wherein the using the screenshot as a mask covering the display area of the terminal interface further comprises: The screenshot is used as a mask in a semi-transparent state covering the display area of the terminal interface. The method of any of claims 1-5, wherein the interface area to be searched Display methods are highlighted or fully transparent; and / or, Other areas on the mask are displayed in grayscale, blurred or translucent form. The method of any of claims 1-6, wherein the searching based on the identified elements further comprises: Removing the mask and pushing the element to a search box located in a display area of the terminal interface; If the user clicks on the first icon located in the display area of the terminal interface, the search is performed according to the element. The method of any of claims 1 to 7, wherein the searching before the searching based on the identified elements further comprises: If the user clicks on the second icon set in the mask, triggering execution of searching for the webpage related to the information according to the identified information. The method according to any one of claims 1 to 8, wherein after the searching according to the identified element, the method further comprises: A search result item related to the element is displayed in the terminal interface display area. The method of any one of claims 1 to 9, wherein the displaying the search result item related to the element in the terminal interface display area further comprises: Search result items related to the element are displayed in the terminal interface display area in the form of a combination of an application card, a text entry, and a graphic. The method of any of claims 1-10, wherein the element comprises at least one of a text, a picture, and a symbol. A system for performing a recognition search based on a plurality of touch operations of a terminal interface, wherein the system comprises: a mask generator, configured to generate a mask covering the display area of the terminal interface in the current interface display area of the terminal when the user triggers the interface touch search request; The touch receiver is adapted to receive a plurality of touch operations of the user on the mask, and select an interface area to be searched on the mask according to a degree of coincidence between the multiple touch operations; The recognition searcher is adapted to identify an element included in the interface area to be searched, and perform a search according to the identified element. The system of claim 12, wherein the touch receiver is further adapted to detect a plurality of touch trajectories generated based on a plurality of touch operations of the user on the mask, in the mask A plurality of to-be-searched interface regions respectively corresponding to the plurality of touch trajectories are displayed, and the display manner of the to-be-searched interface regions is different from other regions on the mask; if two interface regions are to be searched If the coincidence degree is not less than the coincidence threshold, the two to-be-searched interface areas are merged into one to-be-searched interface area, otherwise the two to-be-searched interface areas are regarded as independent two to-be-searched interface areas. The system of claim 12 or 13, wherein the identification searcher is further adapted to select, on the mask, a minimum rectangular frame in which each interface region to be searched is located, and identify the smallest rectangular frame element. The system according to any one of claims 12 to 14, wherein the mask generator is further adapted to screen the current interface display area of the terminal when the user triggers the interface touch search request to obtain the The current interface of the terminal displays a screenshot of the area, and the screenshot is used as a mask covering the display area of the terminal interface. The system of any of claims 12-15, wherein the mask generator is further adapted to use the screenshot as a mask in a translucent state covering the display interface of the terminal interface. The system according to any one of claims 12-16, wherein the display manner of the interface area to be searched is displayed in a highlighted or fully transparent form; and / or, Other areas on the mask are displayed in grayscale, blurred or translucent form. The system of any of claims 12-17, wherein the identification searcher is further adapted to remove the mask and push the element to a search box located in a display area of the terminal interface ; When the user clicks on the first icon disposed in the display area of the terminal interface, the identification searcher is further adapted to perform a search according to the element. The system according to any one of claims 12 to 18, wherein when the user clicks on the second icon provided in the mask, the recognition searcher is further adapted to trigger execution of the search based on the identified information The information related to the web page. The system of any of claims 12-19, wherein the system further comprises Also includes: A search presenter adapted to display search result items related to the element in the terminal interface display area. The system according to any one of claims 12 to 20, wherein the search presenter is further adapted to display and display the combination of an application card, a text entry, and a graphic in the terminal interface display area. Element-related search result items. The system of any of claims 12-21, wherein the element comprises at least one of a text, a picture, and a symbol. A computer program comprising computer readable code, when the computer readable code is run on a computing device, causing the computing device to perform the terminal based interface according to any one of claims 1-11 multiple times Touch operation to identify the search method. A computer readable medium storing the computer program of claim 23.

Images