WO2014190951A1 - System and method for mapping occluded area - Google Patents

Abstract

A system and method for mapping an occluded area. The invention solves the problem of finger occlusion of content displayed on a touch screen or of content depicted on a touch pad, as well as occlusion of the point at which the finger contacts the touch screen, when a user is manipulating a touch pad, touch screen, or other such device. The invention also solves the problem of some system-level virtual buttons occluding content of a window under said buttons. The invention mainly makes use of such means as creating mapping areas and mapping the contact point of a finger. Through the use of mapping areas to dynamically map and display content in an area covered by a finger or by a virtual button, the invention solves the occlusion problem currently present in the art.

Description

System and method for mapping occluded areas Technical field

The invention belongs to a human-computer interaction control system and method, and is mainly used for controlling a handheld computer, such as a smart phone, a tablet computer, a smart portable tester, etc., and can also be used for remote control of general desktop computers, set-top boxes, smart televisions and the like. The device and its pointer are displayed.

Background technique

The existing touch screen technology has defects. When it is applied to a mobile phone, this problem is called a big finger problem, that is, the finger is much larger than the mouse pointer, and the target that is desired to be clicked is blocked. When the target is particularly small and dense, the special It is easy to make mistakes. For this reason, the UI interface designer has to set the buttons and menu fonts of each application to be relatively large, which makes the functions of the mobile phone application unable to be more and more dense. In addition, multiple news links on the webpage are difficult to do, and this problem is more prominent.

technical problem

In the manipulation method of controlling the handheld computer, there is such a control method, which uses the gyro in the handheld computer to monitor the rotation posture of the operator on the handheld computer, thereby controlling the movement of the mouse pointer in the screen, and the system is divided in the screen. An area used to display a set of virtual mouse buttons that function as traditional mouse buttons. This type of control is very promising in handheld computer devices such as mobile phones, PADs, and handheld smart testers. For this type of control, the patent application filed in 2012 is for a handheld computer device. A detailed description of the control system and method. But there are some drawbacks in this way, that is,

1. The virtual button lacks tactile feedback. When we press the mouse or the remote control button, we can not only feel the vibration of the button being pressed, but also feel the feedback position of the button and the finger, and so on. Feedback information is used to adjust the position of the finger on the mouse, the remote control, and to control the finger to switch between multiple buttons. The acquisition of these feedback information is done without visual information but completely through the sense of touch. However, when the virtual button on the touch screen is manipulated, the feedback information cannot be obtained by the touch, so that if the operator does not often observe the position of the finger on the virtual button, it is difficult to ensure that the finger is in the correct position. operating. It should be noted that, at this point, the virtual mouse system based on dynamic sensing has obvious disadvantages compared with the traditional touch screen manipulation method, because the pointer, the finger and the button of the traditional touch screen are trinity, and based on dynamic sensing. In the virtual mouse system, the pointer and the button are separated, which requires the observer's eye's observation point to constantly move between the pointer and the virtual button, which causes the operator to be significantly uncomfortable.

2. The finger blocks the virtual mouse button. As described above, the operator needs to constantly observe the relative position of the finger and the virtual mouse button when operating the virtual mouse button, so that the correct operation can be performed, but the finger itself obscures the button when the system When there is only one virtual mouse button, the operator can roughly guess the position of the button by referring to the object around the button, but when the system has multiple virtual mouse buttons and closely arranged together, the operation becomes very difficult. In fact, there are similar difficulties in the operation of the traditional touch screen. There are two solutions. One is to distribute the screen objects that can be triggered as sparsely as possible, and the trigger object is as large as possible. The second is to touch the finger. Doing an extended cross line moves with the movement of the finger, and always keeps the intersection of the cross line coincides with the touch point of the finger, so that the operator can guess the finger by extending the part of the finger occlusion area by the cross line. The location of the contact point. However, the two methods of the prior art are not ideal for solving the precise operation of a plurality of closely arranged virtual mouse buttons, because multiple buttons may be blocked by fingers, especially the adjacent boundary regions of their buttons are almost certain. It will be blocked by the finger, and it is difficult for the operator to distinguish which button the finger's contact point is on.

3. The virtual button window itself obscures a part of the display content, and opens a virtual mouse button area in the screen of the device such as a mobile phone or a tablet computer, which inevitably takes up a certain screen space, which requires other programs to coordinate with it, avoiding this An area displays its content, but it is obviously unlikely that many application developers will follow a certain rule to modify their programs. Considering that the tablet often changes the holding direction, sometimes horizontally and sometimes vertically, the position of the virtual mouse button needs to be changed, and the solution to this problem becomes more complicated.

Technical solution

Summary of the invention:

In the prior art, there are many applications of touch panels and touch screens, but their applications are subject to their own technical characteristics, and there are still some defects, mainly including the following aspects,

1. When the finger operates the touchpad and the touch screen, the finger itself will hinder the operator's line of sight. If the touch screen is used, the finger will block the operator's line of sight, making it impossible for the operator to see the contact of the finger with the touch screen. The position and its relative position to the object displayed on the display. If you are using a touchpad, I have imagined such a product. On the remote control, you can use the touchpad to draw several virtual buttons to replace the normal buttons. However, there is no such thing in the prior art. The root cause of the application is the problem of the finger obscuring the operator's line of sight.

2. In some applications, such as mobile phone systems that use dynamic sensing, the system level needs to create a virtual button on the touch screen, and this virtual button needs to remain at the top of the display content, such a virtual button will block the original screen Some other content is displayed, and manipulated.

The present invention is mainly directed to the above problems, and has been innovated to solve the above problems by creating a mapping window or other type of mapping area in the display screen. The manner and type of mapping are mainly divided into the following two types, but not limited to these two methods.

1. Create a mapping area to map the virtual buttons that are blocked by the finger, or other types of objects that can be triggered.

2. In the above mapping area, a mapped contact point is displayed for mapping the contact point between the display finger and the virtual button on the touch screen.

3. Create a mapping area to map the objects that can be triggered by the virtual button, or other content.

In terms of the specific form of the product and the placement of the virtual buttons, the following two methods are mainly divided, but not limited to these two methods.

1. The virtual button is displayed on the touch display screen of the computer. This method is mainly applied to mobile phones, tablets or other devices with touch screens. This form of product may require the use of the above three mapping methods.

2. The virtual button is in the non-display area on the touch screen of the mobile phone. The current smart phone uses this area as a function key such as a home button, and can be used as a virtual button after adopting the method of the present invention.

3. The virtual button is placed on the touchpad of the smart phone, that is, a small touchpad is placed on the front or side of the smart phone, and the virtual button is dedicated. Of course, prior art mobile phones do not have such a design, but it is entirely possible to adopt this method after using the present invention.

4. The virtual button is on the remote control's touchpad, on the mouse's touchpad or on the external device such as the notebook's own touchpad. This product form may only need to use the first and second mapping methods described above. Of course, if a touch panel with a display function is used on a device such as a remote controller, it may be required. At the same time, the above three mapping methods are used.

The specific implementation system of the present invention and its implementation steps are described below

A system and method for mapping an occluded area, including a touch pad or a touch screen, a dynamic sensing system, a manipulation system, a virtual button module, and a virtual button mapping module,

1. The state sensing system monitors the dynamics of the mobile phone, the remote controller, and the like, and delivers the dynamic data to the control system, and the control system controls the mouse pointer to move on the display screen according to the dynamic data. The dynamic sensing system mentioned here can be a gyro, a gravity acceleration sensor, an electronic compass or the like in a general sense. The laser coherent light sensor and the LED incoherent light sensor in the existing mouse can be used as sensors for monitoring dynamics.

2. The virtual button module is configured to open at least one virtual button area on the touchpad or the touch screen, and establish at least one virtual button in the area for accepting operator touch manipulation and manipulating the operator The instructions are sent to the control system. In applications such as smartphones, you may only need to open a virtual button area, and it is enough to place 1 or 3 virtual buttons in this area, but in applications with larger screens such as tablets, you can have multiple Virtual button areas, and let these virtual button areas be flexible. The touchpad, or the touch screen and the virtual button area opened thereon may be integrated with the main display screen of the computer, or may be two independent components, which are mainly divided into the following cases. , but not limited to the following situations,

1) In a portable device such as a smart phone or a tablet computer, the touch panel or the touch screen is the main display screen of the computer, and the virtual button module opens up in the main display screen and occupies a small place as a virtual button area. One or more virtual buttons are displayed in this area; of course, in applications such as tablets, the display screen is large, and multiple virtual button areas can be opened in different places.

2) In the prior art smart phone, the touch panel is larger than the display screen, and the portion of the simple touch panel of the larger part is generally referred to as the non-display area of the touch screen, and is generally used as the Home button and returned. The use of function keys such as keys can be used in the present invention as a virtual button in the present invention, but it is necessary to place it above the screen instead of below, which is convenient for the user to operate.

3) A small touchpad can be placed in the area outside the front screen of the phone or tablet, or on the side, as a virtual button. Of course, it is also possible to combine the above two points, and place a plurality of adjacent virtual buttons at the junction of different types of touch trigger regions, and assign different functions to them, and use them as multiple function keys.

4) In a device such as a desktop computer, a smart TV, or an IPTV, the touch panel is located on a remote controller, and on the simple touch panel without a display function, multiple layers can be drawn by etching, printing, or the like. Buttons, and use software to make the function of multiple virtual buttons;

5) In a device such as a desktop computer or a smart TV, the touch screen can be located on the remote controller. On the remote controller of the touch screen with display function, multiple buttons can be displayed according to different applications, and the software can be used together. The function of multiple virtual buttons is equivalent to the combination of the first two methods. In practice, there is also an application method in which the smart phone is used as a remote controller;

3. The virtual button mapping module is configured to open at least one mapping display area on the touch display screen, and map and display the shape and position of the virtual button on the touch panel or the touch screen. The mapping display area can be fixed, or can be moved according to actual conditions, and can even be dynamically moved together with the mouse pointer. From a programming point of view, the mapping area and the mouse pointer can even be combined into one form. Corresponding to the above step 2, the mapping display area described herein may be integrated with the main display screen of the computer, or may be two independent components, which are mainly divided into the following cases, but are not limited to the following Several situations,

1) In a portable device such as a smartphone or a tablet, the mapping display area and the virtual button area are both on the main display screen.

2) In a device such as a desktop computer or a smart TV, the mapping display area is on the main display screen of the computer or the television, and the virtual button is on the remote controller;

3) In a device such as a desktop computer or a smart TV, the remote controller of the device has a touch screen function, and the mapping display area is on the touch screen of the remote controller. Of course, the above manner can also be combined, and a mapping display is established on the main display screen of the computer or the television. region;

4. The virtual button mapping module is further configured to use a mapping manner in the mapping display area to express a touch panel, or a position of a touch point detected by the touch screen in the virtual button area, and a virtual button thereof Positional relationship between. Among the expression forms described, the most straightforward expression is to display a mapped contact point in the mapping display area, and the contact point mapping of the mapping is displayed in the virtual button area, or the touch panel detects The location of the contact point. Of course, if the control system supports multi-point operation on the virtual button, the mapped display area should also be able to display multiple mapped contact points.

5. The occluded area mapping module is configured to open a mapping display area on the touch display screen for mapping and displaying the content blocked by the virtual button area or the operator's operation tool and the limb; the step and The sixth step below is only useful when the virtual button is built on the touch display screen with the display function. If the virtual button is built on a simple touch panel that does not have the display function, since there is no occlusion original There are problems with the display content, so these steps and functions are not needed;

6. When the operator moves the pointer to the occluded area, that is, the area enclosed by the virtual button area, or the operator's finger, palm, or prosthesis, or other operating tool, the The occluded area mapping module is further configured to display a mapped mouse pointer within the mapped display area described above. The positional relationship between the mapped mouse pointer and other content displayed in the mapping area reflects the positional relationship between the actual mouse pointer and the occluded content in the area blocked by the virtual button area or the operator's finger. This step is only useful when the virtual button is built on the touch display screen with display function;

7. After the operator performs a touch control with a finger to trigger the virtual button established by the virtual button module, the control system triggers the manipulation object pointed by the mouse pointer, or the occluded manipulation object, or triggers the virtual mouse by the manipulation system. The pointer is the manipulation object pointed to in the mapping area of the occlusion area of the virtual button. Of course, in the actual product, multiple buttons can be placed in the virtual button area, and some buttons may not be used to trigger the object pointed by the pointer, but no matter where the mouse pointer points, the button is directly triggered to trigger a certain Specific functions, such as the Home button and the Back button function.

The meaning and content of the technical terms in the present invention are further explained as follows:

Virtual button

In the present invention, it is necessary to use a touch panel or an area on the touch screen to simulate functions of components such as a mouse button, a scroll wheel, etc. This functional area is called a virtual mouse button, but based on the characteristics of the touch panel or the touch screen, some can be made. The function that the traditional mouse button does not have, so it can be simply referred to as a virtual button.

2. Virtual button area

In the present invention, it is necessary to establish an area in the touch panel or the touch screen. In this area, several virtual buttons are placed in a centralized manner, and of course, some areas may be free, which are not used as any function buttons, as shown in FIG. The entire area 10 is a virtual button area, and the area 100 is the area where the finger is placed when the finger is idle, so that the finger does not have to be left floating when it is idle.

3. Mapping

In the present invention, it is necessary to use one display area to map the position and state of each element in the area of the operator or the area blocked by the virtual button, and the mapping refers to the relationship between the elements of the two, so that The operator can see and understand the situation in the occluded area, and the mapping manner can be a mapping completely imitating the original style, for example, using lines to map lines, and using a moving point position to map another moving point. The position of the line, color, block, size and other elements in the mapping area in the present invention are not necessarily identical to the mapped area, and are not in the general sense of mapping, or projection; or even completely The idea of separating from the image and the phenotype mapping, using an abstract, ideographic mapping method to express the positional relationship between the finger contact point and each virtual button, for example, mapping or expressing a point with the color or shape of the line or color block. s position.

4. Virtual button mapping area

In the present invention, when the operator operates the touch panel or the touch screen, the finger needs to be placed on the upper operation, the contact point between the finger and the touch screen or the touch panel is small, and the finger itself obscures the operator's line of sight, so that the operator It is difficult to see the exact location of the contact point. Therefore, in the present invention, it is necessary to open a window or area in the display screen to map the position of the touch point, and also to display one or several virtual buttons, so that the operator can determine the displayed content by observing the display content of the mapping area. The position of the contact point blocked by the finger and the relative positional relationship between the contact point and the button.

5. Area blocked by virtual buttons

In the present invention, it is sometimes necessary to open a virtual button area or window in the touch screen, in which the original content of the screen may be blocked, which we call the area blocked by the virtual button.

6. Area blocked by fingers

In the present invention, when the operator operates the touch panel or the touch screen, the operator's finger, palm, or prosthesis, or other operating tools may block the content of the button or the like depicted on the touch panel, or block the touch screen. The displayed buttons and the like, even if the virtual button displayed on the touch screen can be made transparent, the human finger cannot be made transparent, and the finger will block the lower window or the desktop. The content displayed, we call this area the area that is blocked by the finger.

7. Mapped area of the occluded area

In the present invention, in order to allow an operator to perform operations such as clicking on an object in an area blocked by a virtual button or an operator's finger, palm, or prosthesis, or other operation tool, it is necessary to open up at another location. An area maps the contents and objects originally displayed in this area to facilitate manipulation of these objects. This mapping area is called the mapping area of the occluded area.

8. The area occluded by the mapped area

In the above concept, in order to display the area blocked by the finger, a virtual button mapping area is established; in order to display the area occluded by the virtual button, a mapping area occluded by the virtual button is established. However, the two mapping areas each occupy and block an area and the content originally displayed in the screen, and such areas are collectively referred to as an area blocked by the mapped area.

9. Mapped touch points

In the present invention, during the operation of the touch panel or the touch screen, a finger or other operated limbs and tools may block touch points and buttons, so that the operator's naked eye cannot accurately see the specific In order to solve this problem, the present invention adopts a method of opening a mapped display area, as shown in FIG. 4, wherein 212 of the 20 areas is a mapped contact. The point, which maps the actual contact point 112, allows the operator to see the motion state of the contact point and its positional relationship with the button outside of the area that is blocked by the finger.

10. Mapped mouse pointer

In the present invention, in order to enable an operator to perform an operation such as clicking on an object in an area occluded by the virtual button, a mapping area occluded by the virtual button is established, in which not only the occlusion area but also the occlusion area is displayed. The content is also mapped to display a mouse pointer, as shown by 313 in the 30 area of Figure 5, so that the operator can clearly see the relative positional relationship between the pointer and the object, in order to accurately trigger the object. Such a pointer is called a mapped mouse pointer.

11. Touchpad

In the present invention, the function and application range of the touchpad commonly used in notebook computers are expanded and innovated, and can be used not only in general notebook computers, but also on remote controllers, and even on the mouse instead of the mouse. Button. Because it is used in the touch screen on mobile phones, it is technically a product technology that combines a transparent touch panel with a liquid crystal display. Of course, there are separate manufacturing methods in the production process, and then the combination method and the one-time manufacturing synthesis method. However, its essence is a combination of the two technologies; there is also such a combination of technical methods in the rear projection display of computers and even electronic whiteboards. It should be particularly noted that in the prior art smart phones, the touch panel is larger than the display screen, and the large portions of the simple touch panels are used as the Home button and the return button, which can be utilized in the present invention. A similar area is used as a virtual button in the present invention, but it needs to be placed above the screen instead of below, which is convenient for the user to operate. In the present invention, the manner of implementation with the touchpad can be implemented by a touch screen, so the touchpad mentioned in the present invention can be either an opaque touch panel in the conventional sense or a touch screen. Transparent touchpad.

12. Touch screen

In the current technology, the touch screen technology has developed rapidly, and the ultrasonic touch screen, the resistive screen, the capacitive screen, and the like are constantly being updated. The present invention can be applied not only to the touch screen of the prior art, but also to the newly emerging and existing touch screens in the future. In a technique with similar external functions, for example, when operating on a virtual screen, although there is no substantial touch, the function of the touch screen can also be realized by a virtual touch on the action, such as a virtual screen based on glasses launched by Google. Technology can also implement touch screen-like functions. The invention can also be applied to these technologies. Therefore, the touch screen of the present invention also includes a virtual screen.

13. Computer touch controller,

In the present invention, the trigger function in the touch panel or the touch screen can be used instead of the digital television remote controller, the dynamic sensing remote controller used on the smart TV and the IPTV, the traditional mouse, the shaking mouse, the aerial wireless mouse, and the like. The button function on the top, and these controllers that are independent of the main display of the computer and send the control instructions to the computer do not have a unified name in Chinese, and such controllers are constantly innovating both in name and function. For example, someone recently loaded software on a mobile phone and used the touch screen of the mobile phone to complete the function of the IPTV remote control. This application has not yet been fixedly called. I think it is appropriate to refer to such a device for controlling a computer as a computer controller. In the present invention, touch control is required, so this new device is called a computer touch controller.

Beneficial effect

1. Solved the problem that the finger operates the touch screen and the finger blocks the content displayed on the touch screen.

2. When solving the operation of the touchpad separated from the screen, it is necessary to constantly keep the eye moving between the display screen and the touchpad;

3. Alleviate the complicated operation caused by the mutual occlusion of multiple application windows on the screen.

DRAWINGS

Figure 1 shows the virtual button 10, the virtual button mapping area 20, and the mapping area 30 of the virtual button occlusion area in a smartphone screen. Of course, in the actual system, their size, shape, and position can be flexible. Adjustment, and mapping areas 20 and 30 can be combined to display in one area. In order to be more clearly shown, the latter figure no longer displays the screen of the entire mobile phone or the handheld computer, but directly displays the virtual button area and each mapping area.

Figure 2 shows the division of the virtual button area, and the 10 area is the entire virtual button area, where 100 is the area where the thumb is idle, and no trigger action is performed when the finger is placed in this area; 101 102 is a virtual button. When the finger moves in this area, the control system will trigger the object pointed to by the mouse pointer. Of course, in practice, there can be multiple virtual mouse buttons that can be triggered.

Figure 3 shows that when the thumb of the operation is placed in the virtual button area, it blocks part of the area, and the other parts are the same as those described in Fig. 2. The area of the occlusion is 111, and the contact area of the thumb and the touch screen is 112. 112 is always under the occlusion of 111. In order to solve this problem, a cross wire extending at a contact point 112 is outwardly extended, and dynamically moves following the movement of the contact point, so that the operator can estimate the position of the contact, and the present invention does not Using this technique, the drawings are drawn only to illustrate the shortcomings of the prior art. It should be noted that in the figure, the area blocked by the finger is included in the virtual button area, but in actual operation, it may be out of the range.

FIG. 4 shows that in the present invention, a mapping area is established beside the virtual button area 10, and two virtual buttons on the 10 areas and the contact points of the finger with the touch screen are displayed in the mapping area 20 by means of mapping display. Wherein, 201 maps 101, 202 maps 102, 200 maps 100, and in particular, the contact points 112 are dynamically moved with the operator's finger, so 212 is also moved and dynamically mapped 112, so that the operator can observe, Predict the relative position between each virtual button and the contact point.

As shown in FIG. 5, in the present invention, a mapping area 30 is formed below the virtual button area 10, which is called a mapping area of the occlusion area by the virtual button, for mapping and displaying the area blocked by the virtual button area or the finger. The window below it or the content displayed on the desktop. When the mouse pointer 113 enters the 10 zone or the 111 zone, a mapped mouse pointer is also displayed at the corresponding position of the mapping zone.

FIG. 6 shows a technical solution in which the contents and lines of the virtual button mapping area are simplified and moved together with the mouse pointer in the present invention. Of course, the mapping area of the occlusion area of the virtual button can also be moved together with the mouse pointer binding, and since it only needs to be displayed when the pointer enters the area occluded by the virtual button, it can also be considered to share the same with the virtual button mapping area. Areas, interlaced, but the content is uncertain, so it cannot be shown in the figure.

Embodiments of the invention

Example 1

We first set up in an application environment such as a large-screen smartphone. The control system of the smartphone uses the gyro in the mobile phone to monitor the operator's rotation posture of the handheld computer, thereby controlling the movement of the mouse pointer in the screen, that is, the operator. When the mobile phone is rotated left and right, the mouse pointer moves left and right; when the operator rotates the mobile phone up and down, the mouse pointer moves up and down; in addition, as shown in the area 10 of FIG. 1, the system divides an area in the mobile phone screen to display a set of virtual mouse buttons. , the function of the traditional mouse button, when the mouse pointer points to an icon, by pressing or toggle the virtual mouse button, thereby triggering the object pointed by the mouse pointer. As shown in FIG. 2, the 10 area is the entire virtual mouse area, where 100 is the area where the thumb is idle, and no action is triggered when the finger is placed in this area; 101, 102 are two The virtual mouse button, when the finger moves in this area, the control system will trigger the object pointed to by the mouse pointer.

Considering the technical characteristics of the touch display screen, in the virtual mouse button area, not only can the function of pressing and clicking by the traditional mouse be realized, but also many other functions can be realized, for example, the analog two-way toggle button can be realized by sliding, and even the image is used. It is a complicated gesture to draw a circle to generate a special trigger event, so it is not suitable to call it a virtual mouse button. It can be called a virtual button.

Of course, there can be multiple virtual buttons that can be triggered in the actual application. In the actual operation of such a system, we found that the finger obscured the virtual button, as shown in Fig. 3, the thumb occlusion area 111 is large, and the area 112 where the thumb actually touches the screen is small, and the coverage of the finger occlusion area is Down, and this small contact area is really used to manipulate the virtual button. Because the operator is blocked by the thumb, it is difficult to determine which specific part of the finger occlusion area is in the touch area, so as to accurately trigger The virtual buttons 101 or 102, so often cause misoperation. If the cross-hair extension method of the prior art is adopted, as shown in FIG. 3, the control system displays a cross line with the contact point of the finger as an intersection point, so that the operator can extend the finger occlusion area according to the extension of the cross line. Partly, the specific location of the contact point is inferred, but the shape and boundary of the two virtual buttons are also blocked by the thumb, so the operator can only operate by remembering and imagining the virtual button, which is still very inaccurate.

The solution of the present invention is such that, as shown in FIG. 4, a mapping area of a virtual button area is further divided next to the virtual button area or other places not covered by the finger, and not only the virtual button is mapped in the mapping area. The area also maps the contact area of the display finger. As shown in FIG. 4, the entire 20 area maps 10 areas of the virtual button, wherein 200 map area 100, 201 maps area 101, 202 map area 102, and most importantly, area 212 dynamic The area 112 where the thumb is in contact with the screen is mapped so that the operator can know the precise positional relationship between the contact point of the finger on the screen and the virtual button by observing the state of the mapping area, so that the operation can be accurately performed.

Of course, there is such a mapping area on the screen, which will definitely conflict with the original displayed object. These conflicts can be resolved from the perspective of software processing and display processing. First, the conflict is displayed, in order to minimize the mapping area. The influence of the display of the occluded area can be designed as a line as simple as possible, except that the area other than the mapped area of the line and the contact point is uniformly set to a transparent attribute, and the line and the contact point itself can also be Set to a semi-transparent property so that the operator can see other objects occluded through the mapped area. Secondly, it is a problem that triggers conflicts. In a traditional multi-window, multi-tasking operating system, although there is also a translucent window application, the operating system always triggers the object pointed by the mouse pointer in the uppermost window, in the present invention. However, the operating system is required to ignore the uppermost mapping window and trigger the object displayed in the window below it. In the existing system architecture, to achieve this, although it is somewhat difficult, it can still be achieved.

In order to solve the problem of display and click operation of the area blocked by the virtual button mapping area, the mapping area can also be dynamically moved. For example, the mapping area is originally on the left side of the virtual button, as shown in FIG. 4, but We can make such an effect in the program, that is, when the mouse pointer enters the mapping area 20, the mapping area automatically jumps from the left side of the virtual button to the lower side thereof, and when the mouse pointer leaves the position, the mapping area Then jump back automatically. A further improvement is that the mapping area can be made completely dynamic and moves along with the mouse pointer, so that the mapping area does not hinder the operation of the operation object pointed by the mouse pointer. For details, refer to the details in Embodiment 3. description.

Derived technological advantage

By adopting the invention, the operator's finger can perform more precise operations on a plurality of adjacent virtual buttons, and precisely because the more precise operation can be performed, the operation mode can break through the existing operation mode, thereby deriving a The new operation mode of the series, for example, the operation of the touch screen in the prior art is mainly a click and slide operation, in which the operator must frequently press and raise the finger during the operation, because the operator needs to frequently look under the finger The position of the object such as an icon, a virtual button, etc., after the method of the present invention, the operator's finger can always be placed on the touch screen without leaving, and is triggered by entering the position occupied by the virtual button, as shown in FIG. 3 when the finger is the same as the touch screen. When the contact point 112 is in the 111 area, no trigger is performed. When the contact point moves from the 111 area to the 101 area, the trigger is equivalent to the function of the left mouse button being pressed, and when the contact point 112 moves from the 101 area back to the 111 area, The trigger is equivalent to the function that the left mouse button is released. The same principle applies to the operation of the right mouse button represented by the 102 area. This eliminates the need for the operator to continuously press the finger up, and only needs to reciprocally move the finger between multiple areas on the screen to generate a variety of different triggering operations.

The above method can be further extended to use the finger to operate the general icons and buttons on the screen of the mobile phone or the tablet, but the manner of the sliding trigger may conflict with the existing screen-drawing function, that is, in the existing system, press The lower finger, the sliding finger, and the raised finger represent the screen flipping action, and the screen swipe function is required to use the above functions in the present invention to trigger a general screen object. Since this method has not been found to trigger a screen object such as a general icon and has a great advantage compared with the conventional click triggering method, it is not described as a separate embodiment.

Specific implementation steps

A system and method for mapping an occluded area, comprising a dynamic sensing system, a control system, a virtual button module, and a virtual button mapping module, and implementing the following steps,

1. The dynamic sensing system monitors the dynamics of the mobile phone or the tablet and delivers the dynamic data to the control system. The control system controls the mouse pointer to move on the display screen according to the dynamic data.

2. The virtual button module is configured to open at least one area on the touch screen, and establish at least one virtual button and a form of displaying a button in the area for accepting an operator touch manipulation and transmitting the manipulation instruction of the controller to Control the system. Of course, this virtual button can also be partially or completely placed in the non-display area of the touch screen. In the current mobile phone, the non-display area is used as a function key for placing a Home button, a return button, and the like. The touch control mentioned here can be in various forms, which can simulate the click and long press of the real mouse button; it can also be a sliding operation to simulate the wheel, the two-way toggle button; or even the circle and the beat Hooks and other more complex manipulation postures, and when doing these postures, the fingers may not even be lifted and lowered as frequently as in the conventional operation, but always perform various sliding actions on the screen. The above operations on the virtual button can be combined with the action of the pointer to derive more functions, for example, the traditional drag operation is achieved by combining the button operation and the pointer movement. Virtual buttons can have more features than actual buttons, so you can make more combinations of features in the future.

3. The virtual button mapping module is configured to open at least one mapping display area on the touch display screen, and map and display a form of the virtual button established by the virtual button module. The mapping area mentioned here may be the same size as the virtual button area, or may be enlarged or reduced; the display form of the mapping area may be completely consistent with the virtual button, or may be simplified, or even abstractly representing the virtual button. Morphology; the positional shape of the mapping area can be a fixed position, a dynamic position, or even a dynamic movement with the mouse pointer.

4. The virtual button mapping module is further configured to map the touch point of the display operator in real time in the mapping area displayed by the virtual button mapping module when the operator touches the virtual button displayed by the virtual button module. The touch point map displays the contact point of the finger blocked by the finger in the virtual button area with the touch screen, and its positional relationship with each virtual button. The external form of the touch point of the above mapping may be the original irregular elliptical form of the touch point, or may be an abstract origin, a ring, or even a small cross star.

5. After the operator triggers the button displayed by the virtual button module with the finger, the manipulation system triggers the manipulation object pointed by the mouse pointer in the display screen. Of course, in the actual product, multiple buttons can be placed in the virtual button area, and some buttons may not be used to trigger the object pointed by the pointer, but no matter where the mouse pointer points, the button is directly triggered to trigger a certain Specific functions, such as the Home button and the Back button function.

Mode for Invention

Example 2

There is such a control method in the manipulation method of controlling the handheld computer, and the gyro in the handheld computer is used to monitor the operator's operation posture such as the rotation and shaking of the handheld computer, thereby controlling the movement of the mouse pointer in the screen. As shown in Figure 2, the system divides an area on the screen to display a set of virtual buttons, which function as a traditional mouse button. Of course, in order to control the system at any time, this virtual button area needs to be displayed on the top layer, but in the mobile phone. A virtual button area is opened in the screen of a device such as a tablet computer, which inevitably takes up a certain amount of screen space and blocks part of the screen space, so that when the mouse pointer needs to point to an object below the area and triggers it, there is an obstacle. Of course, you can set the virtual button to be semi-transparent or even completely transparent, so that you can see the mouse pointer and object below, but in actual operation, the operator should put the finger on it. Still, the line of sight is blocked, as shown in FIG. 5, the occlusion area 111 of the thumb blocks the pointer 113. In order to solve this obstacle, the present invention devises a temporary window for real-time dynamic display. When the mouse pointer moves into the area 100 blocked by the virtual button, a mapping window can be temporarily popped near the virtual button area, as shown in FIG. 5. In the area 30, the screen displays the screen content occluded by the virtual button area; at the same time, a mapped mouse pointer 313 is displayed in the mapping window to dynamically map the actual mouse pointer and the occlusion area. The object, and the relative position between them, of course, after starting to map this actual mouse pointer, the actual mouse pointer can be displayed or not, but its actual coordinate position and the actual content of the occlusion are The interior of the handheld computer still exists. The mapping content displayed in the mapping window and the mapped mouse pointer are all for the operator to see and speculate the content occluded by the virtual button area, and can see and estimate the relative position of the mouse pointer and the object in the occlusion area. And can press the virtual button at the right time to trigger the object you want to manipulate.

Of course, to implement the present invention, it is also necessary to consider and solve the following problems.

1. The display area of the mapping area, the mapping area mentioned here is the mapping area of the occlusion area by the virtual button. If a long-displayed display area is opened to display the area blocked by the virtual button area, of course, this may cause another problem. It is this mapping display area that obscures another area, which becomes a looping problem. So there are several ways to solve it, but not limited to these methods.

1) The time-sharing display can be divided into multiple time segments by time, and the content of the original window of the screen and the content of the mapping area are displayed in turn, for example, the time is equally divided into 0.5 seconds for each segment, and two time segments, that is, 1 second, are used. Change the mapping area to a transparent state, that is, the content of the second layer in the display screen that is occluded under the mapping window. Then use a time segment, that is, 0.5 seconds to display the contents of the mapping window, and then use 1 second to change the mapping area to a transparent state, so that the operator can see the contents of both during the switching display. The advantage of this method is that the operator can see the contents of the occluded area without any additional operations. The disadvantage is that the reciprocating flashing display seems to be more laborious.

2) The pointer enters the area occluded by the virtual button and temporarily jumps out of the new window to display the content of the occluded area. The advantage of this method is that the display of other areas is less affected, but the disadvantage is that when the operator wants to see the occlusion by the virtual button When you are in the area, you need to move the mouse pointer to the occlusion area, so that the system will pop up a temporary window to display the contents of the occluded area.

3) Further improvements can be made to the previous method, that is, the virtual button window and the buttons in it are represented by simple lines, and most other areas of the non-line are set to be transparent, so that only the entire 10 areas are included. The 111 sub-area also has an occlusion of the thumb. In other parts of the area, the operator can still see the content of the screen displayed under the occlusion of the virtual button, and the mouse pointer, only when the pointer enters the occlusion area of the finger. A jump out of the mapping window area 30 is triggered. Since the 111 area moves with the finger of the person, the manipulation system needs to calculate according to the position of the contact point of the finger and the screen. As for how to estimate the size of the operator's finger and the size of the occlusion area, it can be artificially already setup.

4) The virtual button mapping area is overlapped and displayed in the same area. In Embodiment 1, it can be seen that the system itself has to open a virtual button mapping area 20, and the best implementation manner of this area is to use translucent display only simple lines. The way, therefore, this area can adopt the superposition mapping method, which is used as the virtual button mapping area to display the shape of the virtual button and the position of the finger touch point, and display the content of the area blocked by the virtual button, and can also become Translucent state to show what the third layer of this area should have displayed. This method is more complicated and difficult to understand. The explanation is further explained below. First of all, if the system does not have a virtual button and there is no virtual button mapping area, the screen displays its originally displayed content. This content may be an icon object on a general desktop or a text displayed by a text editor. For the content of the application, we refer to the content displayed in this layer as the content of the C layer. After the virtual button area is added, because this area is blocked by the operator's finger because of the operation, a virtual button mapping area needs to be created next to it so that the operator can see the contact point between the finger and the screen. The relative position between the virtual buttons, we call this layer A layer, because the content that the A layer needs to display can be represented by simple lines, so the A layer can be made into a translucent form, only the line part The other areas are transparent, so that we can see the information displayed by both the A and C layers. Now it is necessary to superimpose and display the content of the area occluded by the virtual button in this area, which is more complicated. It is necessary to insert a layer B layer between the A layer and the C layer, and use this layer to display the occlusion by the virtual button. For the content of the area, the display mode of this layer can refer to the above two methods. For example, the content of the B layer can be displayed normally, but the content of the B layer is displayed only when the mouse pointer enters the area blocked by the virtual button. Or at regular intervals, the content of the B layer is displayed for a while, and then the B layer becomes transparent, so that the content of the C layer is displayed on the screen. In this way, the operator can see the different contents of the three layers in this window or area.

5) The window area of the mapping area of the virtual button mapping area and the area occluded by the virtual button is made into a window form that can be dynamically moved. For example, the mapping area is originally on the left side of the virtual button, as shown in FIG. 5, but we This effect can be made in the program, that is, when the mouse pointer enters the mapping area 30, the mapping area automatically jumps from the bottom of the virtual button to the left side thereof, and when the mouse pointer leaves the position, the mapping area is automatically Jump back. A further improvement is that the mapping area can be made into a completely dynamic window, and the window moves along with the mouse pointer, so that the mapping area does not hinder the operation of the operation object pointed by the mouse pointer, and generally the operator's line of sight is It is moved with the pointer, and the manner of following the movement can make the operation of the operator more comfortable. For the specific embodiment, refer to the detailed description in Embodiment 3.

2. The display problem of the mapped mouse pointer, in the present invention, there may be two simultaneous mouse pointers on the screen, plus a mapping point of a finger contact point, three dynamic moving objects are so complicated, it is easier to let The confusing situation is that the mouse pointer just disappears from the position next to the virtual button, and a mapped mouse pointer is jumped from another place. This will make the new operator more confused, so the mapping point and mapping of the finger touch point are needed. The mouse pointer and the real mouse pointer are greatly different in color, form, or flashing form for the operator to recognize.

3. The mouse pointer and the mapped mouse pointer trigger the problem of the object. In the present invention, unlike the prior art, in the prior art, the mouse pointer only triggers an object located in the window at the top of the screen, even if the uppermost window is Transparent or translucent, it does not trigger objects in the window under its occlusion. In the present invention, there is no object in the virtual button area located in the uppermost layer that can be triggered by the mouse pointer, and only can be triggered by the finger. Object, so by modifying the existing operating system, it is still possible to have the mouse pointer trigger the object on the lower window or desktop. Of course, you can also use the content in the mapping window and the mapped mouse pointer to trigger. After jumping out of the mapping window, in the mapping window, the content of the lower window and the mapped mouse pointer are at the top, with the traditional The algorithm of the operating system can be implemented.

Specific implementation steps

A system and method for mapping an occluded area, including a dynamic sensing system, a control system, a virtual button module, and an occlusion area mapping module, and implemented according to the following steps,

1. The dynamic sensing system monitors the dynamics of a handheld computer such as a mobile phone or a tablet computer, and delivers the dynamic data to the control system, and the control system controls the mouse pointer to move on the display screen according to the dynamic data;

2. The virtual button module is configured to open at least one virtual button area on the touch display screen, and establish at least one virtual button in the area, display the shape of the button, accept operator touch manipulation, and send the controller's manipulation indication For the control system; the touch control mentioned here can be in various forms, which can simulate the real mouse button to click and long press; or it can be a sliding operation to simulate the wheel, the two-way toggle button; or even Draw circles, tick and other more complex manipulation postures, and when doing these postures, the fingers may not even be lifted and lowered as frequently as in the conventional operation, but always perform various sliding actions on the screen. The above operations on the virtual button can be combined with the action of the pointer to derive more functions, for example, the traditional drag operation is achieved by combining the button operation and the pointer movement. Virtual buttons can have more features than actual buttons, so you can make more combinations of features in the future.

3. The occlusion area mapping module is configured to open at least one mapping display area on the touch display screen for mapping content displayed by the virtual button area or the operator's operation tool and the limb; the mapping mentioned here The area can be the same size as the area blocked by the virtual button area, and can also be enlarged or reduced; the display form of the mapping area can be completely consistent with the content in the area blocked by the virtual button, and can also be simplified; the positional form of the mapping area You can use a fixed position, a dynamic position, or even a dynamic movement with the mouse pointer.

4. The occluded area mapping module is further configured to display a mapped mouse pointer within the mapped display area as described above when the operator moves the pointer to the occluded area. The positional relationship between the mouse pointer of the map and other contents displayed in the mapping area, and the positional relationship between the actual mouse pointer and the content blocked by the virtual button area or the operator's operation tool and the limb The mouse pointer mapped here can be exactly the same as the actual mouse pointer, and can also adopt different shapes and colors, and even display modes such as blinking to highlight the difference from the actual mouse pointer.

5. After the operator performs a touch control with a finger to trigger the button displayed by the virtual button module, the control system triggers the occluded manipulation object pointed by the mouse pointer, or the manipulation system triggers the manipulation of the virtual mouse pointer in the mapping area. Object.

BEST MODE FOR CARRYING OUT THE INVENTION

Example 3

According to the mapping principle in Embodiment 1, the present invention can be applied not only to mobile phones and tablet computers, but also to remote controllers of smart TV, IPTV, and digital TV set-top boxes. I have considered it a long time ago, why not use similar The principle of a touchpad or a touchscreen in a notebook computer, using a virtual button placed on the touchpad on the remote control to replace the traditional rubber button to control the TV or IPTV? My conclusion at the time was that the traditional buttons provided the operator with tactile-based feedback so that the operator could know the posture, position, and position of the finger with the touch of the finger while staring at the TV screen without looking at the remote control. Which button is currently being touched, and the touchpad cannot provide such feedback. It is estimated that many technicians have similar ideas, but they have never seen a remote control based on a touchpad or a touch screen. Of course, some enthusiasts have recently developed a software on a mobile phone that is displayed on a mobile phone with a touch screen. Buttons to control TV, IPTV, etc., but there has never been a formal manufacturer using this technology, and it is estimated that it is based on the lack of tactile feedback on its technology. This problem can be easily solved by the principle of the present invention.

There are two basic types of remote controls, one is a traditional remote control controlled entirely by buttons, and the other is a dynamic remote control with a built-in dynamic sensing system in the remote control to monitor the operator's movements and Convert to the movement of the pointer on the screen, then use the buttons on the remote to confirm the operation. However, the present invention can be used in both of the above remote controllers.

This embodiment describes a remote controller using a touch panel or a touch screen. From a technical point of view, the touch screen actually covers a transparent touch panel on the display screen, of course, if the remote controller is used like a mobile phone. A touch screen with display function, even with CPU and intelligent processing capabilities, can provide more functions, including the functions described in the previous embodiment 1 and embodiment 2, but these extra functions are not in the embodiment of the present invention. Within the scope of innovation. Therefore, the following description will not be repeated, directly representing the two with a touchpad.

First of all, the method of replacing the buttons on the remote control mouse and the dynamic remote control with the touchpad is mainly used for the smart TV, IPTV, or the remote controller of the computer. For the convenience of description, it is called computer and dynamic. remote control. The dynamic remote control system has a dynamic sensing system such as a gyroscope for monitoring the user's control posture, and converting the signal reported by the gyro into the movement of the mouse pointer on the computer screen in the remote controller or its supporting firmware. Of course, this conversion work can also be implemented with a computer software system. The buttons on the dynamic remote controller of the prior art are also traditional buttons, which are similar in form and function to the mouse, keyboard button, click to trigger the operation object such as the icon pointed by the mouse pointer, or directly Trigger a feature. In the present invention, one or several touch panels can be used instead of a plurality of function buttons, and several regions are respectively divided on the touch panel to represent several buttons. If used directly, the operator's experience is not even as good as the traditional one. The first drawback of the button is that the operator lacks tactile feedback. If the user does not know how much the finger is pressed on the button without looking at the remote control, if it is to be moved from one button to another, it cannot be based on the touch. The movement is completed, so that the viewpoint of the operator's eyes must frequently reciprocate in the computer screen, and the use experience is poor; secondly, as in the foregoing embodiment 1, when the buttons are relatively small and arranged in a dense manner, The finger itself will block the border of the button, causing misoperation. According to the idea described in the above embodiment 1, a button mapping area can be opened on the display screen of the computer or the television, and the shape of the button and the touch point of the finger on the touch panel are displayed in the mapping area. In this way, the operator can operate the computer without having to move the line of sight frequently between the screen and the remote controller, but can always operate on the screen. Of course, the size of the mapping area can be as large as the touchpad area on the dynamic remote controller, and can also be enlarged or reduced, and even only a part of the area on the touch panel can be mapped, and those areas that are not mapped can be used for other purposes. .

If you make further improvements, you can make this mapping area or window a dynamically moveable mode, and let it move with the mouse pointer, or even integrate it with the mouse pointer. The advantage of this is the operator's The line of sight can always follow the mouse pointer without moving the line of sight from time to time to that fixed mapping area. This method has obvious advantages when users operate large-screen smart TVs, because when operating a mobile phone or a tablet computer, the screen area is small, the operator's line of sight does not move much, and even the afterglow of the eyes can be seen. When it comes to the mapping area, when the user operates the large-screen smart TV, if the operator uses a fixed mapping area, it is obviously difficult, and the dynamic mapping area that follows the movement of the mouse pointer is obviously labor-saving.

On the basis of the above, further improvements can be made. If the mapping area or window is moved together with the mouse pointer and the general mapping method is adopted, the operator may feel that this large mapping area is swaying on the screen. Going, hindering the operator to watch, the simple and direct mapping method is to directly use the phenotype to display the original button layout and shape mapping. For example, Figure 2 shows a normal similar mapping area, but we can The lines of the mapping area shown in Figure 2 are simplified, omitting the borders around them, and the color of the button body. In addition to the lines, the rest of the mapping area is transparent, and only the remaining lines and the system mouse pointer Combine it and follow it to move into a shape as shown in Figure 6, where 601 is the pointer of the system, followed by a curve like two tails, which is actually the two buttons in the mapping area. The curve of the boundary, so that the operator can also push the relative position of the two points 501 and 502 by the contact point 212 displayed in the mapping area. In which the position of the finger on the touch pad to determine how to move a finger to trigger a button. As for the size of the mapping area, it can be consistent with the mapped area on the touch panel, or the size ratio of the mapping display can be changed by the user's setting, and even the horizontal and vertical display scale relationship can be changed by setting.

On the basis of the above, it is also possible to further improve the idea of completely separating the mapping of the phenotype of the image, and to express the positional relationship between the finger contact point and each virtual button by using an abstract and ideographic mapping manner, for example, The color map indicates the position between the contact point and each button. In FIG. 6, the mapping point 212 of the contact point may be absent, and the color of the lines 501 and 502 is used to represent the distance relationship between the contact point and the two buttons. When the distance is near, the line is red, and the distance is orange, green, and blue, so that the operator can estimate the positional relationship between the finger contact point and each virtual button by observing the colors of the two lines. If the virtual button is not in the 2D plane of a square on the touch screen, but a long strip on the side of the phone, ie a one-dimensional line segment, then abstract the button into two points on the line segment, then use abstraction The way to express the positional relationship between the finger and each button is easier. Of course, if there is only one virtual button, it is even simpler. You can use a color change of a color block to indicate its positional relationship. You can even use a number to indicate its positional relationship. You can even consider using a cartoon image and its screen. The body posture map displayed above indicates the direct positional relationship between the touch point and each button. Of course, if you use this ideographic mapping method to express, you can derive a lot of cool styles, and I won't go into details here.

In the simulation test, we found that the manner in which this mapping area follows the movement of the mouse pointer also produces an unexpectedly excellent effect. In the method of mapping the display area in the fixed area described in Embodiment 1, there is a disadvantage that The mapping area itself will block the content displayed on a part of the screen. Even if the virtual button is mapped in a semi-transparent display manner, there is a problem that the pointer can only trigger the object in the uppermost window by calling the underlying driver module in the prior art. It is impossible to trigger an object that is under the occlusion of the mapping window. It is necessary to modify the existing underlying drivers responsible for display, mouse driver, object triggering, etc. to solve this problem. However, after implementing the method in this case, the mapping area dynamically follows the pointer movement, and the mapping area will never obscure the object pointed by the pointer that the operator wants to trigger. Of course, there is a small problem with the manner in which such a mapping area dynamically follows the pointer movement as shown in FIG. 6. When the pointer moves to the leftmost side of the screen, of course, there is no problem, but when the pointer moves to the far right of the screen, The mapping area is already outside the screen display area, completely invisible. This problem is easier to solve. It is to monitor the mouse pointer or the horizontal coordinate of the edge of the mapping area by the program. When the pointer moves to the right of the screen. When the map display area has reached the far right of the screen, the map switch is moved to the left side of the pointer by program control, and the problem is solved. Of course, if the usual display area is on the left side of the pointer, the process is reversed, that is, when the pointer moves to the left side of the screen, the display area is switched to perform the switching movement. Of course, when the pointer moves to the upper or lower boundary of the screen, the same principle of switching can be used.

The scope of use of the method and system described in this embodiment can be further extended to a conventional mouse, using a touchpad instead of a mouse button, and a touchpad mapping area is bound to the mouse pointer in the computer screen. The most vulnerable part of a traditional mouse is its button. After using this method, you can replace the two buttons and scroll wheel of a traditional mouse with a touchpad. The advantage is that the number, function, shape, and position distribution of the mouse buttons can be set by software, which is much more flexible than the fixed style of the existing mouse hardware buttons. The form can even be changed according to the current application window. For example, if the current system is a CAD system, the number of virtual buttons can be changed from 2 to 3; if it is currently a read-only text window, the second is The function of the third button can be changed to the up and down page turning function keys; and the operator's finger can not only perform pressing and releasing operations, but also can perform various gestures such as swiping and circle drawing. Further, the touchpad of the mouse may be transparent and may have a display screen below it, that is, the button area thereof is a miniature touch screen, and the touch screen may also be curved; as can be seen from the above description, the present invention After the technology described in this embodiment is applied to a conventional mouse, the conventional mouse can be greatly improved in both style and function.

Specific implementation steps

A system and method for mapping an occluded area, comprising a computer touch controller, a dynamic sensing system, a control system, a virtual button module, and a virtual button mapping module,

1. The dynamic sensing system monitors the dynamics of the remote control and delivers the dynamic data to the control system. The control system controls the movement of the mouse pointer on the display screen according to the dynamic data. The dynamic sensing system mentioned here can be a gyro, a gravity acceleration sensor, an electronic compass or the like in a general sense. The laser coherent light sensor and the LED incoherent light sensor in the existing mouse can be used as sensors for monitoring dynamics.

2. The virtual button module is configured to open at least one area on the touchpad of the computer touch controller or the touch screen, and establish at least one virtual button in the area for accepting the operator's touch control and the controller's The steering instructions are sent to the control system. The virtual button module mentioned here can be located in the host as an intelligent processing module, or in a remote controller with intelligent processing capability; the touch control described here can be in various forms, and can be clicked and long by mimicking a real mouse button. Press the operation; it can also be a sliding operation to simulate the wheel, the two-way toggle trigger button; even a more complicated manipulation posture such as circle, tick, etc., and the fingers can not even be used as traditional operations when doing these postures. Frequently lifting up and down, but always doing a variety of different sliding actions on the touchpad. The above operations on the virtual buttons can be combined with the actions of the pointers to derive more functions, such as traditional drag operations, mouse gesture operations, etc., by combining button operations and pointer movements. Virtual buttons can have more features than actual buttons, so you can make more combinations of features in the future.

3. The virtual button mapping module is configured to open at least one mapping display area on the display screen, and map the shape and position of the virtual button on the touch panel of the remote controller or the touch screen. The mapping area mentioned here may be the same size as the virtual button, or may be enlarged or reduced; the display form of the mapping area may be completely consistent with the virtual button, or may be simplified, or even abstractly representing the form of the virtual button; The positional shape of the mapping area can adopt a fixed position, a dynamic position, or even a dynamic movement with the mouse pointer. From a programming point of view, the mapping area and the mouse pointer can be integrated into one. form.

4. The virtual button mapping module is further configured to use a mapping manner in the mapping display area to express a touch panel, or a position of a touch point detected by the touch screen in the virtual button area, and between the contact point and the virtual button. Positional relationship.

5. After the operator performs a touch control with a finger to trigger the button displayed by the virtual button module, the control system triggers the occluded manipulation object pointed by the mouse pointer, or the manipulation system triggers the manipulation of the virtual mouse pointer in the mapping area. Object. In the actual product, multiple buttons can be placed in the virtual button area, and some buttons can not be used to trigger the object pointed by the pointer, but no matter where the mouse pointer points, the button is triggered directly after triggering a program. Specific functions that are set, such as the Home button and the Back button function.

Industrial applicability

Sequence table free content

Claims (1)

1. A control system for mapping an occluded area, comprising a touch panel and a touch screen, comprising: a virtual button module and a virtual button mapping module; The virtual button module is configured to open at least one virtual button area on the touchpad or the touch screen, and establish at least one virtual button in the area for accepting operator touch manipulation and transmitting the operator's manipulation indication Give the control system; The virtual button mapping module is configured to open at least one mapping display area on the display screen, and map and display the shape of the virtual button on the touch panel or the touch screen; The virtual button mapping module is further configured to use a mapping manner in the mapping display area to express a touch panel or a position of a touch point detected by the touch screen in the virtual button area; 2. A method of mapping an occluded area, characterized in that the system adopts the following steps, 1) The virtual button module opens a virtual button area on the touchpad or the touch screen, and establishes at least one virtual button in the area for accepting the operator's touch manipulation and transmitting the operator's manipulation instruction to the manipulation system; 2) The virtual button mapping module opens a mapping display area on the display screen, and displays the shape of the virtual button on the touch panel or the touch screen. 3) The virtual button mapping module is further configured to express, by using a mapping manner in the mapping display area, a position of a touch pad or a touch point detected by the touch screen in the virtual button area; 3. A control system for mapping an occluded area, comprising a touch panel, a touch screen, and a dynamic sensing system, comprising: a virtual button module, an occlusion area mapping module, The virtual button module is configured to open an area on the touch display screen, display and establish at least one virtual button, accept operator touch manipulation, and send the controller's manipulation instruction to the manipulation system; The occluded area mapping module is configured to open at least one mapping display area on the touch display screen, configured to map and display content that is blocked by the virtual button area or the operator's operation tool and the limb; When the operator moves the pointer to the occluded area, the occluded area mapping module is further configured to display a mapped mouse pointer in the mapping display area; 4. A method of mapping an occluded area, characterized in that the system adopts the following steps, 1) The virtual button module opens an area on the display screen, displays the shape of the button, accepts the operator's touch control, and transmits the controller's manipulation instruction to the manipulation system; 2) The occluded area mapping module opens a mapping display area on the touch display screen, and displays the content blocked by the virtual button area or the operator's operation tool and the limb; 3) When the operator moves the pointer to the occluded area, the occluded area mapping module displays a mapped mouse pointer in the mapped display area; 4) 5. The control system according to claim 1, wherein the touch panel or the touch screen is located on a remote controller of a computer system; 6. The manipulation system according to claim 1, wherein the map display area is moved together with the mouse pointer or integrated with the mouse pointer. 7. The virtual button mapping module according to claim 1 expresses the position of the touch point detected by the touch panel or the touch screen in the virtual button area, wherein the expression is in the above-mentioned mapping display area. Displaying the mapped contact point, the mapped contact map showing the position of the touch pad detected by the touch panel or the touch screen in the virtual button area; 8. The method of mapping an occluded area according to claim 2, wherein the mapping display area is located in a remote controller of a computer system or a touch screen on a mouse; 9. The control system according to claim 3, wherein the touch screen is located on a remote controller of a computer system; 10. The method of mapping an occluded area according to claim 4, wherein the touch screen is located on a remote controller of a computer system;