KR20100039194A - Method for displaying graphic user interface according to user's touch pattern and apparatus having the same - Google Patents

Abstract

The present invention relates to a method of displaying a graphical user interface (GUI) according to a user contact pattern and an apparatus having the same. In particular, in the present invention, the GUI display method according to the operation hand is a touch sensing step of detecting a user contact to the device in a device having a tactile sensor, the contact to determine the contact pattern of the user according to the detected user contact And a display step of displaying a pattern determination step and a GUI corresponding to the determined contact pattern of the user. According to the present invention, a user can operate a device having a touch screen with one hand without covering the touch screen screen. In addition, by displaying the GUI according to the position of the finger holding the device, the user can conveniently input an operation command.

Description

TECHNICAL FOR DISPLAYING GRAPHIC USER INTERFACE ACCORDING TO USER'S TOUCH PATTERN AND APPARATUS HAVING THE SAME}

The present invention relates to a method of displaying a GUI and a device having the same, and more particularly, to a method of displaying a GUI according to a user contact pattern and a device having the same.

Recently, devices using touch screens are widely used in real life. The touch screen is a display designed to allow a user to directly touch the screen by using a finger or pen-like touch mechanism, and the device having the screen recognizes the touched portion and executes a corresponding command or moves the position of the cursor accordingly. Means the device. Since the touch screen does not require a space where a key input unit is formed, it is suitable for a small terminal and is widely used in mobile devices.

A user generally operates a device with a touch screen with one or both hands and executes the corresponding function. When a user operates a device having a touch screen with both hands, the device is fixed with one hand and a touch operation is performed with the other hand so that the device can be operated without covering the screen displayed on the touch screen. However, when operating the device with one hand, a problem occurs in that the screen displayed on the touch screen is covered by the operating finger.

10A illustrates a screen in which a menu icon is touched by a left thumb while a menu icon is displayed on a touch screen. When the user touches the music menu icon, the game menu icon, the screen menu icon, and the schedule menu icon are covered by a finger that operates. In addition, if a part of the thumb touches the icon of the hidden part, a function not intended by the user may be executed.

10B is a screen in which a scroll bar is operated by the thumb of the left hand while the Internet access screen is displayed on the touch screen. Since the scroll bar is generally displayed on the right side of the display screen, when the user manipulates the scroll bar with his left thumb, the user may hide the internet access screen by his or her thumb. Unintended touch can be made.

In the case of a device for controlling an application using a tactile sensor, the application may be controlled by simply inputting a contact to a specific part of the device without using a touch screen or a keypad. In this case, the device may display a GUI for guiding a contact requiring input on a display screen. However, when the device displays the GUI in a predetermined form without considering the contact pattern of the user, the position where the user's finger touches and the position where the GUI is displayed may not match. Since the size of the hands and fingers of each user and the type of holding the device is different for each user, displaying a GUI suitable for all users is limited. If the position where the user's finger touches and the position where the GUI is displayed do not match, confusion may occur when the user controls the application.

Therefore, in a device having a tactile sensor, there is a need for a method of displaying a GUI according to a user contact pattern.

An object of the present invention is to provide a method for displaying a GUI suitable for a hand to operate in operating a device equipped with a touch screen and a tactile sensor.

Another object of the present invention is to provide a method for displaying a GUI according to a position of a finger of a user in contact with the device in a device provided with a tactile sensor.

Another object of the present invention is to provide an apparatus using the method.

In the method for displaying a GUI according to an embodiment of the present invention, a touch pattern for detecting a user contact with a device having a tactile sensor and determining a contact pattern of a user according to the detected user contact in the touch sensing step And determining the GUI corresponding to the determined contact pattern of the user.

In the method for displaying a GUI according to the first embodiment of the present invention, the contact pattern determination step may include determining a user's contact pattern according to the detected user's contact, and estimating a user's manipulation hand according to the determined user's contact pattern. It is characterized by.

In the method for displaying a GUI according to the first embodiment of the present invention, the contact pattern determination step may include generating a unit group by grouping sensor units that sense a contact among adjacent sensor units among a plurality of sensor units constituting the tactile sensor. Generating contact pattern information based on the generated unit group, comparing the generated contact pattern information with previously stored contact pattern information, and the generated contact pattern information included in a range of previously stored contact pattern information. In this case, the method may include estimating the manipulation hand based on the hand information corresponding to the generated contact pattern information.

In the method for displaying a GUI according to the first embodiment of the present invention, the contact pattern determination step may include generating a unit group by grouping sensor units that sense a contact among adjacent sensor units among a plurality of sensor units constituting the tactile sensor. Determining a sensor in which a unit group having the maximum number of sensor units is located (hereinafter, 'maximum unit group sensor'), determining whether an additional unit group exists in the maximum unit group sensor, and no additional unit group exists. Otherwise, estimating the manipulation hand with the left hand when the maximum unit group sensor is the left sensor, and estimating the manipulation hand with the right hand when the maximum unit group sensor is the right sensor.

In the method for displaying a GUI according to the first embodiment of the present invention, the contact pattern determination step may include generating a unit group by grouping sensor units that sense a contact among adjacent sensor units among a plurality of sensor units constituting the tactile sensor. The method may include calculating the number of unit groups located in the left sensor and the right sensor, and estimating the manipulation hand based on the calculated number of unit groups.

In the method for displaying a GUI according to the first embodiment of the present invention, the display step includes arranging icons along the line of the thumb of the left hand when the manipulation hand is estimated to be the left hand, and when the manipulation hand is estimated to be the right hand, the right hand thumb. You can arrange icons along the line of your finger.

In the method for displaying a GUI according to the first embodiment of the present invention, the display step includes displaying a scroll bar on the left side of the display screen when the manipulation hand is estimated to be the left hand, and displaying a scroll bar when the manipulation hand is estimated to be the right hand. Can be arranged on the right side of the.

In the method for displaying a GUI according to the second embodiment of the present invention, the contact pattern determination step is characterized by determining the grip type of the user based on the contact position input by the user.

In the method for displaying a GUI according to the second embodiment of the present invention, the contact pattern determining step may further include determining whether the contact pressure input by the user corresponds to a pressure equal to or greater than a preset intensity.

The method for displaying a GUI according to a second embodiment of the present invention may further include determining a GUI arrangement pattern based on a currently running application and the determined grip type of the user.

In the method for displaying a GUI according to the second embodiment of the present invention, the displaying step may include displaying the GUI at a predetermined point of a display screen set in correspondence with a contact position input by a user.

In the method of displaying a GUI according to a second embodiment of the present invention, the displaying step may be performed by changing the GUI according to a contact pressure input by a user.

The apparatus for displaying a GUI according to an embodiment of the present invention, when receiving a tactile detection signal from a tactile sensor for detecting a user's touch, a display for displaying a GUI, and a tactile sensor, the user's touch according to the tactile detection signal The controller may include determining a pattern and controlling the display unit to display a GUI in response to the determined contact pattern of the user.

According to the present invention, a user may operate a device having a touch screen with one hand without covering the touch screen screen, thereby preventing the user from generating an unintended touch. In addition, in the case of a device which can be operated using the tactile sensor, the user can conveniently input the operation command by displaying the GUI according to the position of the finger holding the device.

In the present invention, 'GUI (Graphic User Interface)' means a graphic displayed on the display screen. The GUI includes windows, icons, scrollbars, and the like, which can be any graphics used by a user to enter commands into the device.

In the present invention, the "operating hand" means a hand used to operate a device having a touch screen. The operating hand includes not only a hand performing a touch operation on the touch screen, but also all hands that are in contact with the device with the touch screen. That is, the hand holding the device may also correspond to the manipulation hand. The manipulation hand according to the embodiment of the present invention may be left hand, right hand or both hands.

In the present invention, the "tactile sensor" and the "tactile sensor" correspond to a sensor for detecting a user's touch. The tactile sensor may be positioned on the side of the device as a component that is distinguished from the touch sensor constituting the touch screen. When the user grips the device, the tactile sensor recognizes contact of the user's hand and transmits a tactile sensor to the controller. The tactile sensor may be configured as a tactile sensor or a combination of a pressure sensor and a touch sensor capable of measuring the degree of pressure and the position of contact / pressure. In addition, the tactile sensor may be configured as a left sensor and a right sensor, the left sensor and the right sensor may be composed of a plurality of sensor units, respectively. According to an embodiment of the present invention, the tactile sensor may be formed on the upper side and the lower side of the device, or may be formed on all sides of the device.

In the present invention, the 'sensor unit' is an element constituting the tactile sensor, and each sensor unit senses a user's touch. The tactile sensor includes a plurality of sensor units, and the number of sensor units is determined according to the type and size of the sensor.

In the present invention, the 'unit group' refers to a unit set including sensor units arranged in succession (adjacent) among the sensor units sensing a contact. The unit group is used to generate contact pattern information, and the position where the unit group is formed, the number of sensor units included in one unit group, and the like may be changed according to a method of holding a device by a user.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. At this time, it should be noted that the same components in the accompanying drawings are represented by the same reference numerals as possible. In addition, detailed descriptions of well-known functions and configurations that may blur the gist of the present invention will be omitted.

1 is an internal configuration diagram of an apparatus 100 referred for description of an embodiment of the present invention.

The apparatus 100 according to the embodiment of the present invention may correspond to any device having a touch screen, such as a mobile communication terminal, a portable terminal including a PDA, a computer, and a TV. The device 100 having a touch screen includes a tactile sensor 110, a storage 120, a touch screen 130, and a controller 140.

The tactile sensor 110 detects the touch of the user's hand when the user grips the device 100. The tactile sensor 110 may be configured as any one of a tactile sensor, a touch sensor, and a combination of a pressure sensor and a touch sensor capable of measuring a degree of pressure and a position of contact / pressure. The tactile sensor 110 may be located at the side of the device 100, but is not limited thereto. According to the exemplary embodiment of the present invention, the tactile sensor 110 may be located on all sides of the device 100. FIG. 1B illustrates a state in which the tactile sensor 110 is located on the side of the device 100. When the tactile sensor 110 detects the touch of the user's hand, the tactile sensor 110 transmits the tactile sensor to the controller 140. The tactile sensor 110 may be configured as a left sensor and a right sensor, and the left sensor and the right sensor may be configured as a plurality of sensor units, respectively.

The storage unit 120 stores all data related to the device 100. According to an exemplary embodiment of the present invention, the storage unit 120 stores data and contact pattern information related to a method of estimating a manipulation hand.

The touch screen unit 130 is a component that displays data and senses a user's input, and includes a display unit 132 and a touch sensor unit 134.

The display unit 132 may be formed of an LCD or the like, and displays various display data and operation states generated in the device on the screen. In the present invention, the display unit 132 serves to display a GUI corresponding to the manipulation hand. In addition, the display unit 132 serves to display the GUI according to the position of the finger holding the device 100.

The touch sensor unit 134 detects whether a user's hand touches the touch screen. The touch sensor unit 134 may be configured as a capacitive touch sensor or a pressure sensor, and not limited thereto. Any sensor capable of detecting a touch of an object may be used. The touch sensor unit 134 transmits a touch signal to the controller 140 when a touch of the user's hand is detected. The touch signal may include whether a touch of the touch mechanism is sensed and touch coordinate information of the touch mechanism. In the apparatus for displaying a GUI according to the second embodiment of the present invention, the touch sensor unit 134 may be omitted.

The controller 140 performs a function of controlling the overall operation of the portable terminal. According to the exemplary embodiment of the present invention, when the controller 140 receives the tactile sensor signal from the tactile sensor 110, the controller 140 determines a user's contact pattern according to the tactile sensor signal. In addition, the display unit 132 displays a GUI corresponding to the determined user contact pattern.

2 is a flowchart illustrating a method of displaying a GUI according to a manipulation hand according to the first embodiment of the present invention.

In step 210, the tactile sensor 110 detects a user's touch. The user's contact is made by the user holding the device 100. When the tactile sensor 110 detects a user's touch, the tactile sensor 110 transmits a tactile sensor to the controller 140. In step 220, the control unit 140 receives a tactile sensing signal from the tactile sensing sensor unit 110. In step 230, the controller 140 determines the user contact pattern, and estimates the manipulation hand according to the determined user contact pattern. The process of determining the user contact pattern and estimating the manipulation hand is described in detail with reference to FIGS. 3 to 5. The controller 140 displays the GUI corresponding to the manipulation hand after completing the user contact pattern determination and manipulation hand estimation process. Thereafter, the controller 140 determines whether an additional tactile detection signal is received in step 250, and when receiving the signal, the controller 140 performs a user contact pattern determination and manipulation hand estimation process in step 230. The additional tactile detection signal corresponds to a signal transmitted by the tactile detection sensor unit 110 to the control unit 140 by detecting the user's contact when the device 100 is newly gripped by changing the shape of the user's existing grip. . If the controller 140 does not receive the additional tactile sensing signal, the display unit 132 maintains the current GUI display state. Thereafter, the user inputs a command to the device 100 by manipulating the GUI displayed on the touch screen unit 130.

3 is a flowchart illustrating a specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

In step 310, the controller 140 generates a unit group based on the tactile detection signal received from the tactile sensor 110. The unit group refers to a unit set including sensor units arranged in succession (adjacent) among the sensor units that detect a contact.

Sensor units and unit groups are shown in FIGS. 6A-7B. 6A is a view showing the device 100 held by the left hand and a tactile sensor 110 positioned on the side of the device 100. The left sensor refers to the sensor located on the left side of the device 100, the right sensor refers to the sensor located on the right side of the device 100. Each sensor consists of a plurality of sensor units. The number of sensor units can vary depending on the size of the sensor unit. That is, as the size of the sensor unit decreases, the number of sensor units included in the same space increases. In the present specification, for the sake of clarity, the description will be made based on the assumption that the tactile sensor unit comprises 23 sensor units. In the left sensor of FIG. 6A, the sensor unit to which the left palm is in contact is indicated by hatching. In the right sensor of FIG. 6A, the sensor unit in which the index finger, the middle finger, the ring finger, and the hand (the little finger) are in contact with each other except for the thumb is indicated by hatching. If sensor units that sense a user's contact are grouped by adjacent sensor units, nine sensor units arranged in succession of the left sensor are grouped into one unit group. Two sensor units arranged in succession of the right sensor are grouped into one unit group, and a total of four unit groups are formed in the right sensor.

The controller 140 generates a unit group in operation 310 and then generates contact pattern information based on the generated unit group. The contact pattern information may be changed depending on how the user grips the device 100. The contact pattern information includes information about the number of unit groups formed in each sensor, the location where the unit groups are formed, the distance between the formed unit groups, the number of sensor units included in the unit groups, and the pressure sensed by each sensor unit. May be included.

Referring to FIG. 6A, one unit group is formed in the contact pattern information of the left sensor, a unit group is formed at a position from the 12th sensor unit to the 20th sensor unit, and the unit group includes nine sensor units. May be included. 4 unit groups are formed in the contact pattern information of the right sensor of FIG. 6A, and each unit group is formed at positions of the 4th, 5th, 9th, 10th, 14th, 15th and 19th, 20th sensor units, From the top, there are three sensor unit gaps between the first and second unit groups, three sensor unit gaps between the second and third unit groups, and between the third and fourth unit groups. There are three sensor unit intervals, and the four unit groups may each include two sensor units.

The controller 140 extracts contact pattern information stored in the storage 120 in step 330. The stored contact pattern information stores contact pattern information according to a user's gripping method. There may be a plurality of stored contact pattern information, and different contact pattern information is stored according to each holding method. In the contact pattern information stored in the storage unit 120, the number of unit groups, the position at which the unit groups are formed, the distance between the unit groups formed, the number of sensor units included in each unit group, and each sensor unit detected Information about pressure may be included.

The controller 140 compares the contact pattern information generated in step 340 with the extracted contact pattern information. The controller 140 compares the number of unit groups, the position at which the unit groups are formed, the spacing between the formed unit groups, and the number of sensor units included in each unit group.

The controller 140 determines whether the contact pattern information generated in operation 350 is included in a range of stored contact pattern information. When the generated contact pattern information is included in the range of the stored contact pattern information, the generated contact pattern information perfectly matches the stored contact pattern information or when the error range is set in the storage unit 120, the generated contact The pattern information is included in the error range of the stored contact pattern information. The error range may be set for each of the number of unit groups, a location where unit groups are formed, an interval between unit groups to be formed, and a number of sensor units included in each unit group.

As a result of the determination, when the generated contact pattern information is included in the stored contact pattern information, the controller 140 estimates an operation hand corresponding to the contact pattern information generated in step 360. The storage unit 120 stores information on the corresponding operation hand for each contact pattern information. When the generated contact pattern information is included in the stored contact pattern information range, the controller 140 estimates an operation hand corresponding to the generated contact pattern information. The putative manipulation hand may be the left hand or the right hand.

If the generated contact pattern information is not included in the stored contact pattern information range, the controller 140 estimates the manipulation hand with both hands in step 370. After finishing the manipulation of hand manipulation, the controller 140 proceeds to step 240 and displays a GUI corresponding to the manipulation hand.

4 is a flowchart illustrating another specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

In step 410, the control unit 140 generates a unit group by grouping each of the sensor units arranged in series among the sensor units that detect a contact. The controller 140 calculates the number of sensor units included in each unit group in step 420. One unit group present in the left sensor of FIG. 6A includes nine sensor units. The four unit groups present in the right sensor of FIG. 6A each comprise two sensor units.

After calculating the number of sensor units for each unit group, the controller 140 determines which sensor is the maximum unit group sensor in step 430. The maximum unit group sensor refers to a sensor in which a unit group having the maximum number of sensor units is located. The maximum unit group sensor may be either a left sensor or a right sensor. The controller 140 determines one of the left sensor and the right sensor. In FIG. 6A, the unit group including the most sensor unit is a unit group including nine sensor units existing in the left sensor, and the controller 140 determines the maximum unit group sensor as the left sensor. In FIG. 6B, the unit group including the most sensor units is a unit group including nine sensor units existing in the right sensor, and the controller 140 determines the maximum unit group sensor as the right sensor. In the same manner, the controller 140 determines the maximum unit group sensor as the left sensor in FIG. 7A, and the maximum unit group sensor as the right sensor in FIG. 7B.

If it is determined that the maximum unit group sensor is the left sensor, the controller 140 determines whether an additional unit group exists in the left sensor in step 440. The additional unit group refers to a unit group except a unit group in which the number of sensor units is maximum. In the left sensor of FIG. 6A, only a unit group having the maximum number of sensor units exists and no additional unit group exists. On the other hand, in the left sensor of FIG. 7A, in addition to the unit group in which the number of sensor units is the largest, there is a unit group composed of the third to fifth sensor units. When there is no additional unit group as in the left sensor of FIG. 6A, the controller 140 estimates the manipulation hand as the left hand. The unit group having the maximum number of sensor units is a portion where the palm of the user's left hand is in contact, and the absence of an additional unit group means that the thumb of the left hand is not in contact with the tactile sensor. In this case, the user is likely to touch the touch screen with the thumb of the left hand, and both the hand holding the device 100 and the hand touching the touch screen correspond to the left hand, and the controller 140 controls the operation hand to the left hand. It is estimated.

The same applies to the case where the manipulation hand is assumed to be the right hand. If the controller 140 determines that the unit group having the maximum number of sensor units is located in the right sensor in step 430, the controller 140 determines whether an additional unit group exists in the right sensor in step 460. As a result of the determination, when no additional unit group exists, the controller 140 estimates the manipulation hand as the right hand in step 470.

The controller 140 determines whether an additional unit group exists in the left sensor or the right sensor in step 440 or 460, and if present, estimates the manipulation hand with both hands in step 480. If an additional group of units is present, it means that the thumb of the hand holding the device 100 has touched the tactile sensor of the device 100, in which case the user can hold the device 100 with one hand. Since the touch screen is likely to be touched by a hand other than a hand to hold and hold, the control unit 140 estimates the manipulation hand with both hands.

FIG. 5 is a flowchart illustrating still another specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

In operation 510, the controller 140 generates a unit group by grouping the sensor units arranged in succession among the sensor units that detect the contact. In step 520, the controller 140 calculates the number of unit groups located in each sensor. The sensor may include a left sensor and a right sensor, and the controller 140 calculates the number of unit groups located in the left sensor and the number of unit groups located in the right sensor.

The number of unit groups located in the left sensor of FIG. 6A is one, and the number of unit groups located in the right sensor of FIG. 6A is four. There are four on the left sensor and one on the right sensor, two on the left sensor and four on the right sensor in FIG. 7A, four on the left sensor and two on the right sensor in FIG. 7B. .

In operation 530, the controller 140 determines whether the number of unit groups is three or more in the left sensor and the number of unit groups in the right sensor is one or less. If the requirement is satisfied, the controller 140 estimates the manipulation hand as the right hand in step 540. When the number of unit groups is 3 or more in the left sensor, at least 3 or more of the right hand index finger, ring finger, middle finger, and holding finger are in contact with the left sensor, and the number of unit groups is 1 in the right sensor. In the case of less than a dog, only the palm of the right hand contacts the tactile sensor 110, and the thumb of the right hand does not touch the right sensor. In this case, the user is likely to touch the touch screen with the thumb of the right hand, and both the hand holding the device 100 and the hand touching the touch screen correspond to the right hand, so that the controller 140 controls the operation hand with the right hand. It is estimated.

The same applies to the case where the manipulation hand is estimated with the left hand. The controller 140 determines whether there are three or more unit groups of the right sensor and one or less unit groups of the left sensor in step 550. If the requirement is satisfied, the controller 140 estimates the manipulation hand as the left hand in step 560.

If the number requirement of the group of units corresponding to each step is not satisfied in steps 530 and 550, the controller 140 estimates the manipulation hand as both hands in step 570.

8 is a diagram illustrating a screen displaying a GUI according to an embodiment of the present invention.

Reference numeral 810 of FIG. 8 corresponds to a display screen indicating a menu icon when the manipulation hand is estimated to be the left hand. The controller 140 arranges a menu icon along a line of the thumb of the left hand from the upper left to the lower right of the display screen. The user may touch the icon using the thumb of the left hand and execute a function corresponding to the touched icon. Since the menu icons are arranged along the copper wire of the left hand thumb, the menu icons are not covered by the left hand thumb, and a part of the left hand thumb touches the touch screen to prevent the user from touching an icon not intended.

Reference numeral 820 of FIG. 8 corresponds to a display screen indicating a menu icon when the manipulation hand is estimated to be the right hand. The controller 140 arranges a menu icon along a moving line of the right thumb. Reference numeral 830 of FIG. 8 denotes a screen displaying a menu icon when the manipulation hand is estimated to be both hands. When the operation hand is estimated to be both hands, the GUI is displayed according to the general GUI display method. The general GUI display method can be changed according to the setting.

9 is a diagram illustrating a screen displaying a GUI according to another exemplary embodiment of the present invention.

Reference numeral 910 of FIG. 9 corresponds to a display screen showing a scroll bar when the manipulation hand is estimated to be the left hand. The controller 140 displays the scroll bar in a vertical direction on the left side of the display screen. The user can operate the scroll bar using the thumb of the left hand. The user may operate the scroll bar without covering the display screen with his left thumb, and may prevent the user from touching an icon or content not intended by the user.

Reference numeral 920 of FIG. 9 denotes a screen for displaying a scroll bar when the manipulation hand is estimated to be the right hand. The controller 140 displays a scroll bar in a vertical direction on the right side of the display screen. The user may operate the scroll bar by using the right thumb of the user without covering the display screen, and may prevent an unintended touch by the user. Reference numeral 830 corresponds to a display screen indicating a scroll bar when the manipulation hand is estimated to be both hands. When the manipulation hand is estimated to be both hands, a general GUI is displayed, and the controller 140 displays a scroll bar on the right side of the display screen. The general GUI display method can be changed according to the setting.

11 is a flowchart illustrating a method of displaying a GUI according to a user contact pattern according to a second embodiment of the present invention. The second embodiment of the present invention may be applied when a user inputs a command to the device 100 using the tactile sensor 110. Therefore, the second embodiment of the present invention can be applied to a device that is not based on a touch screen.

When the user inputs a contact to the device 100, the tactile sensor 110 detects the user's contact in operation 1110. The contact of the user may be made by the user holding the device 100. The user may hold the device 100 with one hand as shown in FIG. 12A, and the device 100 may be held with both hands as shown in FIG. 13A. When the tactile sensor 110 detects a user contact, the tactile sensor 110 transmits a tactile sensor, which includes information about a position and pressure of the user, to the controller 140. The controller 140 receives the tactile sensing signal from the tactile sensor 110 in step 1120, and determines a user contact pattern based on the tactile sensing signal received in operation 1130. When determining the user contact pattern, the controller 140 determines the grip type of the user based on the contact position and the pressure information of the user. The storage unit 120 according to an exemplary embodiment of the present invention may store data on a grip type list corresponding to a user's contact position and pressure. The controller 140 extracts a grip form corresponding to the contact position and the pressure of the user from the storage 120. For example, as illustrated in FIG. 12A, the user grips the device 100 with his left hand to detect that the sensor located on the left side of the device 100 has touched two places, and the sensor located on the right side has four places. When detecting that a contact has occurred, the controller 140 determines that the user grips the device 100 with his left hand.

The controller 140 displays the GUI at a predetermined point corresponding to the contact location of the user in step 1140. First, the controller 140 checks the currently running application and selects the corresponding GUI before displaying the GUI. For example, when the idle screen application is currently running, the controller 140 selects menu icons as a GUI to be displayed. When the camera application is currently running, the controller 140 may select an icon and a zoom-in-out scroll bar corresponding to a shooting function as a GUI to be displayed. Subsequently, the controller 140 determines a GUI arrangement pattern in consideration of a currently executed application and a user's gripping form. Referring to FIG. 12A, when the controller 140 determines that the currently running application is a standby screen application and determines that the user is holding the device 100 with his left hand, at least one of the detection, stop, ring finger, and possession The GUI arrangement pattern for arranging the menu icons so as to be close to the contacted portion is determined.

The controller 140 displays the GUI according to the GUI array pattern determined by controlling the display unit 132. At this time, the controller 140 checks the position where the user contact is input and displays the GUI at a predetermined point of the display unit 132 corresponding thereto. That is, the controller 140 displays the GUI at a predetermined point corresponding to the position in consideration of the position where the user's finger or palm touches.

12A illustrates a screen displaying a menu icon when a user touches an input in a standby screen application. In FIG. 12A, an icon is displayed horizontally at a position where the user's index finger, middle finger, and ring finger contact each other. According to an exemplary embodiment of the present invention, the controller 140 may control the display unit 132 to sequentially display menu icons having a high frequency of use. The storage unit 120 stores the rank of a menu having a high frequency of use, and when the user grips the device 100 in the idle screen application execution state, the controller 140 stores a high frequency of use stored in the storage unit 120. The rank of the menu may be checked, and the display unit 132 may be controlled to sequentially display a menu icon corresponding thereto. The menu icons displayed on the display unit 132 may be changed by the user's setting.

According to an exemplary embodiment of the present invention, the controller 140 may change and display the GUI according to the pressure of the user contact. Referring to FIG. 12A, when the user grips the device 100 and presses it hard to increase the contact pressure, the controller 140 controls the display unit 132 to move the icon position to the right edge of the display unit 132. It can be moved and displayed. That is, the controller 140 may control the display unit 132 to move and display the icon so as to be adjacent to the part where the index finger, middle finger, and ring finger are in contact. When the application is set to be executed when the icon is in contact with the right edge of the display unit 132, the user may adjust the contact pressure and determine whether to execute the application while checking the position of the icon displayed on the display unit 132.

In addition, the controller 140 may control the display unit 132 to change the size of the GUI according to the contact pressure of the user and display the same. Referring to FIG. 12A, when the user grips the device 100 by increasing the pressure, the controller 140 may control the display unit 132 to change the size of the icon and display the same. In addition, the controller 140 may control the display unit 132 to display a highlight effect on an icon when a user's contact with a large pressure is input. The controller 140 changes the position or shape of the GUI according to the contact pressure of the user and displays the GUI, thereby allowing the user to adjust the contact pressure.

13A illustrates a screen displaying a function icon when a user touches an input in a camera application. In FIG. 13A, the user grips the device 100 using both hands, and the device 100 is gripped using the thumb and index finger of the left hand and the thumb and index finger of the right hand. In FIG. 13A, the display unit 132 displays a photographing function icon directly under the part where the user's right hand index finger contacts, and displays a zoom-in-out scroll bar just above the part where the user's right hand thumb contacts. In FIG. 13A, when the user inputs a contact to the apparatus 100 by increasing the pressure of the right hand detection, the controller 140 may control the display unit 132 to move and display the photographing function icon upward. That is, the controller 140 may control the display unit 132 to move and display the photographing function icon so as to be close to the part where the right hand index finger contacts. In addition, when the user inputs a contact to the apparatus 100 by increasing the pressure of the right hand thumb, the controller 140 executes a zoom-in function and controls the display unit 132 to change the shape of the scroll bar so as to correspond thereto. Can be.

14A illustrates a screen displaying a function icon when a user touches an input in an MP3 application. In FIG. 14A, the user grips the device 100 with his left hand and displays an icon in parallel with the position of the thumb, index finger, middle finger, ring finger, and the hand of the user. According to an embodiment of the present invention, the control unit 140 may display an icon so as to correspond to the position of the thumb, the index finger, the middle finger, the ring finger, and the hand in accordance with a preset icon arrangement pattern. According to an exemplary embodiment of the present invention, an icon corresponding to a thumb, an index finger, a middle finger, a ring finger, and a hand may be changed by user setting. The user can hold the device 100 in a state of increasing the pressure to grip or input a tapping operation to control the MP3 application. At this time, the controller 140 controls the display unit 132 so that the finger The positions, sizes, and expression effects of the icons corresponding to the contacted parts may be displayed differently.

The controller 140 determines whether the contact of the user is changed in step 1150. For example, when the user changes the position and grips the device 100 again, the haptic sensor 110 detects that a new user contact has occurred. The controller 140 determines a user contact pattern from the tactile signal received from the tactile sensor 110, and rearranges and displays the GUI according to the determined user contact pattern.

12B illustrates a screen of changing and displaying a menu icon when re-entering a user contact in a standby screen application. FIG. 12B illustrates a form in which a user inputs a contact to the device 100 by lowering a contact position as compared with FIG. 12A. That is, in FIG. 12B, the positions of the thumb, the index finger, the middle finger, the ring finger, and the holder are all in contact with the lower position of the apparatus 100 as compared with FIG. 12A. The controller 140 recognizes the contact position and the pressure of the user from the tactile detection signal, determines the user contact pattern therefrom, and arranges and displays the icons corresponding thereto. At this time, the controller 140 recognizes that the user's finger position is changed, and changes and displays a menu icon to correspond to the changed finger position. 12A and 12B, the positions of the menu icons are displayed differently.

13B illustrates a screen of changing and displaying a function icon when re-entering a user contact in a camera application. FIG. 13B illustrates a form in which the user is in contact with the user by moving the position of the right hand index finger to the left as compared with FIG. 13A. The controller 140 recognizes that the portion where the right hand indexing contact is changed, and displays the shooting function icon to correspond to the changed finger position. 13B and 13A, the positions of the shooting function icons are displayed differently.

14B illustrates a screen for changing and displaying a function icon when re-entering a user contact in an MP3 application. FIG. 14B illustrates a form in which a user inputs a contact to the device 100 by lowering a contact position as compared with FIG. 14A. That is, in FIG. 14B, the positions of the thumb, the index finger, the middle finger, and the ring finger are all in contact with the lower positions as compared with FIG. 14A. In addition, while the body is in contact with the apparatus 100 in FIG. 14A, the body is not in contact with the apparatus 100 in FIG. 14B. The controller 140 recognizes that the user's finger position has changed and changes and displays a function icon to correspond to the changed finger position. The controller 140 displays a volume control scroll bar at a position corresponding to the change of the contact position of the thumb, displays a back function icon at the changed contact position of the index finger, and plays or pauses at the changed contact position of the middle finger. The icon is displayed and the stop function icon is displayed at the changed contact position of the ring finger. Since the possession does not touch the device 100, the controller 140 does not display the forward function icon on the display unit 132.

As such, the device 100 changes and displays the GUI as the position where the user's finger or the palm is in contact changes, thereby preventing the user from causing confusion in operating the device 100.

On the other hand, the embodiments of the present invention disclosed in the specification and drawings are merely presented specific examples to easily explain the technical contents of the present invention and help the understanding of the present invention, and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention can be carried out in addition to the embodiments disclosed herein.

1 is an internal configuration diagram of an apparatus 100 referred for description of an embodiment of the present invention.

2 is a flowchart illustrating a method of displaying a GUI according to a manipulation hand according to the first embodiment of the present invention.

3 is a flowchart illustrating a specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

4 is a flowchart illustrating another specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

FIG. 5 is a flowchart illustrating still another specific process of step 230 of FIG. 2 in the GUI display method according to the first embodiment of the present invention.

FIG. 6A is a view showing the device 100 held by the left hand and the tactile sensor 110 positioned at the side of the device 100 according to an embodiment of the present invention.

6B is a view showing the gripping device 100 with the right hand and the tactile sensor 110 positioned on the side of the device 100 according to an embodiment of the present invention.

FIG. 7A is a view showing the device 100 held by the left hand and the tactile sensor 110 positioned at the side of the device 100 according to another embodiment of the present invention.

FIG. 7B is a view showing a gripping device 100 with a right hand and a tactile sensor 110 positioned on the side of the device 100 according to another embodiment of the present invention.

8 is a diagram illustrating a screen displaying a GUI according to an embodiment of the present invention.

9 is a diagram illustrating a screen displaying a GUI according to another exemplary embodiment of the present invention.

10A illustrates a screen in which a menu icon is touched by a left thumb while a menu icon is displayed on a touch screen according to the related art.

10B is a screen in which a scroll bar is operated by the thumb of the left hand while the Internet access screen is displayed on the touch screen according to the related art.

11 is a flowchart illustrating a method of displaying a GUI according to a user contact pattern according to a second embodiment of the present invention.

12A illustrates a screen displaying a menu icon when a user touches an input in a standby screen application.

12B illustrates a screen of changing and displaying a menu icon when re-entering a user contact in a standby screen application.

13A illustrates a screen displaying a function icon when a user touches an input in a camera application.

13B illustrates a screen of changing and displaying a function icon when re-entering a user contact in a camera application.

14A illustrates a screen displaying a function icon when a user touches an input in an MP3 application.

14B illustrates a screen for changing and displaying a function icon when re-entering a user contact in an MP3 application.

Claims (28)

In a method for displaying a graphical user interface (GUI) in a device having a tactile sensor, A touch sensing step of sensing user contact with the device; A contact pattern determining step of determining a contact pattern of a user according to the detected user contact; And And a display step of displaying a GUI corresponding to the determined contact pattern of the user. The method of claim 1, The contact pattern determination step And determining a user's contact pattern according to the detected user's contact, and estimating a user's manipulation hand according to the determined user's contact pattern. The method of claim 2, The contact pattern determination step Generating a unit group by grouping sensor units sensing a contact among adjacent sensor units among the plurality of sensor units constituting the tactile sensor; Generating contact pattern information based on the generated unit group; Comparing the generated contact pattern information with previously stored contact pattern information; And And when the generated contact pattern information is included in a range of the pre-stored contact pattern information, estimating a manipulation hand based on hand information corresponding to the generated contact pattern information. . The method of claim 3, The contact pattern information is At least one of a number of unit groups formed in each tactile sensor, a position at which the unit groups are formed, a distance between the unit groups, a number of sensor units included in the unit groups, and a pressure sensed by the sensor unit GUI display method comprising the information on. The method of claim 2, The contact pattern determination step Generating a unit group by grouping sensor units sensing a contact among adjacent sensor units among the plurality of sensor units constituting the tactile sensor; Determining a maximum unit group sensor; Determining whether an additional unit group exists in the maximum unit group sensor; And Estimating the manipulation hand with the left hand when the maximum unit group sensor is the left sensor when there is no additional unit group, and estimating the manipulation hand with the right hand when the maximum unit group sensor is the right sensor GUI display method further comprising. The method of claim 2, The contact pattern determination step Generating a unit group by grouping sensor units sensing a contact among adjacent sensor units among the plurality of sensor units constituting the tactile sensor; Calculating the number of unit groups located in the left sensor and the right sensor; And And estimating a manipulation hand based on the calculated number of unit groups. The method of claim 6,  If the number of unit groups located in the left sensor is three or more, and the number of unit groups located in the right sensor corresponds to one or less, the manipulation hand is estimated as the right hand,  And estimating the manipulation hand to the left hand when the number of unit groups located on the right sensor is three or more and the number of unit groups located on the left sensor is one or less. Display method. The method of claim 2, The display step If the manipulation hand is assumed to be the left hand, icons are arranged along the line of the thumb of the left hand, And when the manipulation hand is estimated to be a right hand, arranging icons along a moving line of the thumb of the right hand. The method of claim 2, The display step If the manipulation hand is estimated to be left hand, a scroll bar is displayed on the left side of the display screen, And the scroll bar is arranged on the right side of the display screen when the manipulation hand is estimated to be the right hand. The method of claim 1, The contact pattern determination step GUI display method, characterized in that for determining the grip type of the user based on the contact position input by the user. The method of claim 10, The contact pattern determination step And determining whether the contact pressure input by the user corresponds to a pressure equal to or greater than a preset intensity. The method of claim 10, And determining a GUI arrangement pattern based on the currently executing application and the determined grip type of the user. The method of claim 1, The display step And displaying the GUI at a predetermined point on the display screen set corresponding to the contact position input by the user. The method of claim 13, The point is GUI display method, characterized in that the point corresponding to the straight line in the direction in which the display screen of the device from the position where the user's contact is input. The method of claim 1, The display step GUI display method, characterized in that for changing the GUI according to the contact pressure input by the user. The method of claim 15, The display step And displaying at least one of a position and a size of the GUI. In a device for displaying a GUI, A tactile sensor detecting a user's touch; A display unit for displaying the GUI; And Receiving a tactile detection signal from the tactile sensor, the control unit determines a user's contact pattern according to the tactile detection signal, and includes a control unit for controlling the display unit to display a GUI corresponding to the determined user's contact pattern Device characterized in that. The method of claim 17, The haptic sensor unit And a left sensor and a right sensor each comprising a plurality of sensor units. The method of claim 17, The haptic sensor unit A device comprising a tactile sensor, a touch sensor and a combination of a pressure sensor and a touch sensor capable of measuring the degree of pressure and the position of contact / pressure. The method of claim 18, The control unit A unit group is generated by grouping the sensor units arranged in series among the sensor units that detect a contact, Generating contact pattern information based on the generated unit group, Comparing the generated contact pattern information with previously stored contact pattern information, And when the generated contact pattern information is included in a range of the previously stored contact pattern information, estimating an operation hand based on hand information corresponding to the generated contact pattern information. The method of claim 18, The control unit A unit group is generated by grouping the sensor units arranged in series among the sensor units that detect a contact, Determine the maximum unit group sensor, It is determined whether an additional unit group exists in the maximum unit group sensor. When the additional unit group does not exist, when the maximum unit group sensor is the left sensor, the manipulation hand is estimated to be the left hand, and when the maximum unit group sensor is the right sensor, estimating the manipulation hand with the right hand. Characterized in that the device. The method of claim 18, The control unit A unit group is generated by grouping the sensor units arranged in series among the sensor units that detect a contact, Calculating the number of unit groups located in the left sensor and the right sensor, If the number of unit groups located in the left sensor is three or more, and the number of unit groups located in the right sensor corresponds to one or less, the manipulation hand is estimated as the right hand,   And when the number of groups located in the right sensor is three or more and the number of group of units located in the left sensor is one or less, the apparatus according to claim 1, wherein the manipulation hand is estimated as the left hand. The method of claim 17, The control unit Device for determining the gripping type of the user by receiving information about the contact position input by the user from the tactile sensor sensor. The method of claim 17, The control unit And receiving information about the contact pressure input by the user from the tactile sensor, and determining whether the contact pressure input by the user corresponds to a pressure equal to or greater than a preset intensity. 24. The method of claim 23, The control unit And determining a GUI array pattern based on the currently running application and the determined grip type of the user. The method of claim 17, wherein the display unit If the manipulation hand is assumed to be the left hand, the icons are arranged along the same line of the thumb of the left hand, or a scroll bar is displayed on the left side of the display screen, And when the manipulation hand is estimated to be a right hand, arrange icons along a line of the thumb of the right hand, or display a scroll bar on the right side of the display screen. The method of claim 17, The display unit And displaying a GUI at a predetermined point corresponding to a position at which the user inputs a contact. The method of claim 17, The display unit And change the position of the GUI according to the contact pressure input by the user.

Images