TWI444867B - Tactile presentation device and control method thereof - Google Patents

Description

Touch display device and control method of touch display device

The present invention relates to a tactile display device, and more particularly to a tactile display device corresponding to a pressing operation for a touch sensor, and a control method of the device.

In recent years, in a portable terminal such as a mobile phone, an input device such as an operation unit or a switch for sensing a user's contact operation includes a touch panel or a touch switch (touch). A touch sensor of a touch sensor or the like. In addition to the portable terminal, it is an information machine such as an electronic calculator or a ticket vending machine, a microwave oven, a home appliance such as a television or a lighting fixture, and a commercial machine (Factory Automation (FA) machine). An input device including a touch sensor is also widely used.

Among such touch sensors, various methods such as a resistive film method, an electrostatic capacitance method, and an optical type are known. However, any type of touch sensor is a touch operation of sensing a finger or a stylus pen, and even if the touch sensor itself is touched, it does not physically shift like a button switch.

For this reason, the operator cannot obtain feedback for the operation even if the touch sensor is touched. Thus, since the operator cannot obtain the operational feeling when performing the contact operation, there is no method for confirming whether or not the contact operation has been appropriately sensed by the terminal. Therefore, in this way In the terminal, there is a case where the operator can easily perform repeated operations such as multiple contact operations at the same position, thereby putting pressure on the operator.

As a method of preventing such repeated contact operations, for example, there is known a method of emitting a sound when the terminal appropriately recognizes a contact operation. Further, for example, it is known to change the display form, that is, a user interface object for inputting an input button for displaying an image or the like in a display portion corresponding to a position at which the contact operation is sensed ( Hereinafter, the display color of the "target" is referred to as a change. If these methods are used, the operator can confirm whether his or her contact operation has been properly sensed by the terminal by hearing or visual.

However, in the case of relying on an audible feedback method, confirmation by an operator in a noisy environment becomes difficult. In addition, when the machine is in a mute state such as a manner mode, it is impossible to cope with the situation by relying on the feedback of the hearing. Further, in the case of the visual feedback method, if the size of the object displayed on the display portion is small, when the operator performs the contact operation by the finger in particular, the target is hidden under the finger and the display form cannot be confirmed. Variety.

In response to this, a feedback method in which the touch sensor is vibrated when the touch sensor senses contact is not provided depending on the sense of hearing or vision (for example, refer to Patent Document 1 and Patent Document 2).

[Patent Document 1]: Japanese Patent Laid-Open Publication No. 2003-288158

[Patent Document 2]: Japanese Patent Laid-Open Publication No. 2008-130055

In the devices disclosed in Patent Document 1 and Patent Document 2, when the contact with the touch sensor is sensed, the fingertip of the operator or the like can be displayed. The touch caused by vibration. Therefore, an operator who uses such a device for a contact operation can recognize by touch that the contact operation performed on the touch sensor has been properly sensed by the device.

However, in order to display a tactile sensation when a contact is sensed using such a device, it is necessary to pre-program the process of displaying the tactile sensation corresponding to a specific condition to an application software (hereinafter, simply referred to as "application"). in. That is, in the device, if an application developed by generating vibration in advance is used, the tactile sensation can be displayed corresponding to a specific condition. However, if an application that does not assume vibration is directly applied to the device, the tactile sensation cannot be properly displayed.

Therefore, in order to properly display the tactile sensation corresponding to a specific condition in the device as described above, it is necessary to newly develop an application or add some changes to an existing application, such as adding a process for displaying a tactile sensation, etc. .

Specifically, when a contact with a touch sensor corresponding to a position of a target such as a button or a button for displaying an image in a display portion of the touch sensitive display device is sensed, the touch sensitive display device must be used The process of displaying a specific tactile sensation is added to the application. Further, even if the contact of the touch sensor corresponding to the position of the target such as the button or the button in which the image is not displayed in the display portion is sensed, it is necessary to make the device not display the tactile sensation.

If the presence or absence of the display of the tactile sensation is controlled in this way, the tactile display device can be operated to the operator when the contact is sensed via the user interface using the touch sensor and the graphical user interface (GUI). The fingertips and the like show the touch caused by the vibration. That is, the touch display device It is possible to notify the operator that the contact of the operator corresponding to the target of the button or the button displayed by the image has been sensed by displaying the tactile sensation. On the other hand, even if the contact with respect to the position of the target such as a button or a button does not exist, the tactile display device can generate no feedback.

In the case of such control, in general, the touch sensitive display device needs to cooperate with an application that manages the state of the GUI when the touch sensor is touched, that is, The status of the target such as a button or button displayed on the display unit.

Hereinafter, the processing assumed when the tactile sensation is displayed on the touch sensitive display device will be described.

Fig. 7 is a block diagram showing a schematic configuration of a tentacity display device assumed. As shown in FIG. 7 , the touch display device 100 includes a control unit 110 , an application execution unit 120 , a display unit 130 , a touch sensor 140 , and a tactile display unit 150 . Further, the control unit 110 includes a display control unit 112 and a tactile sensation control unit 114.

The control unit 110 controls the entire touch display device 100 with the respective functional units of the tactile display device 100 as the first. The application execution unit 120 reads various applications from a memory unit (not shown) and executes processing based on the application. In addition, in the following description, the application executed by the application execution unit 120 has previously added (programmed) a touch sensor corresponding to the position of the target displayed on the display unit 130. When 140 senses the contact, it displays a specific tactile sensation.

The display unit 130 is configured by a liquid crystal display, an organic electroluminescence (EL) display, or the like, and is displayed for An image of a target such as a button or button that reminds the operator of the area to be touched. The touch sensor 140 is generally disposed on the front surface of the display unit 130, and senses a contact operation of an operator's finger or the like on a target displayed in the display unit 130 by using a contact surface of the corresponding touch sensor 140. Further, the touch sensor 140 notifies the application execution unit 120 of the position of the contact sensed on the contact surface via the control unit 110. The touch display unit 150 is configured by a small centrifugal motor, a piezoelectric element, or the like, and vibrates the contact surface of the touch sensor 140 to display a tactile sensation to an object that is in contact with an operator's finger or the like.

The display control unit 112 of the control unit 110 performs control on display, that is, displays an image of a target or the like on the display unit 130, or changes the display form of the image, etc., based on the display data supplied from the application execution unit 120. . The tactile sensation control unit 114 controls the tactile sensation display unit 150 to display a vibration pattern based on a specific condition, in accordance with an instruction from the application execution unit 120.

FIG. 8 is a flowchart schematically showing an example of an operation of the tactile sensation display device 100 to display a tactile sensation based on the sense of contact.

The action of controlling the display of the tactile sensation of the tactile display device 100 is started at the time point when the contact with the contact surface of the touch sensor 140 is sensed. When the operation is started, first, the position of the contact with the contact surface is notified to the application execution unit 120 by the touch sensor 140 (step S100). When the application execution unit 120 receives the notification, it is determined whether the contact position corresponds to the position of the target in which the image or the button is displayed on the image in the display unit 130 (step S102). In step S102, the position of the image at which the contact position and the target of the button or the button are displayed in the display unit 130 is not Correspondingly, the application execution unit 120 ends the operation without instructing the display or the tactile sensation. In this case, the tactile sensation is not displayed.

On the other hand, in step S102, when the position of the contact corresponds to the position of the image of the target such as the display button or the button, the application execution unit 120 supplies the display data to the display control unit 112 for displaying the content. The display of the input has been sensed. Here, the display in which the content is sensed input refers to a display that the operator can visually confirm, for example, the target of a button or a button is displayed in reverse only, or a state indicating that a button or a button has been pressed is displayed. Images and so on. Upon receiving such display material from the application execution unit 120, the display control unit 112 changes the image displayed on the display unit 130 based on the display data (step S104).

Then, the application execution unit 120 instructs the operator that the tactile sensation control unit 114 has sensed the input to display a specific tactile sensation. Here, the display of the specific tactile sensation means providing feedback that can be recognized by the tactile sensation of the operator, for example, causing the contact surface of the touch sensor 140 to vibrate only instantaneously. Upon receiving an instruction to display such a tactile sensation, the tactile sensation control unit 114 performs control in accordance with the instruction to cause the tactile sensation display unit 150 to display a specific vibration mode. Thereby, the tactile sense display unit 150 displays the tactile sensation of the specific vibration mode to the contact object that contacts the contact surface of the touch sensor 140 (step S106). By performing such processing, the tactile sensation display device 100 can display the tactile sensation only when the input to the target of the portion to which the operator is to be touched is sensed via the user interface using the touch sensor and the GUI.

As described above, according to the touch display device 100, if the application corresponds to the action of displaying the tactile sensation, the problem can be performed without problems. Said action. That is, as long as it is an application in which a process of displaying a tactile sensation is incorporated in advance, a specific tactile sensation can be appropriately displayed by the above-described operation.

However, in the touch display device 100, if an application that does not correspond to the display of the tactile sensation is used, in this state, even if the display of the target changes according to the contact operation, the tactile sensation cannot be appropriately displayed. Therefore, in the tactile display device 100, in order to effectively utilize various application resources developed up to now, it is necessary to change various application programs and add a process for displaying a tactile sensation corresponding to a specific condition. The addition of such a change must be made for various applications, which is very laborious and costly.

Under such circumstances, in the future, it is a factor that hinders the development of applications for companies to join touch-sensitive display devices, and may become a major cause of delays in the adoption of touch sensors and GUI platforms.

Further, in the above-described operation, it is determined whether or not the position of the contact corresponds to the position of the target based on the presence or absence of the contact with the touch sensor 140 until the display of the target changes and the touch is displayed. It is a matter to be paid attention by the application execution unit 120. In order to develop or design an application that performs such processing, not only complicated work is required, but also the processing load of the application execution unit 120 that executes the application increases.

Further, for example, in the techniques disclosed in Patent Document 1 and Patent Document 2, the touch sensor is vibrated in response to the contact when the touch sensor senses the contact. Therefore, in the techniques disclosed in Patent Document 1 and Patent Document 2, it is also assumed that even if an application corresponding to the display of the tactile sensation is used, the tactile sensation cannot be appropriately displayed. Especially like A push button switch such as a mechanical push button switch (push button switch) is depicted in a device on a touch sensor, and a touch is sensed by gently touching a touch sensor with a finger or the like. Therefore, it is possible to react to the touch operation of the light touch by the operator in the previous stage of pressing (pressing) the touch sensor, which may cause an erroneous operation unlike the reaction of the drawn image (button).

Further, in the technique described in the patent document, even if the operator unintentionally pushes the button switch (that is, even if no force is applied), the action of touching the touch sensor is lightly touched (contact ) and the tactile sensation is displayed, which also gives the operator an uncomfortable feeling. For example, in the case where the touch sensor is in the form of a resistive film, if the upper conductive film contacts the lower conductive film, the touch is displayed, and in the case of the electrostatic capacitance mode, the touch is detected as long as the electrical signal of the contact is detected. sense.

Further, even in the device for displaying the tactile sensation in the past, the user interface based on the application in which the processing for displaying the tactile sensation by the touch sensor is preliminarily employed is not provided, and the mechanical like is not provided when the input is sensed. A sense of operation like a push button switch. That is, the tactile sensation displayed by the previous tactile display device is at most merely used to inform the operator that the contact has been sensed, and only the action of vibrating the contact surface or the like is performed. Therefore, the tactile sensation displayed by the previous tactile display device cannot provide the operator with a real operational feeling as when the mechanical push button switch is operated.

However, in a user interface using a touch sensor and a GUI, an image of a target such as a mechanical button switch that simulates reality is often displayed. Therefore, in the case of sensing the input of such a target, for the operator In terms of operational sense, it is still better to display the true touch of the mechanical button switch. When the input to the displayed target is sensed, the feedback to the operator can be improved as long as the feedback such as pressing a real button or button can be performed.

In the case of a mechanical push button switch, the operator presses the switch by applying a certain degree of pressing force, and a tactile sensation such as a metal dome switch can be obtained (hereinafter, referred to as a click sensation). ). Similarly, in the case of the mechanical push button switch, the operator can obtain the metal dome switch and the like back to the original by reducing the pressing force to a certain level during the release of the pressing force on the pressed switch. The tactile sensation of the pressed state (hereinafter, referred to as the release tactile sensation). Moreover, in the case of the mechanical push button switch, the operator does not display the click sensation even if the operator touches (contacts) the switch with a very weak pressing force, and since the state of the click sensation is not displayed, the release is not displayed. Touch. In the conventional touch display device using the touch sensor and the GUI, the click feeling and the release feeling like the mechanical button switch cannot be displayed.

In this regard, if a display of a real tactile sensation such as a click sensation or a release sensation is to be realized in the haptic display device, it is necessary to add all the required processing to the application for this purpose. In addition, in order to realize the display of such a tactile sensation, it is necessary to input the detection data of the sensor to the application according to the judgment and instruction of the application. In order to develop or design an application that performs such processing, not only complicated work is required, but also the processing load of the application execution unit 120 that executes the application is gradually increased.

Accordingly, it is an object of the present invention to provide a tactile sensation display device and a control method therefor, which do not waste existing application resources constituting a user interface, and correspond to target-oriented operations. The operation of the person to display the tactile sensation.

A touch display device of the invention that achieves the first aspect of the present invention includes: a touch sensor; a tactile display portion that vibrates a contact surface of the touch sensor; a display portion; and a control portion according to the application The information indicating the tactile sensation supplied by the program software sets the tactile sensation displayed on the tactile sense display unit to the target displayed on the display unit, and controls the tactile sensation when the input to the target is sensed. The display unit displays the tactile sensation set for the target by the tactile display unit pressing the pressing target on the contact surface.

According to a second aspect of the invention, in the touch display device of the first aspect, the control unit is configured to display an indication associated with the display of the tactile sensation outputted together with the information associated with the display by the application software. And setting a tactile sensation displayed by the tactile display unit to the target.

According to a third aspect of the invention, the first aspect or the second aspect of the touch display device further includes: a load detecting unit that detects a pressing load on a contact surface of the touch sensor; The control unit controls the tactile sense display unit to cause the load detecting unit to detect a pressing load that satisfies a load reference indicating a tactile sensation in a state in which an input to the target is sensed. The tactile sense display unit displays a tactile sensation to the pressed object.

Further, in the method of controlling a touch display device according to the fourth aspect of the present invention, in the control method of the touch display device, the touch display device includes: a touch sensor; a touch display unit, Making the touch a contact surface vibration of the sensor; and a display unit; the control method of the tactile display device sets the tactile sensation to an object displayed on the display unit based on information indicating a tactile sensation supplied by the application software a tactile sensation displayed by the display unit, when the input to the target is sensed, controlling the tactile sense display portion to cause the tactile sense display portion to display the target object set for the pressing object pressing the contact surface Touch.

According to the present invention, it is possible to provide a user interface that can effectively utilize an existing application without causing waste and use such an application to make the operability excellent.

The above and other objects, features and advantages of the present invention will become more <RTIgt;

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

The present invention is a tactile sensation display device that sets a tactile sensation to a target based on information for generating a user interface displayed on the display unit and information on tactile sensation supplied by the application software. Moreover, in the display of the tactile sensation, the operation of gently touching (contacting) the touch sensor does not display the tactile sensation, and the operator is consciously pressed (pushing) the touch sensing. The operation of the device will display the touch.

Fig. 1 is a block diagram showing a schematic configuration of a touch display device according to an embodiment of the present invention.

As shown in FIG. 1 , the touch display device 1 includes a control unit 10 , an application execution unit 20 , a display unit 30 , a touch sensor 40 , and a tactile display unit. 50. And the load detecting unit 60. Further, the control unit 10 includes a display control unit 12, a tactile sensation control unit 14, and an input control unit 16.

The control unit 10 controls the entire touch display device 1 with the respective functional units of the tactile display device 1. The application execution unit 20 reads various applications from a memory unit (not shown) and executes processing based on the application. In the present embodiment, the application execution unit 20 executes a case where the application for changing the touch is set to the target in an application that does not originally correspond to the display of the tactile sensation. A process of adding a change in the application to correspond to the display of the tactile sensation will be described later.

The display unit 30 displays an object such as a push button switch (push button switch) as an image. In addition, the push button switch refers to a mechanical button that can be actually pressed like an input button, a button, or the like (hereinafter, collectively referred to as "a button or the like"). In the present embodiment, the display unit 30 displays an object for simulating such a push button switch using an image. The display unit 30 uses, for example, a liquid crystal display panel, an organic EL display panel, or the like. The application execution unit 20 supplies information (display material) constituting an image of the user interface to the display control unit 12 when the user interface is generated or when the image of the target is changed. When the information constituting the user interface is supplied in this manner, the display control unit 12 can cause the display unit 30 to display the display based on the information. In addition, in the drawings and the following description, the "user interface" may be appropriately omitted as "UI".

The touch sensor 40 is disposed on the front surface of the display unit 30, and senses the pressing of the target displayed on the display unit 30 by the operator's finger or the like (pressing the object) by the contact surface of the corresponding touch sensor 40. Operation (contact). Moreover, the touch sensor 40 detects the contact with the contact surface, and detects the contact The position of the contact is notified to the application execution unit 20 via the input control unit 16. The touch sensor 40 is configured by, for example, a sensor of a known method such as a resistive film method, a capacitance method, or an optical type.

The tactile display unit 50 is configured by a piezoelectric vibrator or the like, and vibrates the contact surface of the touch sensor 40. The tactile display unit 50 generates a tactile sensation to the pressing target that presses the contact surface by causing the contact surface of the touch sensor 40 to vibrate. The pressing object that presses the contact surface is, for example, an operator's finger or a stylus pen or the like. The load detecting unit 60 detects a pressing load on the contact surface of the touch sensor 40, and is configured by, for example, an element that linearly reacts to a load such as a strain gauge sensor or a piezoelectric element. Further, the load detecting unit 60 notifies the touch control unit 14 of the detected pressing load on the contact surface of the touch sensor 40.

The display control unit 12 of the control unit 10 performs control on the display based on the display material supplied from the application execution unit 20, and causes the display unit 30 to display an image or the like in an image, or to change the image display or the like. The display control unit 12 receives the display material of the target used when generating the UI displayed on the display unit 30 from the application execution unit 20. Further, when the position at which the operator touches the touch sensor 40 is a position corresponding to the target of starting the specific operation, the display control unit 12 receives the content from the application execution unit 20 to indicate that the input has been accepted. Display data of the target.

The input control unit 16 notifies one or both of the application execution unit 20 and the tactile control unit 14 of the position of the contact of the touch sensor 40 by the operator.

The tactile sensation control unit 14 of the present embodiment includes a storage unit (not shown), and at least temporarily stores information on the tactile sensation set for each target based on an instruction from the application executed by the application execution unit 20. That is, in the present embodiment, as will be described later, the tactile sensation to the target can be set in the tactile sensation control unit 14. Therefore, when the touch to the target is set in the touch control unit 14 in this manner, the input control unit 16 notifies the touch control unit 14 of the position where the touch sensor 40 is in contact, so that the touch control unit 14 Control regarding the display of the tactile sensation can be performed. That is, in this case, the application execution unit 20 does not need to instruct an action regarding the display of the tactile sensation, for example, to receive a notification of the position of the contact of the touch sensor 40 to determine whether the position is a position corresponding to the target. .

When the tactile sensation control unit 14 receives a notification of the position of the contact with the touch sensor 40 from the input control unit 16, it is determined whether or not the position of the contact corresponds to the position at which the tactile sensation is set. When the notified contact position corresponds to the position at which the touch target is set, the tactile sensation control unit 14 controls the tactile display unit 50 to cause the tactile display unit 50 to display the set tactile sensation. Further, in the present embodiment, the load detecting unit 60 may display the tactile sensation when a specific pressing load is detected. That is, the touch control unit 14 satisfies the display touch feeling in the state where the touch sensor 40 senses the contact with the position corresponding to the target at which the specific processing is performed. In the case of the load reference, the tactile display unit 50 is controlled such that the tactile display unit 50 displays the set tactile sensation. Further, the tactile sense display unit 50 generates a vibration of the set specific pattern, thereby pressing the operator's finger or the like that presses the contact surface of the touch sensor 40. The pressed object displays the set touch.

2A and 2B are views showing an example of a mounting structure of the touch display device 1 shown in Fig. 1. Fig. 2A is a cross-sectional view of a principal part, and Fig. 2B is a plan view of a main portion. The display unit 30 is housed and held in the casing 61. The touch sensor 40 is held on the display unit 30 via an insulator 62 including an elastic member. Further, in the touch display device 1 of the present embodiment, the display unit 30 and the touch sensor 40 are formed in a rectangular shape in plan view. Further, in the touch display device 1, the touch sensor 40 is held on the display unit 30 via the insulator 62, and the insulator 62 is disposed on the display area A of the display unit 30 shown by the phantom line in FIG. 2B. 4 corners.

Further, an upper cover 63 is disposed on the frame 61 so as to cover the surface area of the touch sensor 40 away from the display area of the display unit 30. The top cover 63 and the touch sensor are disposed on the cover 63. Between 40, an insulator 64 including an elastic member is disposed. In addition, in the touch sensor 40 shown in FIG. 2A and FIG. 2B, the surface member having the contact surface 40a is composed of, for example, a transparent film or glass, and the back member is composed of glass or acrylic acid resin. . The touch sensor 40 uses a sensor in which the pressing portion is subjected to a small amount of deflection (strain) corresponding to the pressing force or a small amount of deflection of the structure itself if the contact surface 40a is pressed.

On the surface of the touch sensor 40, in the vicinity of each side covered by the top cover 63, a load (pressing force) applied to the touch sensor 40 is respectively provided by bonding or the like. The strain gauge sensor 51 is detected. In addition, on the back surface of the touch sensor 40, in the vicinity of opposite sides, pressure for vibrating the touch sensor 40 is respectively provided by bonding or the like. Electric vibrator 52. That is, in the tactile display device 1 shown in FIGS. 2A and 2B, the load detecting portion 60 shown in FIG. 1 is configured using four strain gauge sensors 51, and the tactile sense display portion 50 uses two piezoelectric vibrations. The device 52 is constructed. Further, the touch display unit 50 vibrates the touch sensor 40 to vibrate the contact surface 40a. In addition, the illustration of the frame body 61, the top cover 63, and the insulator 64 shown in FIG. 2A is omitted in FIG. 2B.

Next, in the present embodiment, an operation of setting the touch corresponding to the target to the tactile sensation control unit 14 based on the instruction of the application to be changed in accordance with the display of the tactile sensation will be described.

In the present embodiment, as described above, by adding a change to the application, the correspondence between the target and the information related to the tactile sensation displayed corresponding to the input to the target is added in advance to the change application. In the case where an application that does not originally correspond to the display of the tactile sensation is attached to all the processes (judgments or indications) of the action of displaying the tactile sensation, it is generally necessary to add various control information. It can be cumbersome and leads to increased costs.

On the other hand, in the present embodiment, information related to the display of the tactile sensation is added to the information related to the display of the UI among the information of the original application. Thereby, when the UI is first displayed on the display unit 30 or when the display of the UI is changed, information related to the display of the UI and information related to the tactile sensation of the target constituting the UI are supplied together. Therefore, the information associated with the display of the tactile sensation is supplied from the application, and thus the tactile sensation control unit 14 can set the tactile sensation to each of the targets constituting the UI when the UI is initially displayed or when the display of the UI is changed. That is, at this time, the touch control The unit 14 sets the tactile sensation displayed on the tactile display unit 50 to the target displayed on the display unit 30 based on the information indicating the tactile sensation supplied from the application executed by the application execution unit 20 located outside the control unit 10. .

Specifically, for example, in an application that uses a UI composed of a target, location information of the target (that is, information indicating which position in the region of the display UI of the display unit 30 is displayed) is usually performed. definition. Moreover, in the application, the size information of the target (that is, the range in which the touch is sensed in the touch sensor 40 corresponding to the position of the target) is also generally defined. In response to this, the type of the tactile sensation to be displayed when the input to the target is sensed, and the information indicating the pressing load, that is, the pressing load, etc., are attached to the application in advance. The position and size information of the target that constitutes the UI used in the program.

In the present embodiment, only by adding at least the information to the application, various kinds of tactile sensations can be displayed for the target constituting the UI to be used according to various conditions. On the other hand, for example, just because the tactile sensation is to be displayed, it is not necessary to attach a process of monitoring the contact of the touch sensor 40 or determining whether the position of the contact corresponds to the position of the target, or the like, to the application. in.

Fig. 3 is a flow chart for explaining an operation of generating a user interface (UI) according to the embodiment. This generation operation is an operation performed when a UI provided by the application is newly generated or a UI provided by the application is changed to a new UI based on an instruction of the application executed by the application execution unit 20. That is, in the present embodiment, when the UI provided by the application is newly generated, it is necessary The tactile sensation must be set for the target that constitutes the UI. Further, since the arrangement of the display target is also changed after the screen display of the UI is changed according to the instruction of the application, in this case, it is necessary to set the tactile sensation to the target constituting the UI that has changed.

The UI generation operation shown in FIG. 3 is started, for example, when a UI provided by an application is newly generated according to an instruction of an application, or when a UI provided by the application becomes a new UI. In addition, in the flowchart shown in FIG. 3, the operation seen from the viewpoint of the control part 10 is demonstrated. When the operation is started, the information (display material) associated with the display of the UI for generating the UI provided by the application is output to the display control unit 12 based on the application executed by the application execution unit 20. Therefore, the display control unit 12 of the control unit 10 receives the information associated with the display of the UI (step S1).

In the application program used in the present embodiment, as described above, the information associated with the display of the UI is added to the information associated with the display of the tactile sensation. Therefore, when the information associated with the display of the UI is output from the application to the display control unit 12, the information associated with the tactile sensation corresponding to the input of the target constituting the UI is also output. For this reason, the tactile sensation control unit 14 of the control unit 10 receives the information associated with the tactile sensation displayed corresponding to the input to the target (step S2).

Upon receiving the information associated with the tactile sensation displayed in response to the input to the target, the tactile sensation control unit 14 sets the tactile sensation displayed on the tactile sensation display unit 50 for each target constituting the UI (step S3). At this time, the set tactile sensation is: based on the information associated with the tactile display attached to the application. Various ideas. For example, the contact position and the contact range in the touch sensor 40 are specified, and the type of the touch (for example, the touch of the metal dome switch or the rubber switch) or the touch is set according to the target displayed in the position and the range. Form (such as the strength of feedback) and so on. Further, a load reference for displaying a tactile sensation is set for each target in accordance with, for example, information associated with the tactile display attached to the application. The load reference for displaying the tactile sensation refers to the specificity when the load detecting unit 60 detects the pressing load that satisfies the specific load reference, and displays the tactile sensation to the pressing target that presses the contact surface of the touch sensor 40. Load benchmark.

Specifically, for example, a relatively small (weak feedback) tactile sensation can be set for a small target. For example, in this case, the frequency at which vibration is generated or the time at which vibration is generated can be increased. On the other hand, a relatively large (strong feedback) touch can be set for a large target. For example, in this case, the frequency at which vibration is generated can be reduced, or the time at which vibration is generated can be prolonged. Further, in the case of a small target, for example, the load reference for displaying the tactile sensation is set to be relatively small, and the tactile sensation can be displayed by the relatively weak pressing of the operator. Further, in the case of a large target, for example, the load reference for displaying the tactile sensation is set to be relatively large, and if the operator does not perform relatively strong pressing, the tactile sensation is not displayed. Further, the tactile sensation control unit 14 stores information on the tactile sensation set for each target in a memory unit (not shown).

After the setting of the tactile sensation is completed in step S3, the control unit 10 performs UI generation processing (step S4). Specifically, the UI generation processing may be an operation or the like that causes the display unit 30 to display and display the UI based on the information related to the display of the UI received by the display control unit 12, or to sense the pair. An association is established between the applications launched when the target is entered.

As described above, by performing the UI generation operation of the present embodiment, the display unit 30 displays the UI, and upon sensing the input of the target constituting the UI, the preparation for displaying the tactile sensation is completed.

Next, after the UI is displayed on the display unit 30, the operation of the tactile sensation display device 1 in response to the pressing operation of the operator will be described.

In the touch-sensing display device 1 of the present embodiment, when the touch sensor 40 senses that the operator touches the target displayed by the image in the display unit 30, the load load detected by the load detecting unit 60 is satisfied. A specific benchmark shows the tactile sensation. When the touch load detected by the load detecting unit 60 satisfies a specific criterion, the touch is detected. The operator touches the touch sensor 40 without unintentionally (by mistake, etc.). Instead, the pressing (pushing) action is consciously performed.

FIG. 4 is a flowchart showing the operation of the touch display device 1 of the embodiment. When the operation of the tactile sensation display device 1 is performed, in accordance with the UI generation operation, an object such as a button can be displayed on the display unit 30 in an image so that the operator's finger, stylus, or the like can be sensed. operating. That is, hereinafter, the UI supplied by the changed application, that is, the UI for setting the touch of the target has been displayed. Further, in the application of this embodiment, when the touch sensor 40 corresponding to the position of the image of the target displayed in the display unit 30 senses the contact, the operator can recognize the feeling of the contact. It is set to perform some changes to the image of the target to display. However, in the present embodiment, such a change in image is not essential.

In the state in which preparation is completed as described above, the tactile sensation of this embodiment is apparent The operation of the display device 1 starts. After the operation of the touch sensitive display device 1 is started, first, the input control unit 16 monitors the presence or absence of contact with the contact surface of the touch sensor 40 by the operator's finger or the like (step S11). When the contact with the contact surface of the touch sensor 40 is sensed in step S11, the input control unit 16 notifies the application execution unit 20 and the tactile control unit 14 of the contact position (step S12).

After receiving the notification of the position of the contact in the touch sensor 40, the application execution unit 20 determines whether the position of the contact corresponds to the position of the image of the target in which the button or the like is displayed in the current GUI (step S13). . When the position of the contact corresponds to the position of the image of the display target, the application execution unit 20 supplies the display material for changing the image of the target based on the application-based setting to the display control unit 12. Upon receiving the display material, the display control unit 12 controls the display unit 30 to change the image of the displayed target (step S14). That is, the touch display device 1 changes the display image of the target in the case where the contact with the target of the button or the like is sensed. When the display image of the target is to be changed, it is preferable to enable the operator to visually recognize that the button has been touched (touched), for example, to indicate that the target has been selected, or to change the color or brightness of the target. The display for changing the image of such a target can be considered in various forms by setting based on the application.

In step S13, the input control unit 16 determines whether the position of the contact corresponds to the position of the image of the display target. Therefore, after the input to the target is sensed in step S13 (that is, after there is contact with the target of the button or the like), the processing of the display of the tactile sensation after step S15 may not pass the response. The program execution unit 20 performs the process. In step S15, the tactile sensation control unit 14 determines whether or not the pressing load detected by the load detecting unit 60 is increased by the pressing of the contact surface of the touch sensor 40 and satisfies a specific criterion. Further, the load detecting unit 60 detects the load based on, for example, the average value of the outputs of the four strain gauge sensors 51. Here, it is preferable that the load that satisfies the specific reference is set to a value of, for example, 1.5 N (Newton) or the like in accordance with the pressing load when the operator performs the normal pressing operation, and the setting can be changed thereafter. Further, since the specific load has taken into account the pressing load (for example, the average value or the like) at the time of the pressing operation based on the intention of the operator, the reference is not set too low. This is done so that the operation in the case where the operator unconsciously touches the light is not recognized as a pressing (pushing) operation, and a pressing feeling to give the operator a real tactile feeling to be described later.

When the pressing load satisfies the specific reference in step S15, the tactile sensation control unit 14 causes the contact surface of the touch sensor 40 to generate a specific vibration, and controls the tactile sense display portion 50 to cause the tactile sense display portion 50 to display the touch. Sense (step S16). In other words, the haptic control unit 14 controls the tactile sensation display unit 50 when the load detecting unit 60 detects the pressing load that satisfies the load reference for displaying the tactile sensation in a state in which the input of the target for executing the specific processing is sensed. The touch display unit 50 is caused to display a tactile sensation. By this control, the tactile display unit 50 displays a tactile sensation to the pressing target that presses the contact surface of the touch sensor 40. The touch displayed here is as described in FIG. 3, and refers to a tactile sensation set based on information received from an application.

Further, in the present embodiment, in the step S16, the tactile sensation displayed by the tactile sensation display unit 50 on the pressing target can be set as a button or button that clicks on the reality. The click sensation as shown in the case of the case. Here, in order to display an actual click feeling to the operator, the tactile sensation display device 1 stimulates the tactile sensation in a state where the operator's pressing feels the stimuli. That is, the tactile sensation display device 1 stimulates the feeling of pressing until the load applied to the touch sensor 40 satisfies the reference for displaying the tactile sensation (for example, 1.5 N), and when the load satisfies the reference, it is driven by the specific driving signal. The piezoelectric vibrator 52 vibrates the contact surface 40a to stimulate the sense of touch. Thereby, the tactile sensation display device 1 can display to the operator the same click sensation as that obtained when the push button switch such as the push button switch (push button switch) is pressed. Therefore, for the operator, even the virtual button switch depicted in the touch sensor can obtain the same true click touch feeling as when the actual button switch is operated, and can be used for the touch sensor. 40 operates so there is no sense of discomfort. Moreover, the operator can operate in conjunction with the "pressing" of the touch sensor 40, thereby preventing erroneous input caused by individual contacts.

The driving signal at the time of the click sensation, that is, the fixed frequency, period (wavelength), waveform, and amplitude of the stimulating touch, can be appropriately set according to the displayed click sensation. For example, when the click feeling represented by the metal dome switch used in the portable terminal is displayed, for example, the touch display unit 50 is driven by a one-cycle driving signal composed of a fixed frequency Sin wave of 170 Hz. . The touch display unit 50 is driven by such a drive signal, and the contact surface 40a is vibrated at about 15 μm in a state where a pressing load that satisfies a specific reference has been applied. Thereby, it is possible to display to the operator such a true click touch feeling as in the case of clicking an actual button.

Moreover, in step S15, the pressing load does not satisfy the specific reference. In other words, the tactile sensation control unit 14 determines whether or not the pressing load detected by the load detecting unit 60 is zero based on the weakening of the pressing of the contact surface of the touch sensor 40 (step S17). When the pressing load detected by the load detecting unit 60 is not zero in step S17, since the operation of the touch sensor 40 by the operator is not released (released), the tactile sensation control unit 14 returns to step S15. Continue processing. On the other hand, when the pressing load detected by the load detecting unit 60 is zero in step S17, it can be determined that the operator has released (released) the operation of the touch sensor 40, and the tactile sensation control unit 14 ends. The action shown in Figure 4. In addition, the operation in step S17 may be performed to determine whether or not the contact with the touch sensor 40 has been sensed, instead of determining whether or not the pressing load detected by the load detecting unit 60 is zero.

On the other hand, if the position of the contact does not correspond to the position of the image of the display target in step S13, the application execution unit 20 ends the operation shown in FIG. That is, in the case where the input to the target of the button or the like is not sensed, or the sensed input does not correspond to the target of the button or the like, the application executing portion 20 does not change the display image of the target. The processing of the operation shown in Fig. 4 is ended. Further, in this case, the input control unit 16 ends the processing without giving the tactile sensation control unit 14 an instruction to display the tactile sensation. Thus, in the case where the touch sensitive display device 1 has sensed the contact with a position other than the target of the button or the like, the operator can recognize that the input to the target is not sensed by not displaying the tactile sensation.

As described above, according to the present embodiment, when an input is sensed for a target that performs a specific operation, and a pressing load that satisfies a specific load reference is detected, the touch that is supplied by the application and set for the target is displayed. sense. Further, according to the present embodiment, it is possible to set an appropriate tactile sensation for each target by adding information related to the display of the tactile sensation to the application, and the operation of displaying the tactile sensation does not need to be performed by the application execution unit 20. get on. Therefore, according to the tactile sensation display device 1 of the present embodiment, when the input is sensed, the tactile sensation corresponding to the target can be displayed, and the existing application constituting the UI can be applied by adding the information associated with the display of the tactile sensation. Program. Further, in the touch display device 1 of the present embodiment, by pressing (pressing) an object such as a button, the operator can recognize that the input of the target has been correctly recognized by the actual click tactile sensation.

Next, the tactile sensation display device 1 will display an operation when the tactile sensation is displayed by detecting the pressing load that satisfies the reference in the state in which the contact with the operator is sensed, and then the tactile sensation is further displayed.

When the normal mechanical push button switch is pressed, the pressing force of the switch is weakened, and the metal dome switch and the like are returned to the original untouched state (hereinafter, To release the touch). In the touch display device 1 of the present embodiment, the true click touch feeling can be displayed in response to the pressing operation, and then the true release touch can be displayed in the middle of the release (release) of the pressing of the touch sensor. Provide a whole series of more natural sense of operation.

Fig. 5 is a flowchart showing the operation of the touch display device 1 of the embodiment. The operation shown in FIG. 5 is started after the operation of displaying the click sensation described in FIG. 4 (step S16). When the operation is started, the touch control unit 14 determines whether the pressing load detected by the load detecting unit 60 is from a specific base due to the weakening of the pressing force by the operator to the operation of the touch sensor 40. The above is changed to be smaller than the specific reference (step S31). In addition, "the change from a specific reference or more to less than the specific reference" means that the press load is changed from a state exceeding a specific reference (or a specific reference or more) to a lower than the specific reference when the press is released. The meaning of the state (or less than a specific baseline).

When it is determined in step S31 that the pressing load has changed from the specific reference or more to less than the specific reference, the tactile sensation control unit 14 controls the tactile display unit 50 so that the tactile display unit 50 displays the setting set by the application. The touch is released (step S32).

Here, the load reference for displaying the tactile sensation may be set to be the same as the above-described load reference for displaying the click sensation at the time of pressing, but is preferably set to be 50% to 80% lower than the load reference for displaying the click sensation at the time of pressing. Value. In this case, when the same position (target) is continuously pressed (that is, the button is continuously tapped), the sequentially pressing and the releasing of the pressing are well matched with the tactile display timing, and can be displayed. A sense of real operation without any sense of coordination. In other words, by setting the load of the reference for displaying the tactile sensation at the time of release to a load smaller than the reference for displaying the tactile sensation at the time of pressing, it is possible to display a tactile sensation without a sense of discomfort. At this time, by setting the load indicating the tactile sensation at the time of release to about 50% or more of the load indicating the tactile sensation at the time of pressing, the operability at the time of the continuous pressing operation can be further improved. In addition, by setting the load indicating the tactile sensation at the time of release to about 80% or less of the load indicating the tactile sensation at the time of pressing, it is possible to cope with a slight load change in the holding state at the time of the continuous pressing operation. In addition, the setting of such a load reference can also be used as information related to the display of the tactile sensation. And pre-attached to the app.

After the release of the tactile sensation is displayed in step S32, the input control unit 16 determines whether the touch sensor 40 has not sensed the contact (step S33). When the touch sensor 40 does not sense the contact, the input control unit 16 notifies the application execution unit 20 of the content whose contact has been released. Further, the application execution unit 20 supplies the display data of the image whose target is changed in accordance with the application-based setting to the display control unit 12. Upon receiving the display material, the display control unit 12 controls the display unit 30 to cause the display unit 30 to change the image of the displayed target (step S34). That is, once the contact is released from the target of the button or the like, the display control portion 12 changes the display image of the target. When the display image of the target is to be changed, for example, the selected target may be displayed as before returning to the selection, or the color or brightness of the target or the like may be changed. By such a change in display, it is preferable that the operator can visually recognize that the contact with the button or the like has been released.

Through the above actions, it can be displayed when the operator feels pressed. )", when released, "唧 ( It is a more realistic touch. Of course, the release touch can use the same driving signal as the click touch. In the present embodiment, the release touch can be set to a form different from the click touch.

As described above, in the tactile sensation display device 1 of the present embodiment, the tactile sensation corresponding to the input to each target is set for each target based on the application to which the information related to the tactile sensation is added. Therefore, the tactile sensation display device 1 can display an appropriate tactile sensation for each target by implementing some changes in the existing application. Further, in the tactile sensation display device 1 of the present embodiment, various tactile sensations such as a click sensation and a release sensation can be displayed in response to an input to a target. Therefore, the touch display device 1 can be effective The existing application resources are utilized without waste, and in addition, a real operational feeling as that obtained when operating a realistic button switch can be provided.

Further, the present invention is not limited to the embodiment described above, and various modifications and changes can be made. For example, the configuration of the hardware of the tactile display device of the present invention is not limited to the configuration of the block diagram shown in FIG. 1, and various modifications and changes are conceivable. That is, for example, a configuration shown in FIG. 6 is also conceivable: a path for notifying the position of the contact with the touch sensor 40 from the input control unit 16 to the application execution unit 20 is not provided, but is always via the touch. The control unit 14 notifies. With this configuration, it is possible to design a portion that notifies the position of the contact with the touch sensor 40 as one portion, and to reduce the required interface and the like, and to design it with ease.

Further, for example, in the above-described embodiment, the configuration including the display control unit 12, the tactile sensation control unit 14, and the input control unit 16 in the control unit 10 has been described. However, the tactile display device of the present invention is not limited to such a configuration, and various configurations can be employed depending on the requirements at the time of design. For example, by having one functional unit function as another functional unit, the other functional units may be omitted or a functional unit may be subdivided into a plurality of functional units.

Further, in the above-described embodiment, the load detecting unit can be configured based on the detection method of the contact of the touch sensor. For example, in the case of the resistive film method, the load can be detected without using a strain gauge sensor as long as the load can be detected based on a change in the output signal based on the change in resistance caused by the contact area. Alternatively, in the case of the electrostatic capacitance method, when the load can be detected based on a change in the output signal based on the change in electrostatic capacitance, the strain gauge sensor may be used instead.

In addition, the tactile display unit is constructed using any number of piezoelectric vibrators Alternatively, a transparent piezoelectric element may be provided on the entire surface of the touch sensor, and the centrifugal motor may be rotated once in one cycle of the drive signal as long as it can exhibit vibration indicating the tactile sensation. Further, when the load detecting unit and the tactile display unit are configured using a piezoelectric element, the piezoelectric element may be shared to constitute the load detecting unit and the tactile display unit.

In the above embodiment, a configuration in which the touch sensors are placed on the upper surface of the display unit is assumed. However, the tactile display device of the present invention is not necessarily required to have such a configuration, and the touch sensor and the display portion may be spaced apart from each other at different locations.

The present invention can also be effectively applied to an input device in which a touch sensor functions as a touch switch that performs an on/off operation.

Further, the touch display device of the present invention drives the piezoelectric element when the pressing load detected based on the output of the piezoelectric element satisfies the load reference for displaying the tactile sensation. Here, when the pressing load detected by the output of the piezoelectric element satisfies the load reference for displaying the tactile sensation, it may mean that the detected pressing load reaches the reference value for displaying the tactile sensation, and the detected pressing load may exceed the display tactile sensation. The reference value may also be when the reference value for displaying the tactile sensation is detected based on the output of the piezoelectric element.

Further, the "display portion" and the "touch sensor" in the description of the present embodiment may be integrated by providing a common substrate or the like with two functions of the display portion and the touch sensor. The device is constructed. As an example of the configuration of the device in which the display unit and the touch sensor are integrated, the photodiode is regularly mixed in the matrix electrode group arranged in a matrix of the liquid crystal panel. A device in which a plurality of photoelectric conversion elements are formed. The device is configured to display an image by a liquid crystal panel. On the other hand, the light of the backlight module for liquid crystal display is reflected by the front end of the pen that contacts the desired position on the surface of the input panel, and the reflection is received by the peripheral photoelectric conversion element. Light, whereby the contact position can be detected.

While the present invention has been described in its preferred embodiments, the present invention is not intended to limit the invention, and the present invention may be modified and modified without departing from the spirit and scope of the invention. The scope of protection is subject to the definition of the scope of the patent application.

1, 2, 100‧‧‧ touch display device

10, 110‧‧‧Control Department

12, 112‧‧‧ Display Control Department

14, 114‧‧‧ Touch Control Department

16‧‧‧Input Control Department

20, 120‧‧‧Application Execution Department

30, 130‧‧‧ Display Department

40, 140‧‧‧ touch sensor

40a‧‧‧Contact surface

50, 150‧‧‧Touch display department

51‧‧‧ strain gauge sensor

52‧‧‧Piezoelectric vibrator

60‧‧‧Load Detection Department

61‧‧‧ frame

62‧‧‧Insulator

63‧‧‧Upper cover

64‧‧‧Insulator

A‧‧‧ display area

S1~S4, S11~S17, S31~S34, S100~S106‧‧‧ steps

Fig. 1 is a block diagram showing a schematic configuration of a touch display device according to an embodiment of the present invention.

2A and 2B are views showing an example of a mounting structure of the touch display device shown in Fig. 1.

Fig. 3 is a view for explaining an operation of generating a user interface in the embodiment;

4 is a flow chart showing a click touch display operation of the touch display device shown in FIG. 1.

Fig. 5 is a flowchart showing a release touch display operation of the touch display device shown in Fig. 1;

FIG. 6 is a block diagram showing another example of a schematic configuration of a tactile display device according to an embodiment of the present invention.

Fig. 7 is a block diagram showing a schematic configuration of a tentacity display device assumed.

FIG. 8 is a flowchart showing the operation of the assumed tactile display device.

1‧‧‧Tactile display device

10‧‧‧Control Department

12‧‧‧Display Control Department

14‧‧‧Tactile Control Department

16‧‧‧Input Control Department

20‧‧‧Application Execution Department

30‧‧‧Display Department

40‧‧‧Touch sensor

50‧‧‧Tactile display unit

60‧‧‧Load Detection Department

61‧‧‧ frame

Claims (2)

A touch sensing device includes: a touch sensor; a touch sensing portion, causing a contact surface of the touch sensor to vibrate; a display portion; a load detecting portion, wherein the load detecting portion detects the touch a pressing load of the contact surface of the sensor; and a control unit that sets the tactile sensation displayed by the tactile display unit to the target displayed on the display unit based on the information indicating the tactile sensation supplied by the application software. In a state where the input to the target is sensed, the load detecting unit detects a pressing load that satisfies a load reference for displaying a tactile sensation, and does not receive an instruction from the tactile sensation display of the application software, and the control station The tactile sense display unit causes the tactile sense display unit to display a tactile sensation set for the target to the pressing target that presses the contact surface. A touch sensing device includes: a touch sensor; a tactile display portion that vibrates a contact surface of the touch sensor; a load detecting portion, the load detecting portion Detecting a pressing load on a contact surface of the touch sensor; and a display portion; wherein the control method of the tactile display device is based on the information indicating the tactile sensation supplied by the application software The target displayed in the portion sets the tactile sensation displayed by the tactile sensation display unit, and in a state where the input to the target is sensed, the load detecting unit detects the pressing load that satisfies the load reference for displaying the tactile sensation, Receiving an indication of a tactile display from the application software, and controlling the tactile display to make the tactile sensation The display unit displays a tactile sensation set for the target to the pressing target that presses the contact surface.