JP2021111074A - Information processing equipment, information processing methods and information processing programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device, an information processing method, and an information processing program which have a simple configuration and can determine an operating state of an input device with high accuracy.
An information processing device includes a housing maintained at a predetermined reference potential and a touch sensor having a touch surface that an operating body can contact, and the touch surface is exposed from a part of the housing. In this state, the touch sensor is housed in the housing, and when the operating body approaches the touch surface, a current value corresponding to the impedance of the path from the touch surface to the housing via the operating body is output. An information processing device that determines the operating state of an input device to be processed, based on a current value acquisition unit that acquires the current value output from the input device and the current value acquired by the current value acquisition unit. The operation state determination unit for determining the operation state of the input device of the user is provided.
[Selection diagram] Fig. 2

Description

The present invention relates to an information processing device, an information processing method, and an information processing program that accept a user's input operation via a touch sensor.

In a portable terminal device such as a smartphone, a touch panel that performs an input operation by bringing an operating body such as a finger into contact with a predetermined position on the display is used.
The mode (operation state) in which the user operates the terminal device is a two-handed operation in which the user holds the terminal device with one hand and operates with the other hand, and the user holds the terminal device with one hand and holds the terminal device. There are one-handed operation in which the operation is performed by the hand, and deferred operation in which the terminal device is placed on a pedestal or the like for operation.
In recent years, there is a problem that the display of a portable terminal device becomes large and the operability by one-handed operation deteriorates, and the terminal device determines the user's operation state (two-handed operation or one-handed operation) as described above. Devices have been developed (see, for example, Patent Documents 1 and 2).

The device of Patent Document 1 detects the contact area when a finger touches the touch panel, and determines whether the thumb touches or another finger touches based on the contact area. Then, when it is determined that the thumb is in contact, it is determined that the operation is one-handed, and when it is determined that a finger other than the thumb is in contact, it is determined that the operation is two-handed.

In the device of Patent Document 2, sensors are arranged on the back surface or the side surface of a housing provided with a touch panel, and information on the number of fingers and the contact position when the portable terminal device is held is acquired from the sensor to obtain a gripping state or a gripping state. Determine the input operation method.

Japanese Unexamined Patent Publication No. 2012-215945 Japanese Unexamined Patent Publication No. 2012-027581

However, even in the case of two-handed operation, the thumb may be brought into contact with the touch surface, and even when a finger other than the thumb is used, the pad of the finger may be pressed for the operation. In such a case, the contact area of the finger in contact with the touch surface becomes large even when the operation is performed with both hands. Further, even in one-handed operation, the tip of the thumb may be brought into contact with the touch surface, and in this case, the contact area becomes small. Therefore, it is difficult to sufficiently improve the determination accuracy in the determination of the operating state based on the contact area of the finger in contact with the touch surface as in Patent Document 1.

On the other hand, in a device such as Patent Document 2, it is necessary to separately provide a sensor in the housing, which complicates the device configuration and increases the manufacturing cost. There is also a problem that it cannot be used with an existing touch panel.

An object of the present invention is to provide an information processing device, an information processing method, and an information processing program which can determine the operating state of an input device with high accuracy with a simple configuration.

The information processing device of the present invention includes a housing maintained at a predetermined reference potential and a touch sensor having a touch surface that can be contacted by an operating body, and the touch surface is exposed from a part of the housing. When the touch sensor is housed in the housing and the operating body approaches the touch surface, a current value corresponding to the impedance of the path from the touch surface to the housing via the operating body is output. An information processing device that determines the operating state of an input device, based on a current value acquisition unit that acquires the current value output from the input device and the current value acquired by the current value acquisition unit. The operation state determination unit for determining the operation state of the input device of the user is provided.

The present invention determines the operating state of the input device such as two-handed operation, one-handed operation, and stationary operation based on the current value output from the touch sensor.
That is, the current value output from the touch sensor changes depending on the impedance value of the path from the operating body touching the touch surface to the ground. Normally, the housing of the input device is grounded by the frame ground, so that the path is an electrical path from the touch sensor to the housing. Therefore, the impedance of the path differs depending on the operating state (two-handed operation, one-handed operation, stationary operation), and as a result, the current value of the signal output when the touch surface is touched also differs depending on the operating state. In the present invention, the operating state can be determined with higher accuracy than before based on such a current value.
Further, in the present invention, it is not necessary to provide a sensor for detecting the gripping state in the housing, and the configuration can be simplified. Furthermore, the present invention can be applied to any input device that employs a capacitive touch panel, and it is possible to determine the operating state of, for example, an existing terminal device such as a smartphone or an input device. can.

The schematic diagram which shows the schematic structure of the terminal apparatus of 1st Embodiment. The block diagram which shows the schematic structure of the terminal apparatus of 1st Embodiment. The schematic diagram which shows the structure of a touch sensor. The figure which shows an example of the operation state which the operation state determination part of 1st Embodiment determines. The figure which shows the equivalent circuit formed at the electrode intersection when a finger which is an operation body is brought into contact with a touch surface. The flowchart of the information processing method by the terminal apparatus of 1st Embodiment. The figure which shows an example of the time change of the current value from the standby start time point. The figure which shows the distribution of the detection signal when the gripping hand is a right hand. The figure which shows an example of the display screen displayed on the touch panel. The figure which shows an example of the display image displayed on the touch panel at the time of one-handed operation in 1st Embodiment. The block diagram which shows the schematic structure of the terminal apparatus of 2nd Embodiment.

[First Embodiment]
Hereinafter, the first embodiment according to the present invention will be described.
FIG. 1 is a schematic diagram showing a schematic configuration of the terminal device 100 of the present embodiment, and FIG. 2 is a block diagram showing a schematic configuration of the terminal device 100 of the present embodiment.
The terminal device 100 corresponds to the information processing device of the present disclosure, and is composed of a portable terminal device such as a smartphone. Specifically, as shown in FIG. 1, the terminal device 100 includes a housing 1, a touch panel 2, and a control circuit unit 3.

[Configuration of housing 1]
The housing 1 is formed in, for example, a thin box shape, and has a display surface 11 on which a touch panel 2 is provided, a back surface 12 on the side opposite to the display surface 11, and a side surface 13 connecting between the display surface 11 and the back surface 12. Be prepared.
The housing 1 houses a touch panel 2, a control circuit unit 3, a battery (not shown), and the like.
Further, the housing 1 is maintained at a predetermined reference potential (GND potential) by being connected to the control circuit unit 3.

[Structure of touch panel 2]
The touch panel 2 is a sensor in which a display module 21 and a touch sensor 22 are integrally combined, and constitutes the input device of the present disclosure together with the housing 1. The display module 21 is an image display device for displaying an image, and is composed of, for example, a liquid crystal panel, an organic EL panel, or the like. The touch sensor 22 is, for example, a sensor laminated on the display module 21, and is composed of a transparent electrode such as ITO.
Although an example in which the touch sensor 22 is stacked on the display module 21 is shown, for example, a touch panel in which the function as a touch sensor is incorporated in the display module 21 may be used.

The touch sensor 22 has a touch surface 220 that the operating body can contact, and the touch surface 220 is housed in the housing 1 so as to be exposed from the display surface 11.
The touch sensor 22 employs a mutual capacitance method among the projected capacitance methods as the touch detection method, and as shown in FIG. 2, a sensor having an electrode structure arranged along the touch surface 220. It has an array 221 and a transmission unit 222 that transmits a drive signal Sa to the sensor array 221 and a reception unit 223 that receives the drive signal Sa via the sensor array 221.

FIG. 3 is a schematic view showing the structure of the touch sensor 22.
As shown in FIG. 3, the sensor array 221 includes a plurality of transmission electrodes Tx (x = 1,2,3 ... n) and a plurality of reception electrodes Ry (y = 1,2,3 ... m). The transmission electrode Tx is a transparent electrode that is longitudinal along the X direction, and a plurality of transmission electrodes Tx are arranged along the Y direction. The receiving electrodes Ry are transparent electrodes that are elongated along the Y direction, and a plurality of receiving electrodes Ry are arranged along the X direction. The electrode intersection Pi where the transmitting electrode Tx and the receiving electrode Ry intersect corresponds to the unit region of the touch surface 220, and a capacitance C ₀ is formed at each electrode intersection Pi.

The transmission unit 222 transmits the drive signal Sa, which is an AC signal, to the plurality of transmission electrodes Tx in order.
The receiving unit 223 receives the driving signal Sa in order from the plurality of receiving electrodes Ry while the driving signal Sa is being transmitted to any of the transmitting electrodes Tx. As a result, the receiving unit 223 sequentially receives the drive signal Sa transmitted from the transmitting electrode Tx to the receiving electrode Ry via each electrode intersection Pi.

Here, when the operating body H such as a finger is close to the electrode intersection Pi, the capacitance corresponding to the operating body H _{interrupts the capacitance C 0} and is coupled to the drive signal Sa from the electrode intersection Pi via the operating body H. Part of the current flows to the ground. As a result, the current flowing through the receiving electrode Ry decreases at the electrode intersection Pi at the position where the operating body H is close to each other. The receiving unit 223 outputs a detection signal Sd according to the amount of current reduction (outflow current value escaped to the operating body H) generated at the receiving electrode Ry. That is, the touch sensor 22 can detect the electrode intersection Pi at which the outflow current value is equal to or higher than a predetermined threshold value as a position where the operating body H is close to the operating body H.

[Structure of control circuit unit 3]
The control circuit unit 3 includes a plurality of circuit boards such as a panel drive board that controls the drive of the touch panel 2, a motherboard composed of a storage circuit such as a memory, a logic circuit, and the like. These circuit boards are connected to the housing 1 at one point or multiple points to maintain the housing 1 at a predetermined reference potential (GND potential).

Further, the control circuit unit 3 has a general hardware configuration for forming a computer, and includes a storage unit 31, a control unit 32, and the like, as shown in FIG.
The storage unit 31 is a recording device composed of a storage circuit such as a memory or a recording medium such as a hard disk. An information processing program (software) for controlling the terminal device 100 is recorded in the storage unit 31.

The control unit 32 is composed of an arithmetic circuit such as a CPU (Central Processing Unit) and a recording circuit such as a RAM (Random Access Memory). The control unit 32 expands the information processing program recorded in the storage unit 31 or the like into the RAM, and executes various processes in cooperation with the program expanded in the RAM. Then, the control unit 32 reads and executes the information processing program recorded in the storage unit 31, so that the current value acquisition unit 321 and the touch detection unit 322, the standby determination unit 323, the parameter calculation unit 324, and the operation state determination unit 325 , And functions as a display changing unit 326 and the like.

The current value acquisition unit 321 acquires the detection signal Sd output from the reception unit 223 at a predetermined cycle. As described above, the detection signal Sd is a signal corresponding to the amount of current decrease when the operating body H approaches at the electrode intersection Pi, and the current value acquisition unit 321 receives the outflow current value (outflow current value) based on the detection signal Sd. Hereinafter, the current value Ia) is acquired.
The touch detection unit 322 detects the touch point of the operating body H with respect to the touch panel 2 based on the detection signal Sd acquired by the current value acquisition unit 321 and calculates the coordinate information of the touch point. For example, the touch detection unit 322 detects the electrode intersection Pi at which the current value Ia based on the detection signal Sd is equal to or greater than the threshold value as the position coordinates of the touch point. Further, when a plurality of electrode intersection Pis whose detection signal Sd is equal to or higher than the threshold value are adjacent to each other, the touch detection unit 322 detects the center of gravity of the region where these electrode intersection Pis are located as the coordinate information of the touch points.

The standby determination unit 323 detects the time when the current value Ia starts to increase (the time when the current value Ia becomes equal to or higher than the threshold value) as the standby start time Pws, and measures the elapsed time from the standby start time Pws to wait. It is determined whether or not a predetermined waiting time Tw has elapsed from the start time Pws.
When the standby determination unit 323 determines that the standby time Tw has elapsed, the parameter calculation unit 324 calculates a determination parameter regarding the change in the current value Ia during the elapsed period of the standby time Tw. The details of the determination parameters will be described later.

The operation state determination unit 325 determines the operation state of the terminal device 100 based on the determination parameters calculated by the parameter calculation unit 324. In the present embodiment, as the operation state determined by the operation state determination unit 325, a two-handed operation, a one-handed operation, and a stationary operation are exemplified.
FIG. 4 is a diagram showing an example of an operation state determined by the operation state determination unit 325 of the present embodiment, in which (A) is an example of two-handed operation, (B) is an example of one-handed operation, and (C) is a stationary operation. Is an example of.
As shown in FIG. 4A, the two-handed operation is an operation state in which the user holds the housing 1 with one hand and performs an input operation such as a tap operation on the touch surface 220 with the other hand.
As shown in FIG. 4B, the one-handed operation is an operation state in which the user holds the housing 1 with one hand and performs an input operation with one hand holding the housing 1.
As shown in FIG. 4C, the stationary operation is an operation state in which the user performs an input operation while the housing 1 is placed on a table or the like.

The display change unit 326 changes the display position of the image to be displayed on the touch panel 2 according to the operation state determined by the operation state determination unit 325.

[Changes in detection signal depending on operating body H and operating state]
The operating body H is an object to be brought into contact with the touch surface 220 of the touch panel 2, and is, for example, a human finger or a conducting instrument (for example, a touch pen) held by a human hand. In the present embodiment, the operating body H will be described below as a human finger.

FIG. 5 is a diagram showing an equivalent circuit formed at the electrode intersection Pi when the finger, which is the operating body H, is brought into contact with the touch surface 220. In FIG. 5, Z _{0 to} Z ₄ , Z _i are impedances of substances existing around the electrode intersection Pi, and correspond to an equivalent circuit including a resistor and a capacitor. Z _i is the impedance of the path from the finger in contact with the touch surface 220 to the frame ground, and varies depending on the operating state.
For example, in the two-handed operation as shown in FIG. 4A, one hand (for example, the left hand) holding the housing 1 comes into contact with the back surface 12 and the side surface 13 of the housing 1, and the fingers of the other hand (for example, the right hand). Touches the touch surface 220. Therefore, it is a path of _{impedance Z 5} from the index finger of the right hand in contact with the touch surface 220 to the housing 1 via the right hand, the right arm, the torso, the left arm, and the left hand.
In the one-handed operation as shown in FIG. 4B, one hand holding the housing 1 (for example, the right hand) comes into contact with the back surface 12 or the side surface 13 of the housing 1, and the finger (for example, the thumb) of the right hand touches the touch surface 220. Contact. Therefore, it becomes a path of _{impedance Z 6} from the thumb of the right hand in contact with the touch surface 220 to the housing 1 through the palm of the right hand.
In the stationary operation as shown in FIG. 4C, the human body does not touch the housing 1, and the finger of the human body, for example, the right hand, touches only the touch surface 220. Therefore, it becomes a path of _{impedance Z 7} from the finger of the right hand to the housing 1 through the human body such as the right hand, the right arm, and the torso, and further through peripheral objects such as a chair and a table.
Impedance Z _i becomes larger as the distance of the path is longer, and becomes larger as an object having a lower conductivity intervenes, so that Z ₆ <Z ₅ <Z ₇ .

As shown in FIG. 5, the transmitting electrode Tx is _{grounded via the impedance Z 4} , the receiving electrode Ry is _{grounded via the impedance Z 3} , and the transmitting electrode Tx and the receiving electrode Ry are grounded through the impedance Z. They are connected to each other by _0.
In a state in which the operation tool H does not touch the electrode intersections Pi, electrically coupled to the receiving electrode Ry and the transmission electrode Tx is only impedance Z _0, the receiving electrode Ry from the transmission electrode Tx, the impedance Z ₀ The drive signal Sa is transmitted via.

On the other hand, when the operating body H (finger) is close to a certain electrode intersection Pi, the transmitting electrode Tx and the receiving electrode Ry are connected via impedances Z1 and Z2 such as the touch surface 220 and the surface of the finger, and are operated. It is grounding through the impedance Z _i of the path according to the state.
As described above, since Z ₆ <Z ₅ <Z ₇ , the amount of current flowing out to the ground through the operating body H increases in the order of one-handed operation, two-handed operation, and stationary operation. That is, the detection signal Sd of the electrode intersection Pi output from the receiving unit 223 increases in the order of one-handed operation, two-handed operation, and stationary operation.

[Information processing method]
FIG. 6 is a flowchart of an information processing method by the terminal device 100.
An information processing method by the terminal device 100 will be described with reference to FIG. During the operation of the terminal device 100, the current value acquisition unit 321 acquires the detection signal Sd output from the reception unit 223 at a predetermined cycle, calculates the current value Ia based on the detection signal Sd, and stores it in the storage unit 31. It is assumed that the current value acquisition step is always performed.

First, the standby determination unit 323 determines whether or not the standby start time point Pws is detected by determining whether or not the current value Ia is equal to or higher than the threshold value Th (step S1). Here, the threshold value Th is set so that the time point at which the current value Ia starts to increase in response to the start of contact of the operating body H with respect to the touch panel 2 can be detected as the standby start time point Pws while preventing erroneous determination due to noise.

In the case of Yes in step S1, the standby determination unit 323 stores the standby start time point Pws in the storage unit 31. Further, the standby determination unit 323 measures the elapsed time from the standby start time Pws and determines whether or not a predetermined standby time Tw has elapsed from the standby start time Pws (step S2). The standby time Tw is set to, for example, an arbitrary time in the range of 60 to 100 msec, preferably 60 msec.
When the standby determination unit 323 determines that the standby time Tw has elapsed (step S2; in the case of Yes), the standby determination unit 323 stores the end time point of the standby time Tw as the standby end time point Pwe in the storage unit 31. After that, the flow proceeds to step S4.

The touch duration of a general single tap operation is about 150 to 200 msec. Therefore, no matter what kind of touch operation the operating body H performs, there is a high possibility that the standby time Tw (60 to 100 msec) elapses within the touch duration. However, there is a possibility that the touch ends earlier than the waiting time Tw elapses.

Therefore, when the standby time Tw has not elapsed from the standby start time Pws (No in step S2), the standby determination unit 323 determines whether or not the touch of the operating body H is completed (step S3). .. When the touch of the operating body H is completed (step S3; in the case of Yes), the standby determination unit 323 stores the time when it is determined that the touch is completed in the storage unit 31 as the standby end time Pwe. After that, the flow proceeds to step S4.
On the other hand, when the touch of the operating body H is not completed (step S3; in the case of No), the flow returns to step S2.
When the current value Ia changes from the threshold value Th or more to less than the threshold value Th, the standby determination unit 323 can determine that the touch of the operating body H is completed.

In step S4, the parameter calculation unit 324 calculates the determination parameter regarding the current value Ia in the period from the standby start time point Pws to the standby end time point Pwe.
FIG. 7 is a diagram showing an example of a time change of the current value Ia from the standby start time point Pws, and shows an example in which the standby time Tw is 60 msec. In FIG. 7, A1 shows a change in the current value Ia in the case of two-handed operation, A2 shows a change in the current value Ia in the case of one-handed operation, and A3 shows a change in the current value Ia in the case of stationary operation. ..
As shown in FIG. 7, in the proximity stage where the operating body H is close to the touch panel 2 (for example, when 40 msec elapses from the standby start time Pws), the difference in the current value Ia depending on the operating state is unlikely to occur, but the touch panel 2 At the initial stage of contact (from the time when 40 msec has elapsed to the time when 60 msec has elapsed) when the operating body H begins to contact, the state of change in the current value Ia depending on the operating state is clearly different. Specifically, in the case of one-handed operation, the amount of change in the current value Ia is the largest, and the amount of change in the current value Ia decreases in the order of the two-handed operation and the stationary operation.
Therefore, the parameter calculation unit 324 sets the maximum value of the current value Ia in the standby time Tw, the maximum rate of change of the current value Ia in the standby time Tw, or the standby time Tw as a determination parameter capable of determining the operation state. Calculate at least one of the integral values of the current value Ia.

After that, the operation state determination unit 325 determines the operation state based on the determination parameter calculated by the parameter calculation unit 324 (step S5; operation state determination step).
For this purpose, for example, correlation information indicating the relationship between each operation state and the numerical range of the determination parameter is recorded in advance in the storage unit 31. Then, the operation state determination unit 325 determines that the operation state in the numerical range corresponding to the determination parameter calculated in step S4 is the current user operation state based on the correlation information stored in the storage unit 31. do.

Further, when it is determined in step S5 that the operation state is a one-handed operation, the operation state determination unit 325 further determines that the hand (grasping hand) in which the user is holding the terminal device 100 is the right hand or the left hand. Determine if there is.
FIG. 8 is a diagram showing the distribution of the signal values of the detection signal Sd when the gripping hand is the right hand, and FIG. 8 (A) is a tap between the center and the left end of the touch panel 2 when the gripping hand is the right hand. (B) is the detection signal Sd when tapping between the center of the touch panel 2 and the right end when the gripping hand is the right hand, and (C) is the detection signal Sd when the gripping hand is the right hand. It is a detection signal Sd when the vicinity of the right end of the touch panel 2 is tapped in the case of the right hand. In FIG. 8, black indicates a position where the signal value of the detection signal Sd is low, and white indicates a position where the signal value of the detection signal Sd is high.
For example, when the gripping hand is the right hand, the movable range of the thumb, which is the operating body H, is centered on the right side of the touch panel 2, so that the tap positions are concentrated on the right side of the touch panel 2.
Further, when the left side of the touch panel 2 is tapped, the angle between the thumb and the touch surface 220 becomes shallow, so that the contact area where the thumb contacts during the tap operation becomes large as shown in FIG. 8 (A). On the other hand, when tapping the right end of the touch panel 2, it is necessary to stand the angle close to 90 ° with respect to the thumb and the touch surface 220, and as shown in FIG. 8C, the contact area that the thumb contacts during the tap operation is large. It becomes smaller.
In the example of FIG. 8, the gripping hand is the right hand, but in the case of the left hand, the distribution of the detection signal Sd is reversed left and right from that of FIG. The same applies when the terminal device 100 is gripped sideways.
Therefore, when the operation state is a one-handed operation, the operation state determination unit 325 can determine the gripping hand based on the position coordinates of the tap point and the distribution of the detection signal Sd centered on the tap point.

After that, the control unit 32 determines whether or not to perform a predetermined process corresponding to the coordinate information of the touch point detected by the touch detection unit 322 (step S6).
FIG. 9 is an example of a display screen displayed on the touch panel 2.
The screen example shown in FIG. 9 is, for example, a home screen 50 displayed on a smartphone or the like. For example, a plurality of icons 51 are arranged side by side in the vertical direction and the horizontal direction. Usually, these icons 51 are arranged side by side at a preset position (first operation area) at equal intervals. Each icon 51 is an operation target image for invoking a predetermined application, and a corresponding application is associated with each icon 51.
When the control unit 32 determines that the touch point is on the image of the icon 51, it determines Yes in step S6 and executes a predetermined process such as starting the application corresponding to the tapped icon 51 (step). S7).

If No is determined in step S6, it means that the touch point is not on the icon 51. In this case, the display changing unit 326 changes the display position of the image to be displayed on the touch panel 2 according to the operation state determined in step S5 (step S8). That is, in the present embodiment, the icon 51 corresponding to the specific application is not tapped, and the blank portion is tapped to change the screen display according to the operation state.

FIG. 10 is a diagram showing an example of a display image displayed on the touch panel 2 during one-handed operation.
For example, when the display changing unit 326 determines in step S5 that the operation is one-handed and the gripping hand is determined to be the right hand, the display changing unit 326 changes the home screen 50 shown in FIG. 9 as shown in FIG. That is, each icon 51 arranged on the home screen 50 is arranged closer to the right side of the gripping hand as a whole.

Further, when the screen change at the time of one-handed operation as shown in FIG. 10 is performed and the operation state is determined to be the two-handed operation in step S5, the display changing unit 326 is in the normal manner as shown in FIG. Change to screen display.
Further, the display changing unit 326 displays only a specific display screen when the operation state is determined to be the stationary state in step S5. For example, even when a push notification display request is received, the display change unit 326 does not display the push notification at the time of the stationary operation, but displays the received push notification when the display is changed to the display for other operation states. Perform processing such as making it. That is, in the case of a stationary operation, the user often wants to always display a predetermined image such as a navigation application or a moving image playback application. Therefore, by not performing other display such as push notification, the user's work is not interrupted and the convenience can be improved.

Although FIGS. 9 and 10 are examples of the home screen 50, the display change unit 326 makes the above display change even when another image corresponding to a predetermined application is displayed. You may. For example, when a character input application for displaying a character input keyboard is started on the touch panel 2, the operation state is one-handed operation, and the gripping hand is the right hand, the keyboard is moved from the center position of the touch panel 2. It may be displayed on the right side by a predetermined distance.

[Action and effect of this embodiment]
The terminal device 100 of the present embodiment includes a housing 1 maintained at a reference potential (GND potential) and a touch sensor 22 having a touch surface 220 to which an operating body H such as a finger can contact. The touch sensor 22 is housed in the housing 1 with the touch surface 220 exposed from the display surface 11 of the housing 1, and when the operating body H approaches the touch surface 220, the operating body H is moved from the touch surface 220. via outputs a detection signal Sd that corresponds to the impedance Z _i of the path leading to the housing 1 of the frame ground.
Then, the control unit 32 of the terminal device 100 reads and executes the information processing program, thereby acquiring the current value based on the detection signal Sd output from the touch sensor 22 of the touch panel 2, and the current value acquisition unit 321. It functions as an operation state determination unit 325 that determines the operation state of the user's terminal device 100 based on the current value acquired by the current value acquisition unit 321.

Impedance Z _i of the path from the touch surface 220 in contact with the operation tool H (finger) to the frame ground of the housing 1 is different respectively for each operation state, corresponding to the detection signal Sd at the time of operating the touch panel 2 The current value Ia to be generated is also a different value. Therefore, the operation state determination unit 325 can determine the operation state based on the current value Ia.
Here, the difference in impedance of the path from the operating body H to the housing 1 is large due to the difference in the operating state, and as shown in FIG. 7, the difference in the current value Ia depending on the operating state appears remarkably. Therefore, in the present embodiment, the operating state can be determined with higher accuracy than the case where the operating state is determined using, for example, the contact area of the finger with the touch surface 220. Further, in order to detect the gripping state of the housing 1 of the user, it is not necessary to provide a sensor other than the touch panel 2 in the housing 1, and the operating state can be determined with a simple configuration. For example, by introducing an information processing program into an existing terminal device 100 such as a smartphone, it is possible to easily determine the operation state of the user.

In the terminal device 100 of the present embodiment, the operation state determination unit 325 includes correlation information indicating the relationship between the determination parameter based on the current value Ia and each operation state, and the current value Ia acquired by the current value acquisition unit 321. To determine the operating status using.
By using such correlation information, the operation state determination unit 325 can easily determine the operation state of the user. Further, there are individual differences in the impedance of the path from the operating body H to the housing 1 via the human body. Therefore, by using the correlation information tailored to each individual, for example, the operation state can be determined more accurately than in the case where the operation state is determined only by a preset threshold value.

The control unit 32 of the terminal device 100 of the present embodiment further functions as a display change unit 326. The display change unit 326 changes the display image to be displayed on the touch panel 2 according to the operation state determined by the operation state determination unit 325.
For example, the display changing unit 326 displays the icons 51 that can be operated by the user evenly when the operation state is determined to be a two-handed operation, and displays the icons 51 when it is determined to be a one-handed operation. The display is brought closer to the peripheral edge of the touch panel 2.
As a result, each icon 51 is arranged at a position where the user can easily perform the tap operation according to the operation state, and the convenience when the user operates the terminal device 100 can be improved.

The operation state determination unit 325 further determines whether the gripping hand is the right hand or the left hand when one-handed operation is determined, and the display change unit 326 determines the icon 51 when it is determined to be the right hand. The icon 51 is moved to the right of the touch panel 2 and the icon 51 is moved to the left of the touch panel 2 when it is determined that the hand is the left hand.
Therefore, since the arrangement of the icons 51 is changed according to the hand in which the user is operating the terminal device 100, the convenience can be further improved.

[Second Embodiment]
Next, the second embodiment will be described. In the first embodiment described above, the operation state determination unit 325 determines the operation state for the determination parameter based on the correlation information stored in advance in the storage unit 31. On the other hand, in the present embodiment, the correlation information for determining the operation state is learned by machine learning.

FIG. 11 is a block diagram showing a schematic configuration of the terminal device 100A of the second embodiment.
As shown in FIG. 11, the control unit 32 of the terminal device 100A of the present embodiment has the current value acquisition unit 321 and the touch detection unit 322, the standby determination unit 323, the parameter calculation unit 324, and the operating state, as in the first embodiment. In addition to functioning as a determination unit 325 and a display change unit 326, it also functions as an operation state learning unit 327 (learning unit).
The operation state learning unit 327 uses the determination parameters detected when the user performs an input operation in each operation state as learning data, and machine-learns the operation state of the user with respect to the determination parameters.

Specifically, the operation state learning unit 327 collects learning data, for example, when the terminal device 100A is operated for the first time by the user after the terminal device 100A is shipped, or when the user requests the operation state resetting process. get.
For example, the operation state learning unit 327 causes the touch panel 2 to display a notification requesting a plurality of tap operations by one-handed operation. When the user holds the terminal device 100A with one hand and taps the screen with the thumb of the one hand, the processes of steps S1 to S4 of the first embodiment are performed, and the process is based on the detection signal Sd output from the touch panel 2. Calculate the judgment parameters. As a result, the operation state learning unit 327 acquires a plurality of determination parameters for one-handed operation as learning data and stores (stores) them in the storage unit 31.
As a one-handed operation, learning data may be acquired when the gripping hand is the right hand and when the gripping hand is the left hand.

Similarly, the operation state learning unit 327 causes the touch panel 2 to display a notification requesting a plurality of tap operations by the two-handed operation, and acquires a plurality of learning data for the two-handed operation.
Further, the operation state learning unit 327 causes the touch panel 2 to display a notification requesting a plurality of tap operations in the deferred operation, and acquires a plurality of learning data for the deferred operation.
In the acquisition of learning data for the stationary operation, the operation state learning unit 327 may display a notification to change the pedestal on which the terminal device 100A is placed to various materials. For example, an operation in which the terminal device 100A is mounted on a metal (conductor) pedestal, an operation in which the terminal device 100A is mounted on a resin pedestal, and an operation in which the terminal device 100A is mounted on a wooden pedestal are requested. You may.
Alternatively, the operation state learning unit 327 requests to input the frequency of the deferred operation, and when the frequency input by the user is equal to or higher than the predetermined frequency, the learning data of the deferred operation using the pedestal frequently used by the user. May be obtained.

In addition, there may be a plurality of types of determination parameters acquired as learning data. For example, the maximum value of the current value Ia, the maximum rate of change of the current value Ia within the standby time Tw, and the integrated value of the current value Ia at the standby time Tw may be acquired as determination parameters. Further, here, the determination parameter based on the current value Ia calculated from the detection signal Sd is used as the learning data, but even if the current value Ia of the detection signal Sd, which is the raw data from the touch panel 2, is acquired as the learning data. good.

Then, the operation state learning unit 327 generates correlation information indicating the relationship between the determination parameter and the operation state based on each acquired learning data.
In the generation of correlation information, for example, the operation state learning unit 327 clusters the acquired learning data into a plurality of classes (operation states) according to the determination parameters, and correlates the operation information corresponding to each class. Generate information.

Alternatively, the operation state learning unit 327 may generate a state determination model in which the determination parameter or the current value Ia of the detection signal Sd is used as the input value and the operation state is used as the output value, using the learning data.
When a state determination model that outputs an operation state is generated by inputting the detection signal Sd, the operation state determination unit 325 is sampled at a predetermined cycle without calculating the determination parameters in steps S1 to S4. The current value Ia of the detection signal Sd is input to the state determination model. As a result, the operation state is output from the state determination model, and the operation state determination unit 325 can easily determine the operation state of the user.
When a state determination model that outputs an operation state is generated by inputting a determination parameter, the processes of steps S1 to S4 are executed as in the first embodiment, and the state determination model is determined in step S5. By inputting the parameters, the operation status can be determined.

[Action and effect of this embodiment]
In the terminal device 100A of the present embodiment, the current value acquired by the current value acquisition unit 321 is accumulated and stored in the storage unit 31, and the operation state learning unit 327 is based on the accumulated current value Ia by machine learning. , Generates correlation information indicating the relationship between the operating state and the current value Ia. For example, a state determination model is generated as correlation information, in which the current value Ia or the determination parameter based on the detection signal Sd is input and the operation state is output.
As a result, the correlation information corresponding to each user can be generated, and the determination of the operation state by the operation state determination unit 325 can be optimized for the user who operates the terminal device 100A. As a result, the operating state can be determined more accurately.

[Modification example]
The present invention is not limited to the above-described embodiment, but also includes the modifications shown below to the extent that the object of the present invention can be achieved.

[Modification 1]
In the second embodiment, the correct answer value of the operation state, the detection signal Sd, and the judgment parameter are acquired as learning data (supervised data) by inputting the tap operation in each operation state, but based on the unsupervised data. Therefore, correlation information such as a state determination model may be generated.
That is, the detection signal Sd at the time of tapping and the determination parameter are stored and stored in the storage unit 31 as the operation history when the user taps the touch panel 2 for the operation of the terminal device 100A. Then, correlation information or a state determination model is generated based on a large number of unsupervised data determination parameters.
The impedance Zi of the path passing through the human body to the frame ground does not fluctuate greatly for each user, and the relationship of _{Z 6} <Z ₅ <Z _{7 is satisfied not only for the user.} Therefore, the magnitude relationship of the judgment parameters in each operation state does not fluctuate greatly for each user, and even for unsupervised data as described above, a large number of judgment parameters are clustered for each operation state. It is possible to learn what the range of judgment parameters is.

Further, when the correlation information is generated based on the learning data, the distribution of the detection signal Sd may be used as the learning data in addition to the determination parameter and the detection signal Sd. In this case, in addition to the determination parameter indicating the change in the current value based on the detection signal Sd, the correlation information for determining the operation state can be generated by taking into account the contact area between the finger and the touch surface, which is the operating body H. can.

[Modification 2]
In the first embodiment and the second embodiment, an example in which the operation state is determined based on the determination parameters is shown, but as the operation state, the hand holding the terminal devices 100, 100A is on either the left or right side, or on the touch panel 2. In addition to the operation method such that the hand in contact is left or right, the type of the operating body H in contact with the touch panel 2 may be further determined.
For example, the impedance of the path from the touch point through the human body to the frame ground is different depending on the two-handed operation with fingers, the two-handed operation with nails, the two-handed operation with a touch pen, and the like. Therefore, in addition to the two-handed operation, the one-handed operation, and the stationary operation, the operating state of the type (finger, claw, touch pen, etc.) of the operating body H may also be determined.

[Modification 3]
In the first embodiment and the second embodiment, the current value corresponding to the detection signal Sd sampled periodically or each detection signal Sd output from a plurality of electrode intersection points Pi centered on a plurality of touch points. The total current value of the above was used as the judgment value, and at least one of the maximum value, the maximum amount of change, and the integrated value of the judgment value in the predetermined standby period was used as the judgment parameter. On the other hand, the peak value of the determination value in a predetermined period may be used as the determination parameter.

[Modification example 4]
In the first embodiment, in step S8, when the display changing unit 326 changes the display position of the image according to the operation state, the operation target image such as the icon 51 is placed on the peripheral edge of the touch panel 2 for one-handed operation. An example of gathering is shown.
On the other hand, the display changing unit 326 may change the display image so that the operation target image such as the icon 51 is aggregated in the center of the screen.

Further, as in the second embodiment, the operation state learning unit 327 determines the determination parameter when the tap operation is performed by the user's one-handed operation, and the touch area centered on the touch point (contact area where the finger touches). Acquire as training data. In this case, the average area when the user touches the touch surface 220 is calculated based on the touch area, and the touch position where the average area is obtained is specified as the optimum position for the user's tap operation. Then, the display changing unit 326 may change the display image so as to aggregate the operation target images such as the icon 51 around the optimum position of the tap operation.

[Modification 5]
In the first embodiment and the second embodiment, examples are shown in which the terminal devices 100 and 100A, which are the information processing devices of the present disclosure, are portable terminal devices such as smartphones and also function as input devices. On the other hand, an input device including a touch sensor and a housing may be connected to a stationary type information processing device such as a personal computer so as to be able to communicate by wire or wirelessly.
Further, in this case, the input device may not be provided with the display module 21.

1 ... Housing, 2 ... Touch panel, 3 ... Control circuit unit, 21 ... Display module, 22 ... Touch sensor, 31 ... Storage unit, 32 ... Control unit, 50 ... Home screen, 51 ... Icon (operation target image), 100 , 100A ... Terminal device (information processing device), 321 ... Current value acquisition unit, 322 ... Touch detection unit, 323 ... Standby determination unit, 324 ... Parameter calculation unit, 325 ... Operation state determination unit, 326 ... Display change unit, 327 ... Operation state learning unit (learning unit), H ... Operating body, Ia ... Current value.

Claims (9)

The housing includes a housing maintained at a predetermined reference potential and a touch sensor having a touch surface that can be contacted by an operating body, and the touch sensor is exposed to a part of the housing. When the operating body is close to the touch surface, the operating state of the input device that outputs a current value according to the impedance of the path from the touch surface to the housing via the operating body is determined. It is an information processing device
A current value acquisition unit that acquires the current value output from the input device, and a current value acquisition unit.
An operation state determination unit that determines the operation state of the user's input device based on the current value acquired by the current value acquisition unit.
An information processing device characterized by being equipped with. In the information processing apparatus according to claim 1,
The operation state determination unit is characterized in that it determines the operation state based on the correlation information indicating the relationship between the operation state and the current value and the current value acquired by the current value acquisition unit. Information processing device. In the information processing apparatus according to claim 2,
A storage unit that accumulates and stores the current value acquired by the current value acquisition unit, and a storage unit.
A learning unit that machine-learns the correlation information based on the accumulated current value,
An information processing device characterized by being equipped with. In the information processing apparatus according to any one of claims 1 to 3.
The input device includes a touch panel in which a display module for displaying an image and the touch sensor are combined.
An information processing device including a display changing unit that changes a display image to be displayed on the touch panel according to the operating state determined by the operating state determining unit. In the information processing apparatus according to claim 4,
In the operation state, the operation state determination unit causes the user to hold the housing with one hand and bring a conductor held by the other hand or the other hand into contact with the touch surface as the operation body. A two-handed operation, which is an operation, or a one-handed operation in which the user grips the housing with one hand and brings a predetermined finger of the one hand into contact with the touch surface as the operating body. Judge if you went,
The display changing unit displays an operation target image that can be operated by the user in a preset first operation area when it is determined to be a two-handed operation, and when it is determined to be a one-handed operation, the display changing unit is said to be the one-handed operation. An information processing device characterized in that an operation target image is displayed closer to the peripheral edge of the touch panel than the first operation area. In the information processing apparatus according to claim 5,
When the one-handed operation is determined, the operation state determination unit further determines whether the one hand is the right hand or the left hand.
The display changing unit is characterized in that the operation target image is moved to the right of the touch panel when it is determined to be the right hand, and the operation target image is moved to the left of the touch panel when it is determined to be the left hand. Information processing device. The information processing apparatus according to any one of claims 1 to 6.
An information processing device further comprising the input device. A housing maintained at a predetermined reference potential and a touch panel having a touch surface with which an operating body can be contacted are provided, and the touch surface is housed in the housing in a state of being exposed from a part of the housing. Information that a computer determines the operating state of an input device that outputs a current value according to the difference in impedance of a path from the touch surface to the housing via the operating body when the operating body comes into contact with the touch surface. It ’s a processing method,
The computer includes a current value acquisition unit and an operation state determination unit.
A current value acquisition step in which the current value acquisition unit acquires the current value output from the input device, and
An operation state determination step in which the operation state determination unit determines the operation state of the user's input device based on the current value acquired in the current value acquisition step.
An information processing method characterized by carrying out. An information processing program that is read and executed by a computer.
An information processing program characterized in that the computer functions as the information processing device according to any one of claims 1 to 7.

Images