JP2014135668A - Portable terminal device - Google Patents

Abstract

One of a plurality of motion detection units is selected without depending on the installation status of a mobile terminal device.
A mobile terminal device having a first motion detection unit and a second motion detection unit arranged at different positions of a housing, the posture detection unit detecting a posture of the housing, and the posture A portable terminal device comprising: a selection unit that determines which one of the first motion detection unit and the second motion detection unit is to be operated based on an attitude detected by the detection unit .
[Selection] Figure 4B

Description

  The present invention relates to a mobile terminal device, and more particularly to a mobile terminal device capable of automatically determining which of imaging units arranged at different positions is operated.

There is a camera as an input device in a portable terminal device. In addition, a gesture recognition function that recognizes a gesture imaged by a camera and performs an operation is becoming common.
For example, in Patent Document 1, when an incoming call is received, the main camera or the sub camera is activated, the subject is imaged to recognize a facial expression, and a different response is returned based on the recognition result. Whether to activate the main camera or the sub camera is determined by the open / close state of the terminal.
Further, Non-Patent Document 1 has a function of recognizing a hand gesture from image information input from a front camera and performing a touchless operation on an incoming call, and performs a reception operation by the gesture. And switch to a speakerphone.

JP 2011-205475 A

Pantech VEGA LTE, press release: http://www.pantech.co.kr/en/board/reportBoardList.do

However, if the technology of Patent Document 1 is applied to a portable device such as a smartphone that does not have an opening / closing mechanism, when one main camera and one sub camera are arranged on the front and back surfaces, one of them is activated. It cannot be determined by the open / closed state or the like. Therefore, there is a problem that the camera cannot be activated in advance and the input image cannot be processed quickly.
An object of the present invention is to select one of a plurality of motion detection units to be activated according to the installation status of a mobile terminal device.

According to one aspect of the present invention, a mobile terminal device having a first motion detection unit and a second motion detection unit arranged at different positions of a housing, the posture detection detecting a posture of the housing. And a selection unit that determines which of the first motion detection unit and the second motion detection unit to operate based on the posture detected by the posture detection unit. A portable terminal device is provided.
Since a selection unit that determines which of the first motion detection unit and the second motion detection unit to operate based on the posture detected by the posture detection unit is provided, how the mobile terminal device is The motion detection unit to be used can be automatically selected.

  The first motion detection unit and the second motion detection unit are respectively disposed on a front surface side and a back surface side of the housing, the posture detection unit is an acceleration sensor, and the selection unit is the acceleration When the acceleration on the surface side measured by the sensor is larger than a certain value (for example, zero), the first motion detector is operated, and the acceleration measured by the acceleration sensor is a certain value (for example, zero). If smaller, the second motion detector is selected to operate. Start-up preparation such as activating only the selected motion detection unit, for example, the imaging unit, turning off the unselected motion detection unit, or performing focusing processing in the selected motion detection unit, for example, the imaging unit, etc. You may make it do.

  In addition, when the posture detection unit cannot detect, for example, it may be set so that either the first motion detection unit or the second motion detection unit is operated. For example, when the acceleration on the surface side measured by the acceleration sensor is substantially zero, it is not possible to determine which motion detection unit should be operated. In such a case, it may be determined in advance which operation detection unit is to be operated.

  And a proximity sensor that detects a nearby obstacle, wherein the selection unit uses the proximity sensor to detect an obstacle when the acceleration on the surface side measured by the acceleration sensor is substantially zero. It is characterized in that the operation detection unit that is not present is operated.

  Moreover, this invention is a portable terminal device which has the 1st motion detection part and the 2nd motion detection part which are arrange | positioned on the different surface of a housing | casing, Comprising: It has a proximity sensor which detects the obstruction which adjoins The portable terminal device is characterized in that the proximity detection sensor is used to operate the motion detection unit having no obstacle.

  Further, a posture detection unit that detects the posture of the housing, and which of the first motion detection unit and the second motion detection unit to operate based on the posture detected by the posture detection unit. And a selection unit for determination. The motion detection unit is an imaging unit or a gesture recognition device.

  According to another aspect of the present invention, there is provided a control method for a portable terminal device having a first motion detection unit and a second motion detection unit that are arranged on different surfaces of a housing, wherein the posture of the housing is changed. A posture detection step to detect, and a selection step to determine which of the first motion detection unit and the second motion detection unit to operate based on the posture detected by the posture detection step. There is provided a control method characterized by comprising:

Also, a control method in a mobile terminal device having a first motion detection unit and a second motion detection unit arranged on different surfaces of a housing, wherein a fault is detected using a proximity sensor that detects a nearby obstacle There is provided a control method characterized by including a step of operating the motion detection unit having no object.
The present invention may be a program for causing a computer to execute the control method described above.

  According to the present invention, since it is possible to automatically determine how the mobile terminal device is placed, it is desired to enable one of a plurality of motion detection units without depending on the installation status of the mobile terminal device. You can choose which way.

It is a functional block diagram which shows the example of 1 structure of the portable terminal device by the 1st Embodiment of this invention. In the housing | casing of a portable terminal device, it is a figure which shows the example in case the main display part and the 1st imaging part are provided in the surface side of the housing | casing, and the 2nd imaging part is provided in the back side. It is a figure which shows the example of arrangement | positioning of the portable terminal device by this Embodiment. It is a flowchart figure which shows an example of the process until gesture recognition start among the gesture recognition processes by this Embodiment. It is a flowchart figure which shows an example of the process flow until the selection process of a gesture recognition device among the gesture recognition processes by this Embodiment. It is a functional block diagram which shows the example of 1 structure of the portable terminal device by the 2nd Embodiment of this invention. It is a figure which shows the example of an external appearance structure of the gesture recognition device using a proximity sensor. It is a flowchart figure which shows the flow of the selection process of the gesture recognition device by this Embodiment. It is a functional block diagram which shows the example of 1 structure of the portable terminal device by the 3rd Embodiment of this invention.

In this specification, the motion detection unit includes an imaging device that captures an image such as a camera and a gesture recognition device that recognizes a gesture, and has a function of detecting an external operation of the mobile terminal device.
Hereinafter, a mobile terminal device according to an embodiment of the present invention will be described in detail with reference to the drawings. Here, a smartphone will be described as an example, but a mobile terminal device having other forms may be used.

(First embodiment)
FIG. 1 is a functional block diagram showing a configuration example of the mobile terminal device according to the first embodiment of the present invention.
As shown in FIG. 1, the mobile terminal device A according to the present embodiment is provided in a housing, for example, a posture detection unit 1 that detects the posture of the housing, and a posture of the housing detected by the posture detection unit 1. A posture determination unit 3 that determines the posture of the housing based on the sensing result regarding the control unit, a control unit (CPU) 5 that controls the entire mobile terminal device A, and a first motion detection that is arranged on different surfaces of the housing And a second motion detection unit (here, the gesture recognition device 2 using an image sensor is taken as an example) 7b. And a display unit (display) 15 provided on at least one surface of the housing, and a storage unit 17. The control unit (CPU) 5 includes a gesture recognition device selection / activation unit 5a.

In the present embodiment, an example is shown in which an acceleration sensor is used as the posture detection unit 1 and a gesture recognition device that performs motion detection using an imaging unit (camera) is used as the motion detection unit. It is also possible to use a gesture recognition device, a sound wave gesture recognition device, or the like.
When using a gesture recognition device, one or a plurality of gesture recognition devices that are actually used in gesture recognition are selected from a plurality of gesture recognition devices (cameras, etc.) according to the output value of the posture determination unit. To do.
Further, a general acceleration sensor measures acceleration based on the magnitude of the force applied to the housing, not the speed change or moving distance per time.
The acceleration sensor will be briefly described below.

Based on Hook's law (f = kx), which is the principle of spring, and Newton's equation of motion (f = ma), from the equation kx = ma, acceleration a is expressed as a = x * (k / m) Can be sought. In other words, since it can be calculated only by multiplying the absolute displacement amount by a constant, it is possible to measure the sum of both gravitational acceleration and other acceleration.
Conversely, an acceleration sensor that can measure gravitational acceleration cannot distinguish between gravitational acceleration and other accelerations. If a component other than gravity is extracted, it is necessary to remove a static component corresponding to the gravitational acceleration from the measured value by performing calibration or the like.

  There are various means to actually act as a spring, such as a piezoelectric element and a strain gauge, but the basic principle of measurement is the same, and the acceleration is nonlinear depending on the [variation x constant] or the variation. Even when it changes, the acceleration can be obtained by a = f (x). In the present invention, it is assumed that an acceleration sensor capable of measuring acceleration including gravitational acceleration is used. However, according to the above method, different acceleration sensors can be used.

  In the following, as shown in FIGS. 2A and 2B, in the casing of the mobile terminal device A, as the gesture recognition device, the first imaging unit 7a is provided on the front side of the casing, and the second is provided on the back side. A case where the imaging unit 7b is provided will be described as an example. A main display unit 15 is also provided on the front side of the housing. The acceleration sensor that functions as the posture detection unit 1 is attached in the casing or on the surface of the casing. A sub display unit may be provided on the back side.

  The X, Y, and Z axes shown in FIG. 2 can be arbitrarily defined. However, in this embodiment, for convenience of explanation, the Z axis is the normal direction of the display screen, and the X and Y axes are respectively The direction along the top and bottom of the housing and the direction along the side surface are used. As the output value of the acceleration sensor, acceleration in the X, Y, and Z axis directions is obtained, and the output value in the Z axis direction is a positive value when the surface is upward (opposite to the direction in which gravitational acceleration acts). Shall be obtained.

  FIG. 3 is a diagram illustrating an arrangement example of the mobile terminal device A according to the present embodiment. FIG. 4A is a flowchart showing an example of the process up to the start of gesture recognition in the gesture recognition process according to the present embodiment, and FIG. 4B is a flowchart showing an example of the process flow until the selection process of the gesture recognition device. FIG.

As shown in FIG. 4A, the process is started (Start: Step S1). When a gesture recognition start request is received, for example, when a call is received, the posture detection unit 1 and the posture determination unit 3 perform posture determination in Step S3. Processing is performed. Next, in step S4, the gesture recognition device selection / activation unit 5a selects one of the gesture recognition devices 7a and 7b, and then in step S5, selects one of the selected gesture recognition devices 7a and 7b. Start. It is also possible to prepare for activation such as activating only the selected one, turning off the one not selected, or performing a focusing process on the selected gesture recognition device.
Thereby, in step S6, the gesture recognition process can be started, and the process up to the start ends (end: step S7).

  FIG. 4B is a diagram showing details of the flow of the posture determination process corresponding to step S3 in FIG. 4A and the gesture recognition device selection process corresponding to step S4. In step S3-1 of step S3, x, y, and z are acquired as output values of the acceleration sensor. Next, in step S3-2, it is determined whether z ≧ k (k is a constant). Here, k is a constant value and is generally 0. That is, when z is larger than a certain value (Yes in step S3-2), as described above, the output value in the Z-axis direction is when the surface is upward (opposite to the direction in which gravitational acceleration acts). Can be determined to be a flat position with the surface of the casing up as shown in FIG. 3A (step S3-3). Therefore, in step S4-1, the gesture recognition device selection / activation unit 5a determines that the front gesture recognition device 7a is valid, and activates the gesture recognition device 7a.

On the other hand, in the case of No in step S3-2, the process proceeds to step S3-4, and as shown in FIG. 3D, it is determined that the back surface of the housing is facing upward, and the process proceeds to step S4-2. Proceeding, it is determined that the gesture recognition device 7b on the back side is valid, and the gesture recognition device 7b is activated.
In addition, k is typically 0, but may not necessarily be 0. For example, if z can take -9.8 <= z <= 9.8, it is good also as k = -1.0.

As described above, according to the present embodiment, it is possible to determine the posture of the mobile terminal device by the acceleration sensor and control the gesture recognition device to be appropriately selected and operated.
Accordingly, in a mobile terminal device such as a smartphone that does not have an opening / closing mechanism, such as a gesture recognition device, for example, the posture detection device automatically determines how the gesture recognition device is placed and uses it. The gesture can be recognized by selecting automatically, so that gestures can be recognized and operated regardless of whether the gesture recognition device is placed face up or face down. There is an advantage of being able to do it. That is, in the state where the front camera is facing down, the gesture cannot be recognized, and the problem that the function does not operate can be solved.

(Second Embodiment)
FIG. 5 is a functional block diagram showing a configuration example of the mobile terminal device according to the second embodiment of the present invention. The difference between FIG. 5 and FIG. 1 is that the proximity sensor 1a is used instead of the acceleration sensor as the posture detection unit.
Proximity sensors include those that use changes in magnetic fields, those that detect changes in capacitance, those that use the return time of reflected ultrasonic waves, and those that use the intensity of reflected infrared light. . In the present embodiment, it is assumed that a proximity sensor that can detect whether or not there is an obstacle that causes the gesture recognition device to recognize a gesture within a distance of 0 to several cm from the sensor. ing.

The proximity sensor can detect whether it is non-contact, that is, whether or not the detection target is approaching without being actually touched, by replacing it with an electrical signal. For that purpose, energy such as magnetic field, electromagnetic wave, light, or sound wave is emitted from the sensor, and the reflected substance or change in energy is captured.
The main methods of the proximity sensor include an induction type, a capacitance type, an ultrasonic type, an electromagnetic wave type, and an infrared type.
The “inductive type” is a proximity sensor that uses a change in a magnetic field. The sensor device has a built-in detection coil that generates a high-frequency magnetic field by passing a current. When the metal to be detected approaches the magnetic field, an induced current flows by electromagnetic induction. Since the impedance of the detection coil changes due to this induced current, the approach of the object can be known by detecting the change. This type is mainly used for proximity sensors targeting metal.

In the “capacitance type”, an electrode constitutes a capacitor (capacitor, storing and discharging electric energy) with an insulator interposed therebetween. For example, since the capacitance of the capacitor (the amount of electric energy in the capacitor) changes as the human body to be checked approaches, the principle is used that it is possible to recognize whether the object has approached by detecting the change.
The “ultrasonic type” receives ultrasonic waves from an ultrasonic transducer and receives reflected waves from an object. By measuring the change in the return time of the reflected wave, it is possible to observe how close or far the object is.
In the same way, it is a method to measure the change in the return time of the reflected wave or measure the distance from the object by measuring the interference of the reflected wave, and it is `` electromagnetic wave type '' that uses electromagnetic waves instead of ultrasonic waves It is.

FIG. 6 is a diagram illustrating an example of an external configuration of a gesture recognition apparatus using a proximity sensor, and FIG. 7 is a flowchart illustrating a process flow.
When the process is started (Start), the value of the proximity sensor is acquired in step S3a-1. In step S3a-2, it is determined whether or not there is an obstacle ahead based on the value of the proximity sensor. In the case of No, it is determined in step S3a-3 that the surface is facing upward, and step S4- 1. It is determined that the gesture recognition device on the front side is effective (FIG. 6A). On the other hand, in the case of Yes in step S3a-2, the process proceeds to step S3a-4, where it is determined that the back surface is facing upward, and in step S4-2, it is determined that the gesture recognition device on the back surface side is valid ( FIG. 6 (b)).

As described above, according to the present embodiment, it is possible to determine the posture of the mobile terminal device using the proximity sensor and control the gesture recognition device to be appropriately selected and operated.
Therefore, in a portable terminal device such as a smartphone that does not have an opening / closing mechanism, such as a gesture recognition device, it automatically determines how the gesture recognition device is placed by using a proximity sensor as a posture detection unit, for example. Since the gesture recognition device can be automatically selected to recognize the gesture, the gesture recognition device can be recognized regardless of whether it is placed face up or face down. There is an advantage that the operation can be performed.

(Third embodiment)
In this Embodiment, it is a general smart phone which has the acceleration sensor 21a and the proximity sensor 21b as an attitude | position detection part, as shown in FIG.
When only an acceleration sensor is provided as in the first embodiment, any gesture recognition device (here, a camera) is operated in the state as shown in FIGS. 3B and 3C. It may be unclear how to control. Therefore, in the present embodiment, the posture detection unit includes the acceleration sensor 21a and the proximity sensor 21b, and the camera selection unit 25a has a substantial acceleration on the surface side in the housing measured by the acceleration sensor 21a. In the case of zero, the camera with no obstacle is operated using the proximity sensor 21b.
As a result, an appropriate camera can be activated and operated regardless of the value of z.
For example, in a gesture recognition device that is a portable device that does not have an opening / closing mechanism or the like and includes a plurality of imaging means, it is possible to appropriately select an imaging means that actually performs imaging so that the gesture or the like can be recognized.

In the above-described embodiment, the configuration and the like illustrated in the accompanying drawings are not limited to these, and can be appropriately changed within a range in which the effect of the present invention is exhibited. In addition, various modifications can be made without departing from the scope of the object of the present invention.
Each component of the present invention can be arbitrarily selected, and an invention having a selected configuration is also included in the present invention.

In addition, a program for realizing the functions described in the present embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute processing of each unit. May be performed. The “computer system” here includes an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

  The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already recorded in a computer system. At least a part of the functions may be realized by hardware such as an integrated circuit.

(Appendix)
The following disclosure of the present invention is included.
(1) A mobile terminal device having a first motion detection unit and a second motion detection unit arranged at different positions of the housing, the posture detection unit detecting the posture of the housing, and the posture detection A portable terminal device comprising: a selection unit that determines which of the first motion detection unit and the second motion detection unit is to be operated based on the posture detected by the unit.
Since it is possible to automatically determine how the mobile terminal device is placed, it is possible to select one of the plurality of motion detection units that is to be enabled without depending on the installation status of the mobile terminal device. it can.

(2-1) The first motion detection unit and the second motion detection unit are respectively disposed on a front surface side and a back surface side of a housing, and the posture detection unit is an acceleration sensor, and the selection And the unit operates the first motion detection unit when the acceleration on the surface side measured by the acceleration sensor is larger than a certain value, and the acceleration on the back side measured by the acceleration sensor is constant. The mobile terminal device according to (1), wherein when the value is smaller than the value, the second motion detection unit is selected to operate.
(2-2) It is set so that either the first motion detection unit or the second detection unit is operated in a case where the posture detection unit cannot detect (1). ) Mobile terminal device.
Either one of the operation detection units can be operated to determine the installation status of the apparatus.

(3) Furthermore, it has a proximity sensor for detecting a nearby obstacle, and the selection unit uses the proximity sensor when the acceleration on the surface side measured by the acceleration sensor is substantially zero. The mobile terminal device according to (2-1) or (2-2), wherein the motion detection unit without an obstacle is operated.
Even in a case where detection is not possible only by sensing from the acceleration sensor (acceleration in the Z-axis direction is 0), it is possible to select one of the plurality of motion detection units to be made effective.

(4) A portable terminal device having a first motion detection unit and a second motion detection unit arranged on different surfaces of the housing, the proximity sensor detecting a nearby obstacle, the proximity A mobile terminal device that operates a motion detection unit having no obstacle using a sensor.
Since it is possible to automatically determine how the mobile terminal device is placed, it is possible to select one of the plurality of motion detection units that is to be enabled without depending on the installation status of the mobile terminal device. it can.

(5) The mobile terminal device according to any one of (1) to (4), wherein the motion detection unit is an imaging unit or a gesture recognition device.
(6) The mobile terminal device according to any one of (1) to (5), wherein the motion detection unit is an imaging unit or a gesture recognition device.

(7) A control method in a mobile terminal device having a first motion detection unit and a second motion detection unit arranged on different surfaces of the housing, the posture detection step for detecting the posture of the housing; And a selection step for determining which of the first motion detection unit and the second motion detection unit is to be operated based on the posture detected by the posture detection step. Method.
(8) A control method in a portable terminal device having a first motion detection unit and a second motion detection unit arranged on different surfaces of a housing, using a proximity sensor for detecting an obstacle nearby A control method comprising the step of operating the motion detection unit having no obstacle.
(9) A program for causing a computer to execute the control method according to (7) or (8).

  The present invention is applicable to a mobile terminal device.

A ... mobile terminal device (gesture recognition device), 1 ... posture detection unit, 3 ... posture determination unit, 5 ... control unit (CPU), 5a ... gesture recognition device selection / activation unit, 7a ... first motion detection unit ( Gesture recognition device), 7b, second motion detection unit (gesture recognition device), 15 ... display unit (display), 17 ... storage unit.

Claims (5)

A mobile terminal device having a first motion detection unit and a second motion detection unit arranged at different positions of a housing,
An attitude detection unit for detecting the attitude of the housing;
A portable terminal comprising: a selection unit that determines which of the first motion detection unit and the second motion detection unit is to be operated based on the posture detected by the posture detection unit apparatus. The first motion detection unit and the second motion detection unit are respectively disposed on the front surface side and the back surface side of the housing,
The posture detection unit is an acceleration sensor,
The selection unit includes:
When the acceleration on the surface side measured by the acceleration sensor is larger than a certain value, the first motion detector is operated,
2. The mobile terminal device according to claim 1, wherein when the acceleration measured by the acceleration sensor is smaller than a certain value, the second motion detection unit is selected to operate. Furthermore, it has a proximity sensor that detects nearby obstacles,
The selection unit includes:
3. The motion detection unit having no obstacle is operated using the proximity sensor when the acceleration on the surface side measured by the acceleration sensor is substantially zero. Mobile terminal device. A mobile terminal device having a first motion detection unit and a second motion detection unit arranged on different surfaces of a housing,
It has a proximity sensor that detects nearby obstacles,
A portable terminal device, wherein the proximity sensor is used to operate the motion detection unit having no obstacle. The mobile terminal device according to claim 1, wherein the motion detection unit is an imaging unit or a gesture recognition device.

Images