KR102836408B1 - Electronic device configured to detect gestures and controlling method thereof - Google Patents

Abstract

An electronic device according to various embodiments may include a gesture sensor, a communication circuit, and at least one processor, wherein the at least one processor may be configured to detect a gesture of a user of the electronic device through the gesture sensor, perform a first operation of the electronic device corresponding to the detected gesture when a size of the detected gesture falls within a first range, and perform a second operation corresponding to the detected gesture when a size of the detected gesture falls within a second range different from the first range, and transmit a first signal through the communication circuit to control at least one of one or more external electronic devices. Various other embodiments are possible.

Description

{ELECTRONIC DEVICE CONFIGURED TO DETECT GESTURES AND CONTROLLING METHOD THEREOF}

Various embodiments relate to an electronic device configured to recognize gestures and a method of controlling the electronic device configured to recognize gestures.

Modern consumers want to conveniently use various electronic devices whenever and wherever they want. In addition, due to the recent IoT (Internet of Things), various sensors, home appliances, and communication devices in our lives are being networked together.

An electronic device can recognize a gesture made by a user while the user is away from the electronic device. The electronic device that recognizes the user's gesture can determine the type of gesture and perform various functions according to the type of gesture determined. For example, the patent publication under publication number 10-2015-0099430 discloses a wearable electronic device that detects a user's gesture and performs various functions according to the detected gesture.

When utilizing the gesture recognition function of an electronic device, only the electronic device performing gesture recognition may be controlled in response to the recognized gesture. In this case, when a user wants to control multiple electronic devices within an IoT environment, there may be inconvenience in that the user must move in front of each electronic device to be controlled, activate the gesture sensor of the electronic device, and perform the gesture. In addition, there may be inconvenience in that the user cannot control an electronic device that does not have a gesture recognition function among the electronic devices within the IoT environment using gestures.

According to an electronic device set to recognize a gesture according to various embodiments and a method of controlling the electronic device set to recognize a gesture, the electronic device can perform an operation corresponding to the detected gesture by itself according to a size range of the detected gesture, or transmit a signal for controlling another device to perform an operation corresponding to the detected gesture.

In addition, according to the electronic device set to recognize a gesture according to various embodiments and the control method of the electronic device set to recognize a gesture, the electronic device can perform different actions in response to the detected gesture depending on the number of other electronic devices in the environment in which the electronic device is placed.

According to various embodiments, an electronic device may include a gesture sensor, a communication circuit, and at least one processor, wherein the at least one processor may be configured to transmit a first signal through the communication circuit to control at least one of one or more external electronic devices to detect a gesture of a user of the electronic device through the gesture sensor, perform a first operation of the electronic device corresponding to the detected gesture when a size of the detected gesture falls within a first range, and perform a second operation corresponding to the detected gesture when a size of the detected gesture falls within a second range different from the first range.

According to various embodiments, an electronic device includes a gesture sensor, a communication circuit, and at least one processor, wherein the at least one processor detects a gesture of a user of the electronic device through the gesture sensor, verifies the number of external electronic devices other than the electronic device, and if the number of the external electronic devices is 1, if the size of the detected gesture falls within a first range, performs a first operation of the electronic device corresponding to the detected gesture, and if the size of the detected gesture falls within a second range different from the first range, transmits a first signal through the communication circuit to control the external electronic device to perform a second operation corresponding to the gesture, and if the number of the external electronic devices is 2 or more, if the size of the detected gesture falls within the first range, determines a first device to perform an operation according to the gesture among the electronic device and the 2 or more external electronic devices in response to the detected gesture, and if the size of the detected gesture falls within the second range different from the first range and the first device is the electronic device, performs the first operation of the electronic device corresponding to the gesture, and if the size of the detected gesture falls within the second range different from the first range and the first device is the electronic device, performs the first operation of the electronic device corresponding to the gesture, and if the size of the detected gesture falls within the second range different from the first range, If the first device is within the range and is one of the two or more external electronic devices, a second signal may be transmitted through the communication circuit to control the first device to perform a third operation corresponding to the detected gesture.

According to various embodiments, a method performed in an electronic device may include an operation of detecting a gesture of a user of the electronic device, an operation of performing a first operation of the electronic device corresponding to the detected gesture when a size of the detected gesture falls within a first range, and an operation of transmitting a first signal through the communication circuit to control at least one of the one or more external electronic devices to perform a second operation corresponding to the detected gesture when the size of the detected gesture falls within a second range different from the first range.

According to various embodiments, an electronic device set to recognize a gesture and a method of controlling the electronic device set to recognize a gesture are provided. Accordingly, the electronic device itself can perform an operation corresponding to the detected gesture according to a size range of the detected gesture, or transmit a signal for controlling another device to perform an operation corresponding to the detected gesture. Accordingly, a user can control an electronic device other than the electronic device detecting the gesture by using the electronic device detecting the gesture. Accordingly, in a scenario where a user wants to control multiple electronic devices in an IoT environment, the inconvenience of having to move in front of each electronic device to be controlled, activate a gesture sensor of the electronic device, and perform a gesture can be reduced. In addition, even if the electronic device that the user wants to control does not have a gesture recognition function, the electronic device that the user wants to control can be controlled by using a gesture for the electronic device detecting the gesture.

In addition, according to an electronic device set to recognize a gesture according to various embodiments and a method of controlling an electronic device set to recognize a gesture, an electronic device located far from a user can be controlled using a large gesture, and an electronic device detecting a gesture located close to the user can be controlled using a small gesture. Accordingly, gesture rules that can be intuitively recognized by a user can be provided.

FIG. 1 illustrates a block diagram of an electronic device within a network environment according to various embodiments.
FIG. 2 illustrates a block diagram of an electronic device configured to recognize gestures according to various embodiments.
Figure 3 illustrates an IoT environment according to various embodiments.
FIG. 4 illustrates a flowchart for explaining the operation of an electronic device according to various embodiments.
Figures 5a to 5d illustrate examples of gestures according to various embodiments.
FIG. 5e illustrates an example screen showing a correspondence between gestures and actions according to various embodiments.
FIG. 6 illustrates a flowchart for explaining the operation of an electronic device according to various embodiments.
FIG. 7 illustrates an example of a screen for selecting an external electronic device to be controlled by gesture, according to various embodiments.
FIG. 8a illustrates a flowchart for explaining the operation of an electronic device according to various embodiments.
Figure 8b illustrates an IoT environment according to various embodiments.
FIG. 9 illustrates a flowchart for explaining the operation of an electronic device according to various embodiments.
FIG. 10 illustrates an example of a screen for selecting an electronic device to recognize a gesture, according to various embodiments.
FIG. 11 illustrates a flowchart for explaining the operation of an electronic device according to various embodiments.

FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) through a first network (198) (e.g., a short-range wireless communication network), or may communicate with the electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input device (150), an audio output device (155), a display device (160), an audio module (170), a sensor module (176), an interface (177), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the display device (160) or the camera module (180)), or may include one or more other components. In some embodiments, some of these components may be implemented as a single integrated circuit. For example, a sensor module (176) (e.g., a fingerprint sensor, an iris sensor, or a light sensor) may be implemented embedded in a display device (160) (e.g., a display).

The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may load a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) into the volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in the nonvolatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor), and a secondary processor (123) (e.g., a graphic processing unit, an image signal processor, a sensor hub processor, or a communication processor) that may operate independently or together therewith. Additionally or alternatively, the auxiliary processor (123) may be configured to use less power than the main processor (121), or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121), or as a part thereof.

The auxiliary processor (123) may control at least a portion of functions or states associated with at least one of the components of the electronic device (101) (e.g., the display device (160), the sensor module (176), or the communication module (190)), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)).

The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or nonvolatile memory (134).

The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

The input device (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., processor (120)) from an external source (e.g., a user) of the electronic device (101). The input device (150) can include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

The audio output device (155) can output an audio signal to the outside of the electronic device (101). The audio output device (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive an incoming call. According to one embodiment, the receiver can be implemented separately from the speaker or as a part thereof.

The display device (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display device (160) can include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display device (160) can include touch circuitry configured to detect a touch, or a sensor circuitry configured to measure a strength of a force generated by the touch (e.g., a pressure sensor).

The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can obtain sound through an input device (150), or output sound through an audio output device (155), or an external electronic device (e.g., an electronic device (102)) (e.g., a speaker or a headphone) directly or wirelessly connected to the electronic device (101).

The sensor module (176) can detect an operating state (e.g., power or temperature) of the electronic device (101) or an external environmental state (e.g., user state) and generate an electric signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., the electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that a user can perceive through a tactile or kinesthetic sense. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) can include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (190) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., the electronic device (102), the electronic device (104), or the server (108)), and performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., the application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). A corresponding communication module among these communication modules can communicate with an external electronic device via a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify and authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module can include one antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) can include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), can be selected from the plurality of antennas by, for example, the communication module (190). A signal or power can be transmitted or received between the communication module (190) and the external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., an RFIC) can be additionally formed as a part of the antenna module (197).

At least some of the above components may be connected to each other and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

In one embodiment, a command or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the electronic devices (102, 104) may be the same or a different type of device as the electronic device (101). In one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to perform at least a part of the function or service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

FIG. 2 illustrates a block diagram of an electronic device configured to recognize gestures according to various embodiments. The electronic device (200) may include at least one processor (210), a communication circuit (220), and a gesture sensor (230). According to various embodiments, the electronic device (200) may be the electronic device (101) described above with reference to FIG. 1. According to various embodiments, the electronic device (200) may be a tablet PC, a wearable device, earphones, or a smart home appliance. In addition to the examples listed in this specification, the type of the electronic device (200) is not limited as long as it includes the processor (210), the communication circuit (220), and the gesture sensor (230).

According to various embodiments, at least one processor (210) may be the processor (120) described above with reference to FIG. 1. The at least one processor (210) may receive an electrical signal from the gesture sensor (230) and detect a gesture of the user based on the received electrical signal. The at least one processor (210) detecting the gesture of the user may include determining a size of the gesture and/or a type of the gesture. The at least one processor (210) may determine a range within which the size of the gesture falls and/or a type of which the type of the gesture falls based on the electrical signal received from the gesture sensor (230).

According to various embodiments, at least one processor (210) may control at least one of an output device (e.g., a display device (160) or an audio output device (155)) of the electronic device (200) or the communication circuit (220) based on the size of the identified gesture and/or the type of the gesture. For example, if the size of the identified gesture falls within a first range that is a relatively small range, the at least one processor (210) may control the output device of the electronic device (200) to perform an operation corresponding to the type of the identified gesture and output a result of performing the operation. In another example, if the size of the identified gesture falls within a range larger than the first range, the at least one processor (210) may control the communication circuit (220) to transmit a signal to control one of the one or more external electronic devices to perform an operation corresponding to the type of the identified gesture.

According to various embodiments, when it is determined that the size of the gesture confirmed by at least one processor (210) falls within a range greater than the first range, the communication circuit (220) may transmit a signal for controlling the external electronic device according to a control signal from the at least one processor (210). According to various embodiments, the communication circuit (220) may transmit a signal for controlling the external electronic device to the electronic device (200) and a server communicating with the external electronic device. In another example, the communication circuit (220) may transmit a signal for controlling the external electronic device to the external electronic device.

According to various embodiments, the gesture sensor (230) may transmit signals of various frequencies, detect a reflected signal reflected from a body part (e.g., a hand, an arm, or a leg) of a user that is away from the electronic device (200), and transmit an electric signal based on the reflected wave to the processor (210). The processor (210) may detect a gesture using the body part of the user based on the electric signal received from the gesture sensor (230).

According to various embodiments, the gesture sensor (230) may use at least one of a 60 GHz frequency signal, an ultra wide band (UWB) signal, a WiFi signal, or a 24 GHz frequency signal transmitted through a WiFi wireless antenna of the electronic device (200). In addition, a gesture sensor (230) using various methods capable of detecting gestures may be used, and the principle by which the gesture sensor (230) detects gestures is not limited to the examples described above.

FIG. 3 illustrates an IoT environment including various electronic devices according to various embodiments. The IoT environment (300) may include an electronic device (310), a server (320), and external electronic devices (331, 332). The electronic device (310) may detect a gesture. As described later in FIG. 4, the electronic device (310) that detects a gesture may perform a first operation corresponding to the gesture detected by the electronic device (310) according to a size range of the detected gesture, or may transmit a first signal to control the external electronic device (331, 332) to perform a second operation corresponding to the detected gesture.

According to various embodiments, the electronic device (310) may transmit a first signal to the server (320) for controlling an external electronic device (331, 332). In this case, the first signal may indicate a size value of a detected gesture, a size range to which the detected gesture belongs, or an identifier of at least one of the external electronic devices (331, 332) that will perform a second operation. According to various embodiments, the first signal may include information about a type of the detected gesture or a second operation corresponding to the detected gesture.

According to various embodiments, the first signal may include indoor positioning information of the electronic device (310). In this case, the server (320) may control the external electronic device (331, 332) whose indoor location is closest to the electronic device (310) among the external electronic devices (331, 332) to perform a second operation corresponding to the detected gesture by referring to the indoor positioning information of the external electronic devices (331, 332).

When the electronic device (310) transmits a first signal to the server (320), the server (320) may transmit a signal to at least one of the external electronic devices (331, 332) to perform a second operation corresponding to the detected gesture based on the first signal received from the electronic device (310). According to various embodiments, the signal transmitted from the server (320) to at least one of the external electronic devices (331, 332) may include information regarding the type of the detected gesture, and at least one of the external electronic devices (331, 332) that has received the information regarding the type of the detected gesture from the server (320) may perform the second operation corresponding to the detected gesture by referring to the correspondence between the type of gesture and the operation stored in the memory of at least one of the external electronic devices (331, 332).

According to various other embodiments, a signal transmitted from the server (320) to at least one of the external electronic devices (331, 332) may indicate a second operation to be performed by at least one of the external electronic devices (331, 332). In this case, the server (320) may generate a signal transmitted to at least one of the external electronic devices (331, 332) by referring to a correspondence between a type of gesture and an operation stored in a memory of the server (320). At least one of the external electronic devices (331, 332) that receives a signal indicating the second operation from the server (320) may perform the second operation.

Alternatively, according to various embodiments, the electronic device (310) may directly transmit the first signal to at least one of the external electronic devices (331, 332). In this case, the electronic device (310) may identify the second operation to be performed by at least one of the external electronic devices (331, 332) by referring to the correspondence between the type of gesture and the operation stored in the memory of the electronic device (310). In addition, the electronic device (310) may identify the external electronic device (331) that will perform the second operation among the external electronic devices (331, 332). For example, the electronic device (310) may identify the external electronic device (331) that is located closest to the electronic device (310) among the external electronic devices (331, 332) as the external electronic device (331) that will perform the second operation. Here, indoor positioning information of the electronic device (310) and the external electronic devices (331, 332) may be used, and the electronic device (310) may query the server (320) to check the indoor positioning information of the external electronic devices (331, 332). Alternatively, the electronic device (310) may check the external electronic device (331) to perform the second operation based on a user's selection of one of the external electronic devices (331, 332) made on the electronic device (310). The first signal transmitted from the electronic device (310) to at least one of the external electronic devices (331, 332) may indicate an identifier of the external electronic device (331) to perform the second operation and the second operation. In this case, the external electronic device (331) that receives the first signal from the electronic device (310) may perform the second operation.

In another example, the electronic device (310) may directly transmit a first signal to at least one of the external electronic devices (331, 332), and the first signal may only indicate a type of gesture detected by the electronic device (310) and may not indicate a second operation corresponding to the type of the detected gesture. In this case, the external electronic device (331) that receives the first signal that only indicates a type of the detected gesture and does not indicate a second operation corresponding to the type of the detected gesture may identify a second operation corresponding to the type of the detected gesture by referring to a correspondence between the types of gestures and operations stored in the memory of the external electronic device (331), and may perform the second operation.

In FIG. 3, the electronic device (310) is a mobile terminal, the external electronic device (331) is a TV, and the external electronic device (332) is shown as a light, which is purely exemplary, and the types of the electronic device (310) and the external electronic devices (331, 332) are not limited to the types shown in FIG. 3.

FIG. 4 illustrates a flowchart (400) for explaining the operation of an electronic device according to various embodiments. In operation 410, at least one processor (210) of the electronic device (200) can detect a gesture through a gesture sensor (230). In operation 420, at least one processor (210) of the electronic device (200) can determine whether a size of the detected gesture falls within a first range. An exemplary meaning of the first range will be described with reference to FIGS. 5A to 5D.

FIGS. 5A to 5D illustrate examples of gestures according to various embodiments. The finger gesture (500a) of FIG. 5A is a gesture of moving one finger (501a) of the fingers (501a, 502a, 503a, 504a, 505a) of the right hand from right to left. In addition to the example illustrated in FIG. 5A, the finger gesture may also include a gesture of moving a finger of the left hand, and may encompass various gestures of moving at least one joint of the fingers.

The wrist gesture (500b) of Fig. 5b is a gesture in which the right wrist (510b) joint moves from right to left, resulting in the right hand (520b) moving from right to left. In addition to the example illustrated in Fig. 5b, the wrist gesture may also include a gesture in which the left wrist is moved, and may encompass various gestures in which the wrist joint is moved.

The elbow gesture (500c) of Fig. 5c is a gesture in which the right elbow (510c) moves from right to left, resulting in the right wrist (520c) and right hand (530c) moving from right to left. In addition to the example illustrated in Fig. 5c, the elbow gesture may also include a gesture in which the left elbow is moved, and may encompass various gestures in which the elbow is moved.

The shoulder gesture (500d) of Fig. 5d is a gesture in which the left shoulder (510d) moves from left to right, resulting in the left elbow (520d), left wrist (530d), and left hand (540d) moving from left to right. In addition to the example illustrated in Fig. 5d, the shoulder gesture may also include a gesture in which the right shoulder is moved, and may encompass various gestures in which the shoulder is moved.

A gesture in which at least one of an arm or a hand moves can be decomposed into at least one combination of the finger gesture, the wrist gesture, the elbow gesture, or the shoulder gesture described above with reference to FIGS. 5A to 5D. In the present specification, the size of a gesture is defined based on at least one of the components of the finger gesture, the wrist gesture, the elbow gesture, or the shoulder gesture that constitutes the gesture, which corresponds to a body part with the largest range of motion, or at least one of the lengths of the trajectory of the body part drawn by the gesture. Here, the range of motion means the maximum length of the trajectory of the body part drawn by the gesture, and among the finger gesture, the wrist gesture, the elbow gesture, and the shoulder gesture, the shoulder gesture has the largest range of motion, and the elbow gesture, the wrist gesture, and the finger gesture have smaller ranges of motion in that order, and the finger gesture has the smallest range of motion.

According to various embodiments, the first range may be defined by which of the gesture components has the greatest range of motion. For example, the first range may encompass gestures whose components have the greatest range of motion, such as a finger gesture component. In this case, a gesture having at least one of a wrist gesture, an elbow gesture, or a shoulder gesture component is not included in the first range. In another example, the first range may encompass gestures whose components have the greatest range of motion, such as a wrist gesture component. In this case, gestures having only at least one of a finger gesture or a wrist gesture component are included in the first range, and gestures having at least one of an elbow gesture or a shoulder gesture component are not included in the first range.

According to various embodiments, the first range may be defined by considering both the component with the largest range of motion among the components of the gesture and the length of the trajectory of the body part drawn by the gesture. For example, the first range may be defined as gestures satisfying the first condition and the second condition falling within the first range, wherein the first condition may be that the component with the largest range of motion among the components of the gesture is a finger gesture component, and the second condition may be that the length of the trajectory of the fingertip drawn by the gesture is less than or equal to a predetermined length. In this case, even if the component with the largest range of motion among the components of the gesture is a finger gesture component, a gesture whose length of the trajectory of the fingertip drawn by the gesture exceeds the predetermined length does not fall within the first range. According to various embodiments, a gesture falling within the first range may be defined as a small gesture, and a gesture not falling within the first range may be defined as a large gesture. According to other various embodiments, a gesture falling within the first range may be defined as a large gesture, and a gesture not falling within the first range may be defined as a small gesture.

According to various embodiments, the first range may be defined as gestures in which the length of a trajectory of a body part drawn by the gesture is within a predetermined length range, regardless of which component of the gesture has the largest range of motion. For example, the first range may be defined as gestures in which the length of a trajectory of a fingertip drawn by the gesture is less than or equal to a predetermined length. According to various embodiments, a gesture in the first range may be defined as a small gesture, and a gesture not in the first range may be defined as a large gesture. According to other various embodiments, a gesture in the first range may be defined as a large gesture, and a gesture not in the first range may be defined as a small gesture. Although gestures using arms and hands have been described with reference to FIGS. 5A to 5D , it will be understood by those skilled in the art that gestures and the first range of gestures may be defined in addition to gestures using other parts of the body, such as feet, legs, torso, neck, or head.

If it is confirmed that the size of the gesture detected in operation 420 falls within the first range, at least one processor (210) of the electronic device (200) may perform a first operation corresponding to the gesture detected in operation 430. According to various embodiments, at least one processor (210) of the electronic device (200) may confirm the first operation by referring to a correspondence between gestures and operations stored in a memory of the electronic device (200). According to various embodiments, the first operation is not limited as long as it is an operation that can be performed by the electronic device (200). For example, when the electronic device (200) is the electronic device (101) described above with reference to FIG. 1 and the first range includes gestures in which a component having the largest range of motion among the components of the gesture is a finger gesture component, a correspondence between gestures and operations may be stored in the memory of the electronic device (200) as in the following example.

Gesture movement Move one finger from left to right Increase the volume of your audio output device Move one finger from right to left Reduce the volume of your audio output device Move one finger from bottom to top Increase the brightness of the display device Move one finger up and down Reduce the brightness of the display device

If it is determined that the size of the gesture detected in operation 420 does not fall within the first range, at least one processor (210) of the electronic device (200) can determine that the size of the gesture detected in operation 440 falls within a second range different from the first range, and transmit a first signal through the communication circuit (220) to control at least one of the one or more external electronic devices to perform a second operation corresponding to the detected gesture.

Here, the second range being different from the first range means that there is no intersection between the first range and the second range. For example, when the first range includes gestures whose component of the gesture has the largest range of motion is a finger gesture component, the second range may include gestures whose component of the gesture has the largest range of motion is a wrist gesture, an elbow gesture, or a shoulder gesture. According to various embodiments, the relationship between the sizes of gestures belonging to the first range and gestures belonging to the second range is not limited.

According to various embodiments, at least one processor (210) of the electronic device (200) may transmit a first signal to the server (320) via the communication circuit (220). According to various embodiments, at least one processor (210) of the electronic device (200) may transmit a first signal to at least one external electronic device via the communication circuit (220). An example of a method for specifying at least one external electronic device controlled to perform a second operation among one or more external electronic devices and an example of content that may be included in the first signal are the same as those described above with reference to FIG. 3, and therefore, are not described herein again.

According to various embodiments, even if the detected gesture is the same, the second action corresponding to the detected gesture may be different depending on the type of at least one external electronic device controlled to perform the second action among the one or more external electronic devices. For example, the correspondence between the gesture and the action depending on the type of the external electronic device may be as follows.

Gesture External electronic devices: Lighting External electronic devices: Audio External electronic devices: TV External electronic devices: Refrigerator Wrist gesture moving the hand from bottom to top Increase lighting intensity Increase audio volume Increase TV volume Increase set temperature Wrist gesture moving the hand from top to bottom Reduce lighting intensity Reduce audio volume Reduce TV volume Reduce set temperature Wrist gesture moving the hand from left to right Lights on Go to next track Go to the channel above Refrigerator on Wrist gesture of moving the hand from right to left Lights off Go to previous track Go to the channel below The refrigerator is off

According to various embodiments, the correspondence between gestures and actions according to the types of external electronic devices as exemplified in Table 2 may be stored in the memory of the electronic device (200) and/or in the memory of the server. Alternatively, each external electronic device may store the correspondence between gestures and actions for the types of external electronic devices in its memory.

In FIG. 4, an example is mainly described in which it is determined whether the electronic device (200) performs the first operation or controls at least one of the external electronic devices to perform the second operation depending on the size of the gesture. However, various criteria may be used in addition to the size of the gesture, depending on various embodiments. For example, if the detected gesture is a gesture by the left hand, the electronic device (200) may transmit a first signal to control at least one of the external electronic devices to perform the first operation, and if the detected gesture is a gesture by the right hand, the electronic device (200) may transmit a first signal to control at least one of the external electronic devices to perform the second operation. In another example, if the detected gesture is a gesture by the same hand as the hand that previously performed the gesture, the electronic device (200) may transmit a first signal to control at least one of the external electronic devices to perform the second operation, and if the detected gesture is a gesture by a different hand from the hand that previously performed the gesture. In another example, the electronic device (200) may transmit a first signal to control the electronic device (200) to perform a first operation if the detected gesture is a gesture by a hand wearing the wearable device, and to control at least one of the external electronic devices to perform a second operation if the detected gesture is a gesture by a hand not wearing the wearable device. In this example, the electronic device (200) may receive information about movement of the wearable device from the wearable device through pairing with the wearable device, and may determine that the detected gesture is a gesture by a hand wearing the wearable device if a difference between a movement range of the wearable device derived from the information about movement of the wearable device and a size of a gesture detected by the gesture sensor (230) is less than or equal to a predetermined value.

FIG. 5e illustrates an example of a screen showing a correspondence between gestures and actions according to various embodiments. The screen of the electronic device (500e) of FIG. 5e includes an object (510e) indicating that an external electronic device performing a second action is a smart TV when a size of a detected gesture falls within a second range. The screen of the electronic device (500e) of FIG. 5e shows a correspondence between gestures and actions in the case where the external electronic device is a TV in Table 2. In the screen of the electronic device (500e) of FIG. 5e, a motion (531e) for moving to an upper channel may be displayed in a direction in which a wrist gesture (521e) for moving a hand from left to right is displayed, a motion (532e) for decreasing TV volume may be displayed in a direction in which a wrist gesture (522e) for moving a hand from upper to lower is displayed, a motion (533e) for moving to a lower channel may be displayed in a direction in which a wrist gesture (523e) for moving a hand from right to left is displayed, and a motion (534e) for increasing TV volume may be displayed in a direction in which a wrist gesture (524e) for moving a hand from lower to upper is displayed.

FIG. 6 illustrates a flowchart (600) for explaining the operation of an electronic device according to various embodiments. In operation 610, at least one processor (210) of the electronic device (200) may output a list of one or more external electronic devices. According to various embodiments, when the electronic device (200) includes a display device, at least one processor (210) of the electronic device (200) may output a list of one or more external electronic devices by displaying a list of one or more external electronic devices on the display device of the electronic device (200). According to various embodiments, when the electronic device (200) includes a voice output device, at least one processor (210) of the electronic device (200) may output a voice regarding the list of one or more external electronic devices through the voice output device of the electronic device (200). For example, if the electronic device (200) is the electronic device (101) described above with reference to FIG. 1, the electronic device (200) may display a list of one or more external electronic devices on the display device (160) or output a list of one or more external electronic devices in voice form through the audio output device (155).

When the electronic device (200) is the electronic device (101) described above with reference to FIG. 1, an example of a screen displaying a list of one or more external electronic devices on the display device (160) is illustrated in FIG. 7. According to various embodiments, a phrase (730) indicating that a device to be controlled by a gesture can be selected and a list of one or more external electronic devices (741, 742, 743) that are candidates for the device to be controlled by a gesture may be displayed on the touch screen (720) of the electronic device (710). In the example of FIG. 7, the one or more external electronic devices include a Galaxy Watch (741), a smart TV (742), and Galaxy Buds (743).

In operation 620, at least one processor (210) of the electronic device (200) can identify a first device among one or more external electronic devices to perform an operation corresponding to a gesture whose size falls within a second range. The meaning of the second range has been described above with reference to operation 420, and therefore will not be described here again.

According to various embodiments, at least one processor (210) of the electronic device (200) may receive a user's input through an input device of the electronic device (200) and identify a first device based on the user's input. In the example of FIG. 7, when the user touches one of the Galaxy Watch (741), the smart TV (742), and the Galaxy Buds (743) on the touch screen (720), the selected device may be identified as the first device.

According to various embodiments, at least one processor (210) of the electronic device (200) can identify the first device based on a gesture detected via the gesture sensor (230). For example, in the example of the screen shown in FIG. 7, at least one processor (210) of the electronic device (200) can highlight and display one of the Galaxy Watch (741), the Smart TV (742), and the Galaxy Buds (743), and highlight an external electronic device one space above according to a gesture of moving a hand from lower to upper side, or highlight an external electronic device one space below according to a gesture of moving a hand from upper to lower side, and when a specific predefined gesture is detected, can identify the highlighted external electronic device as the final selection.

According to various embodiments, when the electronic device (200) includes a voice input device, at least one processor (210) of the electronic device (200) can identify the first device based on a user's voice input through the voice input device.

In operation 630, at least one processor (210) of the electronic device (200) may transmit a signal for controlling the first device to perform output through the communication circuit in response to the first device being identified among the one or more external electronic devices. According to various embodiments, at least one processor (210) of the electronic device (200) may transmit a signal for controlling the first device to perform output to the server (320) through the communication circuit (220). In this case, the server (320) that receives the signal from the electronic device (200) may transmit a signal including a command to perform output to the first device. The signal for controlling the first device to perform output, which is transmitted from the electronic device (200) to the server (320), may indicate an identifier of the first device identified in operation 620 and that the first device has been selected by the user.

According to various embodiments, at least one processor (210) of the electronic device (200) may transmit a signal including a command to perform an output to the first device via the communication circuit (220).

According to various embodiments, various types of outputs may be performed depending on the type of the first device. For example, if the first device includes a display device, the first device may control the display device to output blinking. For example, if the first device includes a display device, the first device may control the display device to display a visual animation. For example, if the first device includes a voice output device, the first device may control the voice output device to output voice. For example, if the first device includes an actuator, the first device may control the actuator to output vibration.

Although operation 630 is included in FIG. 6, operation 630 may be omitted according to various embodiments.

FIG. 8a illustrates a flowchart (800a) for explaining the operation of an electronic device according to various embodiments. In operation 810a, at least one processor (210) of the electronic device (200) can detect a gesture through a gesture sensor (230).

In operation 820a, at least one processor (210) of the electronic device (200) can determine whether the size of the detected gesture falls within the first range. The details described above with respect to operation 420 of FIG. 4 can be equally applied to operation 820a, and therefore are not described herein again.

If it is confirmed that the size of the gesture detected in operation 820a falls within the first range, in operation 830a, at least one processor (210) of the electronic device (200) may perform a first operation corresponding to the detected gesture. The details described above with respect to operation 430 of FIG. 4 can be equally applied to operation 830a, and therefore are not described herein again.

If it is determined that the size of the gesture detected in operation 820a does not fall within the first range, in operation 840a, at least one processor (210) of the electronic device (200) can determine whether the size of the gesture detected falls within a third range among the second ranges. Here, the second range means a range excluding the first range. The third range can be variously defined as described above with respect to operation 420 of FIG. 4, within the limit of falling within the second range.

If it is confirmed that the size of the gesture detected in operation 840a falls within the third range among the second ranges, in operation 850a, at least one processor (210) of the electronic device (200) may transmit a second signal through the communication circuit (220) to control the first external electronic device to perform a third operation corresponding to the detected gesture. Similar to what was described above with respect to operation 440 of FIG. 4, at least one processor (210) of the electronic device (200) may transmit the second signal to the server (320) or to the first external electronic device through the communication circuit (220).

If it is determined that the size of the gesture detected in operation 840a does not fall within the third range among the second ranges, then in operation 860a, at least one processor (210) of the electronic device (200) may determine that the size of the detected gesture falls within a fourth range different from the third range among the second ranges, and transmit a third signal through the communication circuit (220) to control the second external electronic device to perform a fourth operation corresponding to the detected gesture. Similar to what was described above with respect to operation 440 of FIG. 4 , at least one processor (210) of the electronic device (200) may transmit the third signal to the server (320) or to the second external electronic device through the communication circuit (220).

Which devices are specified as the first external electronic device and the second external electronic device will be described with reference to FIG. 8b. FIG. 8b illustrates an IoT environment according to various embodiments. The IoT environment (800b) may include an indoor environment (801b) and a server (850b) located away from the indoor environment (801b). The indoor environment (801b) may include a user (810b), an electronic device (820b), and external electronic devices (830b, 840b) other than the electronic device (820b). The electronic device (820b) may be the electronic device (200) described above with reference to FIG. 2.

According to various embodiments, the first external electronic device may be a device located closest to the electronic device (820b) among the external electronic devices (830b, 840b) in an indoor environment (801b) in which the electronic device (820b) is located, and the second external electronic device may be a device located second closest to the electronic device (820b) among the external electronic devices (830b, 840b). In this case, the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b) may be determined based on indoor positioning information of the electronic device (820b) and the external electronic devices (830b, 840b). According to various embodiments, the electronic device (820b) may receive indoor positioning information of the external electronic devices (830b, 840b) from the external electronic devices (830b, 840b), respectively, and may identify the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b) by comparing the indoor positioning information of the electronic device (820b). According to various embodiments, the server (850b) may receive indoor positioning information from the electronic device (820b) and the external electronic devices (830b, 840b), respectively, and may identify the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b) based on the indoor positioning information.

According to various embodiments, the first external electronic device and the second external electronic device may be specified according to a user's selection. The first external electronic device and the second external electronic device may be specified according to the user's selection in a similar manner to the method described above with reference to FIGS. 6 and 7.

When the electronic device (820b) can identify the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b), the electronic device (820b) can directly transmit (851b) a second signal to the first external electronic device in operation 850a, and directly transmit (852b) a third signal to the second external electronic device in operation 860a. In this case, the second signal can include an identifier of the first external electronic device. According to various embodiments, when a correspondence between a type of gesture and a performed operation for the first external electronic device is stored in the memory of the electronic device (820b), the second signal can indicate a third operation corresponding to the detected gesture. According to various embodiments, when a correspondence between a type of gesture and a performed operation for the first external electronic device is not stored in the memory of the electronic device (820b), the second signal can indicate a type of the detected gesture. The contents that can be included in the third signal are also similar to those of the second signal, so they will not be explained again.

In a case where the verification of the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b) is performed by the server (850b), the electronic device (820b) may transmit (860b) a second signal to the server (850b) in operation 850a, and transmit (860b) a third signal to the server (850b) in operation 860a. In this case, the second signal may include information for specifying the first external electronic device, such as a size value of a detected gesture, a size range to which the detected gesture belongs, or an identifier of the first external electronic device. Here, the size value of the gesture may be any objective data that may represent the size of the gesture. For example, the size value of the gesture may include a maximum value of a displacement of a body part moved by the gesture. If the server (850b) stores a correspondence between the size range of the gesture and the specified external electronic device, the second signal may include, as information for specifying the first external electronic device, a size value of the detected gesture or a size range to which the detected gesture belongs. Conversely, if the server (850b) does not store a correspondence between the size range of the gesture and the specified external electronic device, the second signal may include an identifier of the first external electronic device as information for specifying the first external electronic device. Similarly, depending on whether the server (850b) stores a correspondence between the size value of the gesture and the size range to which the gesture belongs, the second signal may indicate a size value of the detected gesture or a size range to which the detected gesture belongs.

According to various embodiments, depending on whether the server (850b) stores a correspondence between the type of gesture and the action, the second signal may indicate the type of the detected gesture or a third action corresponding to the detected gesture. The content that may be included in the third signal is also similar to that of the second signal, and therefore will not be described in detail.

When the verification of the first external electronic device and the second external electronic device among the external electronic devices (830b, 840b) is performed by the server (850b), the server (850b) can transmit (871b) a signal based on the second signal received from the electronic device (820b) to the first external electronic device. Here, depending on whether a correspondence relationship between the type of gesture and the operation is stored in the first external electronic device, the signal transmitted (871b) from the server (850b) to the first external electronic device can indicate the type of the detected gesture or a third operation corresponding to the detected gesture. In addition, the server (850b) can transmit (872b) a signal based on the third signal received from the electronic device (820b) to the second external electronic device. Here, depending on whether a correspondence between the type of gesture and the action is stored within the second external electronic device, the signal transmitted (872b) from the server (850b) to the second external electronic device may indicate the type of the detected gesture or a fourth action corresponding to the detected gesture.

FIG. 9 illustrates a flowchart (900) for explaining the operation of an electronic device according to various embodiments. In operation 910, the processor (210) of the electronic device (200) may output a list of external electronic devices within an environment in which the electronic device (200) is located. Details regarding operation 610 of FIG. 6 may be similarly applied to operation 910. When the electronic device (200) is the electronic device (101) described above with reference to FIG. 1, an example of a screen displaying a list of one or more external electronic devices on a display device (160) is illustrated in FIG. 10. According to various embodiments, a phrase (1030) indicating that a device to perform gesture recognition can be selected and a list of one or more external electronic devices (1041, 1042, 1043) that are candidates for a second device that is a device to perform gesture recognition may be displayed on a touch screen (1020) of the electronic device (1010). In the example of FIG. 10, one or more external electronic devices include a Galaxy Watch (741), a smart TV (742), and Galaxy Buds (743). According to various embodiments, a list of devices capable of performing gesture recognition among the external electronic devices in the environment where the electronic device (200) is located may be output.

In operation 920, at least one processor (210) of the electronic device (200) may identify a second device among the external electronic devices to detect a subsequent gesture. As to the method of identifying the second device, the details of operation 620 of FIG. 6 may be similarly applied.

In operation 930, at least one processor (210) of the electronic device (200) may transmit a signal indicating that the second device has been identified via the communication circuit (220). According to various embodiments, at least one processor (210) of the electronic device (200) may transmit a signal indicating that the second device has been identified to a server and/or the second device via the communication circuit (220). According to various embodiments, when the signal indicating that the second device has been identified is transmitted to the second device, the second device may activate a gesture sensor of the second device. According to various embodiments, when the signal indicating that the second device has been identified is transmitted to the server, the server may utilize a correspondence between a gesture and an operation corresponding to the second device.

In FIG. 9, an embodiment is illustrated in which the device performing gesture detection is changed from the electronic device (200) to the second device, but according to various embodiments, multiple electronic devices may perform gesture detection, and the corresponding operation may be different depending on which device the gesture is detected on.

FIG. 11 illustrates a flowchart for explaining the operation of an electronic device according to various embodiments. In operation 1110, at least one processor (210) of the electronic device (200) may detect a user's gesture through a gesture sensor (230).

In operation 1120, at least one processor (210) of the electronic device (200) can check the number of external electronic devices other than the electronic device. According to various embodiments, at least one processor (210) of the electronic device (200) can check the number of external electronic devices by querying the server (320) through the communication circuit (220). According to various embodiments, at least one processor (210) of the electronic device (200) can check the number of devices paired with the electronic device (200) as the number of external electronic devices through the communication circuit (220).

In operation 1130, at least one processor (210) of the electronic device (200) can check whether the number of external electronic devices identified in operation 1120 is 1.

If the number of external electronic devices identified in operation 1120 is 1, at least one processor (210) of the electronic device (200) can determine whether the size of the gesture detected in operation 1140 falls within the first range. The details of operation 420 of FIG. 4 can be equally applied to operation 1140.

If it is confirmed that the size of the gesture detected in operation 1140 falls within the first range, at least one processor (210) of the electronic device (200) may perform a first operation corresponding to the gesture detected in operation 1150. The details of operation 430 of FIG. 4 may be equally applied to operation 1150.

If it is determined that the size of the gesture detected in operation 1140 does not fall within the first range, at least one processor (210) of the electronic device (200) may transmit a first signal to control the external electronic device to perform a second operation corresponding to the gesture in operation 1160. The details of operation 440 of FIG. 4 may be equally applied to operation 1160.

If the number of external electronic devices confirmed in operation 1120 is not 1, at least one processor (210) of the electronic device (200) may, in operation 1170, determine whether the size of the detected gesture falls within the first range. Here, the case where the number of external electronic devices confirmed in operation 1120 is 0 is excluded, and the fact that the number of external electronic devices is not 1 may mean that the number of external electronic devices is plural. The details of operation 420 of FIG. 4 may be equally applied to operation 1170.

If it is confirmed that the size of the gesture detected in operation 1170 falls within the first range, at least one processor (210) of the electronic device (200) may, in operation 1180, identify a first device to perform an operation according to the gesture among the electronic device and two or more external electronic devices in response to the detected gesture. That is, if it is confirmed that the number of external electronic devices is plural, a gesture having a size falling within the first range may be used to identify a first device to perform an operation according to the gesture among the electronic device and two or more external electronic devices. For example, gestures having a size falling within the first range may be used to change a highlighted device among the electronic device and two or more external electronic devices, or to finally identify the highlighted device as the first device.

If it is determined that the size of the gesture detected in operation 1170 does not fall within the first range, at least one processor (210) of the electronic device (200) may determine whether the first device is the same device as the electronic device (200) that detected the gesture in operation 1190. According to various embodiments, if the first device has not been confirmed using a user gesture according to operation 1180, the electronic device (200) may be set as the first device by default.

If the first device is identified as the electronic device (200) in operation 1190, at least one processor (210) of the electronic device (200) may perform a first operation corresponding to the gesture in operation 1191. The details of operation 430 of FIG. 4 may be equally applied to operation 1191.

If it is determined in operation 1190 that the first device is not the electronic device (200), at least one processor (210) of the electronic device (200) may transmit a second signal through the communication circuit (220) to control the first device to perform a third operation corresponding to the detected gesture in operation 1192. The details of operation 440 of FIG. 4 may be equally applied to operation 1191.

According to various embodiments, the electronic device (200) includes a gesture sensor (230), a communication circuit (220), and at least one processor (210), wherein the at least one processor (210) is configured to detect a gesture of a user of the electronic device (200) through the gesture sensor (230), perform a first operation of the electronic device (200) corresponding to the detected gesture when a size of the detected gesture falls within a first range, and perform a second operation corresponding to the detected gesture when a size of the detected gesture falls within a second range different from the first range, and transmit a first signal through the communication circuit (220) to control at least one of the external electronic devices (331, 332).

According to various embodiments, the first signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of at least one of the one or more external electronic devices (331, 332), and a type of the detected gesture or the second operation, and the at least one processor (210) may be configured to transmit the first signal to a server.

According to various embodiments, the first signal may indicate a type of the detected gesture or the second operation, and an identifier of at least one of the one or more external electronic devices (331, 332), and the at least one processor (210) may be configured to transmit the first signal to the at least one of the one or more external electronic devices (331, 332).

According to various embodiments, the at least one processor (210) may be configured to output a list of the one or more external electronic devices (331, 332), identify a first device among the one or more external electronic devices (331, 332) to perform an operation corresponding to a gesture belonging to the second range, and, in response to the first device being identified among the one or more external electronic devices (331, 332), transmit a signal through the communication circuit (220) to control the first device to perform the output.

According to various embodiments, two or more external electronic devices (331, 332) are positioned in a space where the electronic device (200) is positioned, and the at least one processor (210) may be configured to transmit a second signal through the communication circuit (220) for controlling a first external electronic device (331, 332) of the two or more external electronic devices (331, 332) to perform a third operation corresponding to the detected gesture when the size of the detected gesture falls within a third range among the second ranges, and to transmit a third signal through the communication circuit (220) for controlling a second external electronic device (331, 332) of the two or more external electronic devices (331, 332) to perform a fourth operation corresponding to the detected gesture when the size of the detected gesture falls within a fourth range different from the third range among the second ranges.

According to various embodiments, the second signal may indicate an identifier of the first external electronic device (331, 332) and a type of the detected gesture or the third operation, and the at least one processor (210) may be configured to transmit the second signal to the first external electronic device (331, 332), and the third signal may indicate an identifier of the second external electronic device (331, 332) and a type of the detected gesture or the fourth operation, and the at least one processor (210) may be configured to transmit the third signal to the second external electronic device (331, 332).

According to various embodiments, the second signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of the first external electronic device (331, 332), and a type of the detected gesture or the third operation, and the at least one processor (210) may be configured to transmit the second signal to a server, and the third signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of the second external electronic device (331, 332), and a type of the detected gesture or the fourth operation, and the at least one processor (210) may be configured to transmit the third signal to a server.

According to various embodiments, the first external electronic device (331, 332) may be a device located closest to the electronic device (200) among the two or more external electronic devices (331, 332), and the second external electronic device (331, 332) may be a device located second closest to the electronic device (200) among the two or more external electronic devices (331, 332).

According to various embodiments, the at least one processor (210) may be configured to output a list of the two or more external electronic devices (331, 332) and identify at least one of the first external electronic device (331, 332) or the second external electronic device (331, 332) among the two or more external electronic devices (331, 332).

According to various embodiments, the at least one processor (210) may be configured to output a list of the one or more external electronic devices (331, 332), identify a second device among the one or more external electronic devices (331, 332) to detect a subsequent gesture, and transmit a signal indicating that the second device has been identified through the communication circuit (220).

According to various embodiments, the electronic device (200) includes a gesture sensor (230), a communication circuit (220), and at least one processor (210), wherein the at least one processor (210) detects a gesture of a user of the electronic device (200) through the gesture sensor (230), checks the number of external electronic devices (331, 332) other than the electronic device (200), and, if the number of the external electronic devices (331, 332) is 1, performs a first operation of the electronic device (200) corresponding to the detected gesture if the size of the detected gesture falls within a first range, and, if the size of the detected gesture falls within a second range different from the first range, transmits a first signal for controlling the external electronic devices (331, 332) through the communication circuit (220) to perform a second operation corresponding to the gesture, and, if the number of the external electronic devices (331, 332) is 2 or more, If the size of the detected gesture falls within the first range, a first device to perform an operation according to the gesture among the electronic device (200) and the two or more external electronic devices (331, 332) in response to the detected gesture is identified, and if the size of the detected gesture falls within the second range different from the first range and the first device is the electronic device (200), the first operation of the electronic device (200) corresponding to the gesture is performed, and if the size of the detected gesture falls within the second range and the first device is one of the two or more external electronic devices (331, 332), a second signal for controlling the first device to perform a third operation corresponding to the detected gesture may be transmitted through the communication circuit (220).

According to various embodiments, the first signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, an identifier of the external electronic device (331, 332), and a type of the detected gesture or the second operation, and the at least one processor (210) may be configured to transmit the first signal to a server, and the second signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, an identifier of the first device, and a type of the detected gesture or the third operation, and the at least one processor (210) may be configured to transmit the second signal to the server.

According to various embodiments, the first signal may indicate an identifier of the external electronic device (331, 332) and a type of the detected gesture or the second operation, and the at least one processor (210) may be configured to transmit the first signal to the external electronic device (331, 332), and the second signal may indicate an identifier of the first device and a type of the detected gesture or the third operation, and the at least one processor (210) may be configured to transmit the second signal to the first device.

According to various embodiments, the at least one processor (210) may be configured to transmit, via the communication circuit (220), a signal to control the first device to perform output in response to the first device being identified.

According to various embodiments, the at least one processor (210) may be configured to output a list of the external electronic devices (331, 332), identify a second device among the external electronic devices (331, 332) to detect a subsequent gesture, and transmit a signal indicating that the second device has been identified through the communication circuit (220).

According to various embodiments, a method performed in an electronic device (200) may include an operation of detecting a gesture of a user of the electronic device (200), an operation of performing a first operation of the electronic device (200) corresponding to the detected gesture when a size of the detected gesture falls within a first range, and an operation of transmitting a first signal through the communication circuit (220) to control at least one of one or more external electronic devices (331, 332) to perform a second operation corresponding to the detected gesture when a size of the detected gesture falls within a second range different from the first range.

According to various embodiments, the first signal may indicate a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of at least one of the one or more external electronic devices (331, 332), and a type of the detected gesture or the second operation, and the operation of transmitting the first signal through the communication circuit (220) may include an operation of transmitting the first signal to a server.

According to various embodiments, the first signal may indicate a type of the detected gesture or the second operation, and an identifier of at least one of the one or more external electronic devices (331, 332), and the operation of transmitting the first signal through the communication circuit (220) may include an operation of transmitting the first signal to at least one of the one or more external electronic devices (331, 332).

According to various embodiments, the method may further include an operation of outputting a list of the one or more external electronic devices (331, 332), an operation of identifying a first device among the one or more external electronic devices (331, 332) to perform an operation corresponding to a gesture falling within the second range, and an operation of transmitting, through the communication circuit (220), a signal for controlling the first device to perform an output in response to the first device being identified among the one or more external electronic devices (331, 332).

According to various embodiments, when two or more of the external electronic devices (331, 332) are located in the space where the electronic device (200) is located, and the size of the detected gesture falls within the second range, the operation of transmitting the first signal through the communication circuit (220) may include, when the size of the detected gesture falls within a third range among the second ranges, the operation of transmitting a second signal through the communication circuit (220) for controlling the first external electronic device (331, 332) among the two or more external electronic devices (331, 332) to perform a third operation corresponding to the detected gesture, and when the size of the detected gesture falls within a fourth range different from the third range among the second ranges, the operation of transmitting a third signal through the communication circuit (220) for controlling the second external electronic device (331, 332) among the two or more external electronic devices (331, 332) to perform a fourth operation corresponding to the detected gesture. there is.

According to various embodiments, the second signal may indicate an identifier of the first external electronic device (331, 332) and a type of the detected gesture or the third operation, and the operation of transmitting the second signal may include an operation of transmitting the second signal to the first external electronic device (331, 332), and the third signal may indicate an identifier of the second external electronic device (331, 332) and a type of the detected gesture or the fourth operation, and the operation of transmitting the third signal may include an operation of transmitting the third signal to the second external electronic device (331, 332).

According to various embodiments, the first external electronic device (331, 332) may be a device located closest to the electronic device among the two or more external electronic devices (331, 332), and the second external electronic device (331, 332) may be a device located second closest to the electronic device among the two or more external electronic devices (331, 332).

According to various embodiments, the method may further include an operation of outputting a list of the one or more external electronic devices (331, 332), an operation of identifying a second device among the one or more external electronic devices (331, 332) to detect a subsequent gesture, and an operation of transmitting a signal indicating that the second device has been identified through the communication circuit (220).

According to various embodiments, the method may further include an operation of outputting a list of the two or more external electronic devices (331, 332), an operation of identifying at least one of the first external electronic device (331, 332) or the second external electronic device (331, 332) among the two or more external electronic devices (331, 332), and an operation of transmitting a signal for controlling the identified at least one of the first external electronic device (331, 332) or the second external electronic device (331, 332) to perform an output through the communication circuit (220) in response to identifying at least one of the first external electronic device (331, 332) or the second external electronic device (331, 332) among the two or more external electronic devices (331, 332).

According to various embodiments, a method performed in an electronic device (200) comprises: an operation of detecting a gesture of a user of the electronic device through the gesture sensor (230); an operation of checking the number of external electronic devices (331, 332) other than the electronic device; an operation of performing a first operation of the electronic device corresponding to the detected gesture when the number of the external electronic devices (331, 332) is 1 and the size of the detected gesture falls within a first range; and an operation of transmitting a first signal for controlling the external electronic device (331, 332) to perform a second operation corresponding to the gesture when the size of the detected gesture falls within a second range different from the first range; an operation of transmitting a first signal through the communication circuit (220) to control the external electronic device (331, 332) to perform a second operation corresponding to the gesture when the number of the external electronic devices (331, 332) is 2 or more and the size of the detected gesture falls within the first range; The method may include: an action of identifying a first device to perform an action; an action of performing the first action of the electronic device corresponding to the gesture if the size of the detected gesture falls within the second range that is different from the first range and the first device is the electronic device; and an action of transmitting a second signal through the communication circuit (220) to control the first device to perform a third action corresponding to the detected gesture if the size of the detected gesture falls within the second range and the first device is one of the two or more external electronic devices (331, 332).

The method may further include an operation of outputting a list of the external electronic devices (331, 332), an operation of identifying a second device among the external electronic devices (331, 332) to detect a subsequent gesture, and an operation of transmitting a signal indicating that the second device has been identified through the communication circuit (220).

The various embodiments of this document and the terminology used herein are not intended to limit the technology described in this document to the specific embodiments, but should be understood to encompass various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B, or C" or "at least one of A, B and/or C" can include all possible combinations of the items listed together. Expressions such as "first", "second", "first" or "second" can modify the corresponding components, regardless of order or importance, and are only used to distinguish one component from another and do not limit the corresponding components. When it is said that a certain (e.g., a first) component is "(functionally or communicatively) connected" or "connected" to another (e.g., a second) component, said certain component may be directly connected to said other component, or may be connected through another component (e.g., a third component).

The term "module" as used in this document includes a unit composed of hardware, software or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally composed component or a minimum unit or part thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (140)) including instructions stored in a machine-readable storage media (e.g., an internal memory (136) or an external memory (138)) that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., an electronic device (101)) according to the disclosed embodiments, which is a device that can call instructions stored from the storage media and operate according to the called instructions. When the instructions are executed by a processor (e.g., the processor (120)), the processor may directly, or under the control of the processor, perform a function corresponding to the instructions by using other components. The instructions may include codes generated or executed by a compiler or an interpreter. The machine-readable storage media may be provided in the form of a non-transitory storage media. Here, ‘non-transitory’ means that the storage medium does not contain signals and is tangible, but does not distinguish between whether the data is stored semi-permanently or temporarily in the storage medium.

According to an exemplary embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded between sellers and buyers as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or online through an application store (e.g., Play StoreTM). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a manufacturer's server, a server of an application store, or a relay server.

Each of the components (e.g., modules or programs) according to various embodiments may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some of the components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some of the operations may be executed in a different order, omitted, or other operations may be added.

Claims (20)

In electronic devices,
At least one sensor,
communication circuit,
Memory set up to store instructions, and
comprising at least one processor,
The above instructions, when executed by the at least one processor, cause the electronic device to:
Detecting a user's gesture through at least one sensor,
If the size of the detected gesture falls within the first range, a first operation related to the electronic device corresponding to the detected gesture is performed,
Including instructions set to transmit a first signal through the communication circuit to control at least one of the one or more external electronic devices to perform a second operation corresponding to the detected gesture when the size of the detected gesture falls within a second range different from the first range,
An electronic device characterized in that whether the size of the gesture falls within the first range is determined if the gesture satisfies a first condition for the user's body part determined according to the maximum length of the trajectory expressed by the gesture and a second condition that the length of the trajectory of the body part expressed by the gesture is less than or equal to a predetermined length. In the first paragraph,
The first signal above is:
a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of at least one of the one or more external electronic devices, and
Contains information about the type of the detected gesture or the second action,
The above instructions, when executed by the at least one processor, cause the electronic device to:
An electronic device, characterized in that it further comprises instructions set to transmit the first signal to a server. In the first paragraph,
The first signal above is:
Information about the type of the detected gesture or the second action, and
comprising at least one identifier of said one or more external electronic devices;
The above instructions, when executed by the at least one processor, cause the electronic device to:
An electronic device, characterized in that it further comprises instructions configured to transmit said first signal to said at least one of said one or more external electronic devices. In the first paragraph,
The above instructions, when executed by the at least one processor, cause the electronic device to:
Output a list of one or more of the above external electronic devices,
Identify a first device that performs an action corresponding to a gesture falling within the second range among the one or more external electronic devices;
An electronic device, characterized in that it further includes instructions configured to transmit a signal through the communication circuit to control the first device to perform an output in response to the first device being identified among the one or more external electronic devices. In the first paragraph,
Based on the location of multiple external electronic devices around the above electronic device:
The above instructions, when executed by the at least one processor, cause the electronic device to:
If the size of the detected gesture falls within a third range among the second ranges, a second signal is transmitted through the communication circuit to control a first external electronic device among the plurality of external electronic devices to perform a third operation corresponding to the detected gesture,
An electronic device characterized in that it further includes instructions set to transmit a third signal through the communication circuit for controlling a second external electronic device among the plurality of external electronic devices to perform a fourth operation corresponding to the detected gesture when the size of the detected gesture falls within a fourth range different from the third range among the second ranges. delete delete delete delete In the first paragraph,
The above instructions, when executed by the at least one processor, cause the electronic device to:
Output a list of one or more of the above external electronic devices,
Among the one or more external electronic devices, identify a second device to detect a subsequent gesture;
An electronic device, characterized in that it further comprises instructions set to transmit a signal indicating that the second device has been confirmed through the communication circuit. A non-transitory storage medium configured to store computer-executable instructions, said instructions, when executed by at least one processor of an electronic device, causing the electronic device to:
Detecting a user's gesture through at least one sensor of the electronic device,
If the size of the detected gesture falls within the first range, a first operation related to the electronic device corresponding to the detected gesture is performed,
Including instructions set to transmit a first signal through a communication circuit of the electronic device to control at least one of the one or more external electronic devices to perform a second operation corresponding to the detected gesture when the size of the detected gesture falls within a second range different from the first range,
A non-transitory recording medium, characterized in that whether the size of the gesture falls within the first range is determined if the gesture satisfies a first condition for the user's body part determined according to the maximum length of the trajectory expressed by the gesture and a second condition that the length of the trajectory of the body part expressed by the gesture is less than or equal to a predetermined length. In Article 11,
The first signal above is:
a size value of the detected gesture, a size range to which the detected gesture belongs, or an identifier of at least one of the one or more external electronic devices, and
Contains information about the type of the detected gesture or the second action,
The above instructions, when executed by the at least one processor, cause the electronic device to:
A non-transitory recording medium, characterized in that it further comprises instructions set to transmit the first signal to a server. In Article 11,
The first signal above is:
Information about the type of the detected gesture or the second action, and
comprising at least one identifier of said one or more external electronic devices;
The above instructions, when executed by the at least one processor, cause the electronic device to:
A non-transitory recording medium, further characterized by comprising instructions configured to transmit said first signal to said at least one of said one or more external electronic devices. delete In Article 11,
The above instructions, when executed by the at least one processor, cause the electronic device to:
Output a list of one or more of the above external electronic devices,
Identify a first device that performs an action corresponding to a gesture falling within the second range among the one or more external electronic devices;
A non-transitory recording medium further characterized by including instructions configured to transmit a signal through the communication circuit to control the first device to perform an output in response to the first device being identified among the one or more external electronic devices. delete delete delete delete delete