WO2025042024A1 - Electronic device for identifying direction of disposition of external electronic device which shares input device, method for operating same, and storage medium - Google Patents

Abstract

According to an embodiment, an electronic device (301) may comprise a first display (360), a communication circuit (390), a memory (330) for storing instructions, and at least one processor (320). According to an embodiment, the instructions, when executed by the at least one processor, may cause the electronic device to perform the following operations: if a pointer displayed on the first display reaches one edge of the first display, the electronic device identifies, on the basis of configured information regarding the direction in which the second display (361) of an external electronic device (302) is disposed with reference to the first display (360) stored in the memory (330), whether the direction in which the point is supposed to move corresponds to the configured direction in which the second display is disposed. If the direction in which the point is supposed to move does not correspond to the configured direction in which the second display is disposed, the electronic device identifies whether at least one condition for changing the configured direction in which the second display is disposed is satisfied. On the basis of the at least one condition being satisfied, the electronic device changes the configured direction in which the second display is disposed so as to correspond to the direction in which the point is supposed to move, and transmits a signal for displaying the pointer after moving same from the first display to the second display to the external electronic device through the communication circuit.

Description

Electronic device for identifying the arrangement direction of external electronic devices sharing an input device, method of operation thereof and storage medium

One embodiment disclosed in this document relates to an electronic device for identifying a placement direction of an external electronic device sharing an input device, a method of operating the same, and a storage medium.

As the mobile communication networks and processor processing speeds of electronic devices such as smart phones have become faster, smart phones equipped with various open operating systems and new services have appeared. As the performance of smart phones has improved to a level similar to that of computers, their frequency of use has also increased compared to devices such as desktops (e.g., PCs, Note PCs, or Desktop computers). Accordingly, smart phone users are storing and using various files on their smart phones rather than on their desktops.

In addition, as the demand for viewing various contents on electronic devices increases, various technologies for sharing such contents are being developed. For example, various technologies are being developed for using electronic devices by connecting electronic devices to external electronic devices according to the user's needs.

The above information may be provided as related art for the purpose of assisting in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

According to one embodiment, the electronic device (301) may include a first display (360), a communication circuit (390), a memory (330) for storing instructions, and at least one processor (320).

According to one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to, when a pointer displayed on the first display reaches an edge of the first display, identify whether the direction in which the pointer is to move corresponds to the set arrangement direction of the second display (361) of the external electronic device (302) based on set information about the arrangement direction of the second display (360) stored in the memory.

In one embodiment, the instructions may be configured to cause the electronic device to identify whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved and the set arrangement direction of the second display do not correspond.

In one embodiment, the instructions may be configured to cause the electronic device to, based on satisfaction of the at least one condition, change the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmit a signal to the external electronic device through the communication circuit to move the pointer from the first display to the second display for display.

According to one embodiment, a method for identifying an arrangement direction of an external electronic device (302) sharing an input device (350) in an electronic device (301) may include an operation of identifying, when a pointer displayed on a first display (360) of the electronic device reaches an edge of the first display, whether the direction in which the pointer is to move corresponds to a set arrangement direction of a second display (361) of the external electronic device (302) based on the first display.

According to one embodiment, the method may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved and the set arrangement direction of the second display do not correspond.

According to one embodiment, the method may include, based on satisfaction of the at least one condition, changing the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display.

According to one embodiment, a storage medium storing at least one computer-readable instruction, wherein the at least one instruction, when executed by at least one processor (320) of an electronic device (301), causes the electronic device to perform at least one operation, wherein the at least one operation may include an operation of identifying, based on set information about an arrangement direction of a second display (361) of an external electronic device (302) with respect to the first display, whether the direction in which the pointer is to be moved corresponds to a set arrangement direction of the second display when a pointer displayed on a first display (360) of the electronic device reaches an edge of the first display.

In one embodiment, the at least one operation may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display.

According to one embodiment, the at least one operation may include, based on satisfaction of the at least one condition, changing the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display.

FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

FIG. 2A is an example screen for setting placement-related information for connected devices according to one embodiment.

FIG. 2b is an example diagram explaining a layout setting for connected devices according to one embodiment and a case where the actual layout is different.

FIG. 3a is an internal block diagram of an electronic device according to one embodiment.

FIG. 3b is a detailed block diagram for identifying the placement direction of an external electronic device according to one embodiment.

FIG. 4 is a flowchart of operations of an electronic device for identifying a placement direction of an external electronic device according to one embodiment.

FIG. 5 is a flowchart illustrating an operation for identifying a first condition for changing the arrangement direction of an external electronic device according to one embodiment.

FIG. 6A is an exemplary diagram illustrating a method for identifying a designated input for moving past one edge of a display of an electronic device toward an external electronic device according to one embodiment.

FIG. 6b is an example screen diagram for explaining a first condition for changing the arrangement direction of an external electronic device according to one embodiment.

FIG. 7 is an example diagram explaining a case where the layout settings for connected devices according to one embodiment correspond to the actual layout.

FIG. 8 is a flowchart illustrating an operation for identifying a second condition for changing the arrangement direction of an external electronic device according to one embodiment.

FIG. 9 is an example screen diagram for explaining a second condition for changing the arrangement direction of an external electronic device according to an embodiment of the present invention.

FIG. 10 is a flowchart illustrating an operation for identifying a third condition for changing the arrangement direction of an external electronic device according to one embodiment.

FIG. 11 is an example screen diagram for explaining a third condition for changing the arrangement direction of an external electronic device according to one embodiment.

Figure 12 is an example screen for setting up a layout when initially connecting devices according to one embodiment.

FIG. 13 is a screen example diagram showing an object for changing the arrangement direction of an external electronic device according to an embodiment of the present invention.

In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components. In addition, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

The processor (120) may control at least one other component (e.g., a hardware or software component) of an electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store result data in a nonvolatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

The auxiliary processor (123) may control at least a portion of functions or states associated with at least one of the components of the electronic device (101) (e.g., the display module (160), the sensor module (176), or the communication module (190)), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing device) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. Such learning may be performed, for example, in the electronic device (101) itself on which the artificial intelligence model is executed, or may be performed through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may additionally or alternatively include a software structure.

The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or nonvolatile memory (134).

The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

The audio output module (155) can output an audio signal to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive an incoming call. According to one embodiment, the receiver can be implemented separately from the speaker or as a part thereof.

The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) can include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) can include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can obtain sound through an input module (150), or output sound through an audio output module (155), or an external electronic device (e.g., an electronic device (102)) (e.g., a speaker or a headphone) directly or wirelessly connected to the electronic device (101).

The sensor module (176) can detect an operating state (e.g., power or temperature) of the electronic device (101) or an external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., the electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that a user can perceive through a tactile or kinesthetic sense. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) can include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery (189) can power at least one component of the electronic device (101). In one embodiment, the battery (189) can include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (190) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., the electronic device (102), the electronic device (104), or the server (108)), and performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., the application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module or a power line communication module). Among these communication modules, a corresponding communication module may communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196) to identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199).

The wireless communication module (192) can support a 5G network and next-generation communication technology after a 4G network, for example, NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), terminal power minimization and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) may support various requirements specified in an electronic device (101), an external electronic device (e.g., an electronic device (104)), or a network system (e.g., a second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL) each, or 1 ms or less for round trip) for URLLC realization.

The antenna module (197) can transmit or receive signals or power to or from the outside (e.g., an external electronic device). According to one embodiment, the antenna module (197) can include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) can include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), can be selected from the plurality of antennas by, for example, the communication module (190). A signal or power can be transmitted or received between the communication module (190) and the external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) can be additionally formed as a part of the antenna module (197).

According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC positioned on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) positioned on or adjacent a second side (e.g., a top side or a side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

At least some of the above components may be connected to each other and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

In one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). In one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to perform at least a part of the function or service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result, as is or additionally processed, as at least a part of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an IoT (Internet of Things) device. The server (108) may be an intelligent server using machine learning and/or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199). The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

In the detailed description below, reference numerals in the drawings may be given to or omitted for configurations that can be easily understood through the preceding embodiments, and their detailed descriptions may also be omitted. The electronic device (101) according to one embodiment disclosed in this document may be implemented by selectively combining the configurations of each embodiment, and the configuration of one embodiment may be replaced. For example, it should be noted that the present disclosure is not limited to a specific drawing or embodiment.

FIG. 2A is an example screen for setting placement-related information for connected devices according to one embodiment.

FIG. 2A illustrates a setting screen (or setting menu) (200) related to a program for controlling an electronic device (e.g., an electronic device (101) of FIG. 1) and an external electronic device (e.g., an external electronic device (102) of FIG. 1) using an input device of the electronic device. The electronic device may provide a function for controlling an external electronic device connected to the electronic device or transferring data through an input device of the electronic device, and such a function may be referred to as a multi-control function. The multi-control function as described above may be provided between a plurality of devices registered with the same user account. In addition, the multi-control item (205) must be set to 'on' in each device to control an external electronic device connected to the electronic device or transfer data through the input device of the electronic device.

When the electronic device is first connected to an external electronic device with which the input device will be shared, the arrangement direction of the external electronic device is set to the default arrangement direction, and the user can directly set (or change) the arrangement direction of the external electronic device based on the electronic device according to its physical location through the setting screen (200). Thereafter, when the external electronic device with which the input device will be shared is reconnected, the electronic device can set the arrangement direction of the external electronic device based on the arrangement direction previously set by the user.

According to one embodiment, identifying the arrangement direction of the external electronic device should be understood as identifying the direction in which the external electronic device is placed with respect to the electronic device. Here, the arrangement direction refers to the arrangement between the display of the electronic device and the display of the external electronic device, and may mean the positional relationship of the display of the external electronic device with respect to the display of the electronic device. Accordingly, the arrangement direction of the external electronic device may be expressed by the display arrangement, the display position of the external electronic device with respect to the display of the electronic device, the display displacement of the external electronic device with respect to the display of the electronic device, or the display direction of the external electronic device with respect to the display of the electronic device.

For example, as illustrated in FIG. 2A, a placement direction between an object (215) representing an electronic device (e.g., a PC) and an object (210) representing an external electronic device (e.g., a tablet) may be set, and an arrangement option of one of the right, left, top, or bottom may be set for the placement direction in which the external electronic device is placed relative to the electronic device. For example, in order to set a state in which the external electronic device (e.g., a tablet) is placed to the left of the electronic device (e.g., a PC), a user may select the object (210) and move it to be placed next to the object (215). According to one embodiment, information about the placement direction may include information about a placement position (or order, index). For example, information about the placement direction may include an order (e.g., front, back) in which the external electronic device is placed in space (or array) relative to the electronic device. For example, information about the placement direction may include an index (e.g., array[0,1], array[1,0], array[1,2], or array[2,1]) at which the external electronic device is placed in space (or array) relative to the electronic device.

If the above arrangement direction is expressed as an array, it is as shown in Table 1 below.

[0][0] [0][1] [0][2] [1][0] [1][1] [1][2] [2][0] [2][1] [2][2]

Referring to Table 1 above, when an index indicating a reference is assumed to be, for example, array[1,1], an index indicating that an external electronic device is placed to the left of the electronic device based on the electronic device may be, for example, array[1,0]. In addition, an index indicating that an external electronic device is placed to the right of the electronic device may be, for example, array[1,2]. In addition, an index indicating that an external electronic device is placed above the electronic device may be, for example, array[0, 1]. In addition, an index indicating that an external electronic device is placed below the electronic device may be, for example, array[2, 1]. After the setting of the arrangement direction is completed through the setting screen (200) as in FIG. 2A, the user can freely work by connecting the display of the electronic device and the display of the external electronic device as if they were one device. The user can move a pointer between the electronic device and the external electronic device or move data and images using the input device of the electronic device.

However, there may be cases where the layout settings input by the user through the setting screen (200) of Fig. 2a for the connected devices are different from the actual layout of each electronic device. There may be cases where the layout direction in which the user places the external electronic device on the left side of the electronic device through the setting screen (200) of Fig. 2a is different from the actual location in which the external electronic device is placed adjacent to the electronic device as in Fig. 2b, for example, the layout direction of the external electronic device.

For example, if the arrangement direction is set so that an external electronic device (e.g., a tablet) is placed on the left side of an electronic device (e.g., a PC), but the external electronic device (e.g., a tablet) (202) is actually placed on the right side of the electronic device (e.g., a PC) (201) as shown in FIG. 2b, a situation may occur where, when a user moves a pointer (220) using an input device, the pointer cannot move any further from one edge of the electronic device (201) toward the external electronic device (202).

According to one embodiment, when sharing an input device between an electronic device and an external electronic device, a method may be required to control movement without restrictions using the input device on the displays of the devices even if the set arrangement direction between the devices is different from the actual physical arrangement direction. Therefore, in one embodiment, after the setting of the arrangement direction (or positional relationship) of the electronic devices is completed, if the actual arrangement directions of the electronic devices are different, an electronic device, an operation method thereof, and a storage medium for identifying the arrangement direction of an external electronic device sharing an input device can be provided so that the arrangement direction can be automatically changed. By doing so, even if the user does not input a change to the actual arrangement, the changed arrangement direction can be automatically reflected, thereby providing a continuous and unified user experience through the electronic devices.

FIG. 3a is an internal block diagram of an electronic device according to one embodiment.

Referring to FIG. 3A, an electronic device (301) (e.g., the electronic device (101) of FIG. 1) according to an embodiment may include at least one processor (320) (e.g., the processor (120) of FIG. 1), a memory (330) (e.g., the memory (130) of FIG. 1), a first display (360) (e.g., the display module (160) of FIG. 1), and a communication circuit (390) (e.g., the communication module (190) of FIG. 1). Here, not all components illustrated in FIG. 3A are essential components of the electronic device (101), and the electronic device (101) may be implemented by more or fewer components than the components illustrated in FIG. 3A.

Each of the electronic device (301) and the external electronic device (302) may be a device such as a smartphone, a laptop computer, or a tablet PC.

The communication circuit (390) may be connected to an external electronic device (302). According to an embodiment, the communication circuit (390) may be connected to one or more external electronic devices (302) through a 1:1 connection method or a 1:n connection method. According to an embodiment, the communication circuit (390) may be wirelessly connected to the external electronic device (302) through various communication methods (or communication links). For example, the communication method may include a Bluetooth connection method. For example, the electronic device (301) and the external electronic device (302) may be devices that are interconnected by logging in using the same user account.

The camera (380) can capture images of a subject (e.g., a user) and may be a front camera. The camera (380) can capture images to detect the user's gaze periodically or non-periodically.

The electronic device (301) can transmit data related to a user input received through at least one input device (350) of the electronic device (301) to an external electronic device (201). The at least one input device (350) can correspond to a device for inputting an object that visually displays a user input on a first display (360) of the electronic device (301). For example, the object can be a pointer for visually indicating a location of a user input on a first display (360) of the electronic device (301) (e.g., the first display (360) of FIGS. 6A to 7) or a second display (361) of the external electronic device (302) (e.g., the second display (361) of FIGS. 6A to 7). For example, the at least one input device (350) may include, but is not limited to, at least one of a mouse, a trackpad, a remote joystick, a gamepad, or a remote controller. The data related to the user input may correspond to information for displaying the pointer corresponding to the user input from the first display (360) of the electronic device (301) to the second display (361) of the external electronic device (302).

As the electronic device (301) communicates with the external electronic device (302), the processor (320) can identify that at least one external electronic device (302) among multiple devices of the same account is neighboring.

The processor (320) can share the input device (350) of the electronic device (301) with the external electronic device (302) when the function for sharing the input device is activated (or turned on) in the above-mentioned communication-connected state. For example, the processor (320) can provide a function for controlling the external electronic device (302) connected to the electronic device (301) or moving data through the input device (350) of the electronic device (301).

The processor (320) can display an object (e.g., a pointer) corresponding to the input device (350) on the first display (360). When the function for sharing the input device (350) is activated (or on), the pointer displayed on the first display (360) can pass an edge of the second display (361) and be displayed on the second display (361) of the external electronic device (302). The processor (320) can output content, such as an image or text output through the first display (360), to the second display (361) of the external electronic device (302) in real time through the communication circuit (390) based on an operation of moving the content using the pointer of the input device. Accordingly, a continuous user experience can be provided between the electronic device (301) registered with the same account and the external electronic device (302).

In one embodiment, after the setting of the arrangement direction (or positional relationship) of an external electronic device (302) based on an electronic device (301) is completed, if the actual arrangement direction of the external electronic device (302) is different, a function for automatically changing the setting of the arrangement direction without entering a setting screen (e.g., setting screen (200) of FIG. 2A) for a program for controlling using an input device can be provided.

According to one embodiment, the processor (320) may identify whether a pointer displayed on the first display (360) reaches one edge of the first display (360). In response to identifying that the pointer displayed on the first display (360) reaches one edge of the first display (360), the processor (320) may identify information about a layout direction stored in the memory (330). The stored information about the layout direction may be set information about a layout direction of a second display (361) of an external electronic device (302) with respect to the first display (360). In addition, the stored information about the layout direction may be information about a layout direction set as a default in the memory (330) of the electronic device (301) or information about a layout direction previously set by a user.

The processor (320) can identify whether the direction in which the pointer is to be moved corresponds to the set arrangement direction of the second display (361) based on the information about the stored arrangement direction. For example, it can be assumed that the information about the stored arrangement direction indicates a case in which the external electronic device (302) is placed on the left side of the electronic device (301), and in reality, the external electronic device (302) is placed on the left side of the electronic device (301).

If the direction in which the pointer is to move past one edge (e.g., the left edge) of the first display (360) corresponds to the set arrangement direction of the second display (361) (e.g., information indicating that the external electronic device (302) is arranged to the left of the electronic device (301), the pointer may be controlled to move and display toward the second display (361) past one edge (e.g., the left edge) of the first display (360). Accordingly, the processor (320) may transmit a signal for displaying the pointer by moving it from the first display (360) to the second display (361) through the communication circuit (390), to the external electronic device (302). For example, the signal for displaying the pointer by moving it from the first display (360) to the second display (361) may include image information including the pointer. For example, the signal may include coordinate values (e.g., pixel position on the x-axis, pixel position on the y-axis) at which the pointer is to be displayed. According to one embodiment, the external electronic device (302) may display the pointer on the second display (361) according to the coordinate values included in the signal.

On the other hand, for example, it can be assumed that information about the stored arrangement direction indicates a case where the external electronic device (302) is placed on the left side of the electronic device (301), but in reality, the external electronic device (302) is placed on the right side of the electronic device (301). In this case, since the external electronic device (302) is located on the right side with respect to the electronic device (301), the user can intuitively move the pointer to the right using the input device.

The processor (320) may identify that when the pointer reaches one edge (e.g., the right edge) of the first display (360), the direction in which the pointer moves past one edge (e.g., the right edge) of the first display (360) does not correspond to the set arrangement direction of the second display (361) (e.g., information indicating that the external electronic device (302) is arranged to the left of the electronic device (301). Accordingly, the pointer may no longer move past one edge (e.g., the right edge) of the first display (360).

In one embodiment, when the direction in which the pointer is to move past one edge of the first display (360) and the set arrangement direction of the second display (361) do not correspond, for example, when a situation occurs in which the pointer does not move past one edge of the first display (360), the processor (320) may identify whether a user input by the input device is an input with the intention of moving the pointer to the second display (361). For example, the processor (320) may identify whether an input for moving the pointer that has reached one edge of the first display (360) is an intended input for changing information about a previously stored arrangement direction.

In one embodiment, the processor (320) may detect a designated input to identify an intended input for changing information about a previously stored layout orientation. In one embodiment, the processor (320) may identify whether an event indicative of a designated input (e.g., a designated gesture) occurs when a user moves a pointer to reach an edge of the first display (360). For example, the processor (320) may detect an event indicative of the designated input in response to identifying that the pointer has been detected at an edge of the first display (360) a specified number of times and/or a specified amount of time.

Upon detection of an event representing the above-mentioned input, the processor (320) may determine that the input of moving the pointer to reach an edge of the first display (360) is an intended input for changing information about the previously stored layout orientation.

In response to detection of an event indicating the above-mentioned specified input, the processor (320) may identify whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied. Here, in order for the user to recognize that the information on the previously stored arrangement direction can be changed, the processor (320) may display an indicator including one edge of the first display (360) on at least a portion of the first display (360). For example, the indicator may be referred to as a border as a role of guiding that the information on the previously stored arrangement direction can be changed, and a method of displaying the indicator will be described later with reference to FIG. 6B. Here, one edge of the first display (360) may correspond to a direction in which the pointer is to be moved among four edges (e.g., top, bottom, left, or right) of the first display (360).

According to one embodiment, the at least one condition may include at least one of: detecting a display direction toward which a user's gaze is directed using the camera (380) of the electronic device (301) or the camera of the external electronic device (302), when a designated key input for changing information about the stored arrangement direction is detected, or detecting that the pointer is positioned on the indicator for more than a designated time. For example, the processor (320) may activate the camera (380) in response to detection of an event indicating the designated input, and capture an image for detecting the user's gaze using the camera (380). In addition, the processor (320) may also capture an image for detecting the user's gaze periodically or aperiodically using the camera (380).

According to one embodiment, the processor (320) may change the information about the stored arrangement direction so that the set arrangement direction of the second display (361) corresponds to the direction in which the pointer is to be moved, based on the satisfaction of the at least one condition. In addition, while changing the information about the stored arrangement direction, the processor (320) may transmit a signal to the external electronic device (302) through the communication circuit (390) to move and display the pointer from the first display (360) to the second display (361). In one embodiment, the processor (320) may, in response to identifying that the at least one condition is satisfied, change the information about the stored arrangement direction after moving and displaying the position of the pointer to the second display (361).

In one embodiment, even if the information about the arrangement direction stored in the program for sharing the input device is not changed, the set arrangement direction of the second display (361) can automatically reflect the positional relationship between the first display (360) of the electronic device (301) and the second display (361) of the external electronic device (302) so as to correspond to the direction in which the pointer is to move. In addition, in one embodiment, if the actual arrangement between the electronic device (301) and the external electronic device (302) is changed after the user completes setting the arrangement direction, the changed arrangement relationship is automatically reflected even if the user does not directly input the changed arrangement relationship, thereby providing a continuous user experience.

According to one embodiment, reference may be made to FIG. 3B to specifically examine the operation of the processor (320). FIG. 3B is a detailed block diagram for identifying the arrangement direction of an external electronic device according to one embodiment.

Referring to FIG. 3B, the electronic device (301) may include a pointer movement detection module (321), a user intention confirmation module (322), an event detection module (323), a setting change module (324), and/or a first display (360) (e.g., the first display (360) of FIG. 3A). In one embodiment, FIG. 3B illustrates each component of the pointer movement detection module (321), the user intention confirmation module (322), the event detection module (323), and/or the setting change module (324), but the operations of each component (321, 322, 323, 324) may also be performed by the processor (320).

In one embodiment, the pointer movement detection module (321) may detect a motion in which a user bumps into a border surface of the first display (360) as the pointer moves toward the edge of the first display (360). For example, the pointer movement may be detected exceeding a specified number of times and/or a specified period of time (e.g., when n or more collisions are detected within a second) on one edge of the first display (360). In response to identifying that the pointer has been detected exceeding a specified number of times and/or a specified period of time on one edge of the first display (360), the pointer movement detection module (321) may request the first display (360) to display an indicator (e.g., a border) on one edge of the first display (360) to determine whether there is an intention to move the pointer past the border surface.

In one embodiment, the first display (360) may display an indicator including a portion of one edge of the first display (360) in response to receiving a request to display the indicator from the pointer movement detection module (321). The first display (360) may wait for a response from the user intention confirmation module (322) while displaying the indicator.

Even if the user does not intend to move the pointer beyond the edge of the first display (360), a situation may frequently occur where the pointer reaches the edge of the first display (360), so that an action to confirm the user's intention can be performed, and thus, a response from the user intention confirmation module (322) in the first display (360) can be waited for.

In one embodiment, the user intention verification module (322) can identify whether a pointer by an input device is positioned on the pointer while the first display (360) displays the pointer (e.g., within b seconds). In response to identifying that the pointer is positioned on the pointer, the user intention verification module (322) can request detection of a designated event, e.g., at least one condition, for changing information about the arrangement direction stored in the event detection module (323), and wait for reception by the event detection module (323) corresponding thereto. On the other hand, in response to not identifying that the pointer is positioned on the pointer, the user intention verification module (322) can request the first display (360) to deactivate the display of the pointer.

In one embodiment, the event detection module (323) may, in response to a request for detection of at least one condition, identify whether an event corresponding to at least one condition occurs. For example, the event detection module (323) may identify whether an event corresponding to at least one condition occurs within a specified time (e.g., within c seconds).

In one embodiment, the event detection module (323) may identify whether an input corresponding to a specific key and/or a specific key combination occurs within a specified time period (e.g., within c seconds). Based on identifying that an input corresponding to a specific key and/or a specific key combination occurs, the event detection module (323) may determine that at least one of the above conditions is satisfied.

For example, a specific key may include an arrow key (e.g., ←, →, ↑, ↓) indicating a direction. For example, it may be assumed that the information about the stored arrangement direction indicates a case where the external electronic device (302) is placed on the left side of the electronic device (301), but in reality, the external electronic device (302) is placed on the right side of the electronic device (301). If the user selects the '→' arrow key while the indicator is displayed as the pointer reaches one edge (e.g., the right side) of the first display (360), the electronic device (301) may change the information about the stored arrangement direction. The information about the stored arrangement direction may be changed (or updated) to indicate a case where the external electronic device (302) is placed on the right side of the electronic device (301).

In one embodiment, the event detection module (323) can identify the user's gaze by capturing a user using the camera (380) within a specified time (e.g., within c seconds) and analyzing the captured image. To identify the user's gaze, the event detection module (323) can capture the user using a camera of an external electronic device (302) connected to the electronic device (301) and identify the user's gaze from the captured image. For example, when the event detection module (323) identifies that the identified user's gaze is directed toward the second display (361), the event detection module (323) can determine that at least one of the conditions is satisfied based on identifying that the direction of the user's gaze corresponds to the direction in which the pointer is moved past one edge of the first display (360) and toward the second display (361).

In one embodiment, the event detection module (323) may detect, using a timer, whether a pointer by an input device is positioned on the pointer for a specified period of time (e.g., d seconds) while the first display (360) is displaying the pointer. For example, in response to detecting that the pointer is positioned on the pointer until the timer expires, the event detection module (323) may determine that the at least one condition is satisfied. On the other hand, in response to not detecting that the pointer is positioned on the pointer before the timer expires or within the specified period of time (e.g., within c seconds), the event detection module (323) may request the first display (360) to deactivate the display of the pointer.

In one embodiment, the setting change module (324) may change the information about the stored arrangement direction in response to receiving an event from the event detection module (323) indicating that at least one condition is satisfied. For example, the setting change module (324) may automatically change the positional relationship between the first display (360) of the electronic device (301) and the second display (361) of the external electronic device (302) so that the set arrangement direction of the second display (361) corresponds to the direction in which the pointer is to move. For example, the setting change module (324) may store the arrangement relationship between the electronic device (301) and the external electronic device (302) in synchronization with the actual physical arrangement relationship without entering a setting screen for a program for sharing an input device (e.g., setting screen (200) of FIG. 2A).

According to one embodiment, the electronic device (101, 201, 301) may include a first display (160, 360), a communication circuit (190, 390), a memory (130, 330) storing instructions, and at least one processor (120, 320).

According to one embodiment, the instructions, when executed by the at least one processor, may be configured to cause the electronic device to, when a pointer displayed on the first display reaches an edge of the first display, identify, based on the set information about the arrangement direction, whether the direction in which the pointer is to be moved corresponds to the set arrangement direction of the second display.

In one embodiment, the instructions may be configured to cause the electronic device to identify whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved and the set arrangement direction of the second display do not correspond.

In one embodiment, the instructions may be configured to cause the electronic device to, based on satisfaction of the at least one condition, change the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmit a signal to the external electronic device through the communication circuit to move the pointer from the first display to the second display for display.

In one embodiment, the instructions may be configured to cause the electronic device to transmit a signal to the external electronic device through the communication circuit to move the pointer from the first display to the second display and display it, if the direction in which the pointer is to be moved corresponds to the set arrangement direction of the second display.

In one embodiment, the instructions may be configured to cause the electronic device to detect a designated input for moving the pointer past an edge of the first display toward the second display when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display, and to identify whether at least one condition for changing the set arrangement direction of the second display is satisfied in response to detection of the designated input.

In one embodiment, the instructions may be configured to cause the electronic device to detect the specified input in response to identifying that the pointer has been detected at an edge of the first display a specified number of times and/or for a specified period of time.

In one embodiment, the instructions may be configured to cause the electronic device to, in response to detection of the designated input, display an indicator including an edge of the first display, and, based on detecting that the pointer is moved over the indicator, identify whether at least one condition for changing a set arrangement orientation of the second display is satisfied.

In one embodiment, the instructions may be configured to cause the electronic device to identify that at least one condition for changing the set layout orientation of the second display is satisfied in response to detecting a designated keystroke for changing the set layout orientation of the second display.

In one embodiment, the instructions may be configured to cause the electronic device to identify that at least one condition for changing the set arrangement orientation of the second display is satisfied in response to a user's gaze being detected toward the second display using a camera of the electronic device or the external electronic device.

In one embodiment, the instructions may be configured to cause the electronic device to identify that at least one condition for changing the set orientation of the second display is satisfied in response to detecting that the pointer has been positioned on the indicator for more than a specified period of time.

In one embodiment, the instructions may be configured to cause the electronic device to count a time during which the pointer is positioned on the pointer based on detecting that the pointer is moved over the pointer, and to display the counted time on the first display.

In one embodiment, the instructions may be configured to cause the electronic device to display an object for connection with the external electronic device on at least a portion of an edge of the first display corresponding to a direction toward the external electronic device, and to transmit a signal to the external electronic device through the communication circuitry to move the pointer from the first display to the second display in response to a selection of the object.

FIG. 4 is a flowchart of operations of an electronic device for identifying a placement direction of an external electronic device according to an embodiment of the present invention. Referring to FIG. 4, the operation method may include operations 405 to 425. Each operation of the operation method of FIG. 4 may be performed by at least one of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, or the electronic device (301) of FIG. 3A), at least one processor of the electronic device (e.g., the processor (120) of FIG. 1 or the processor (320) of FIG. 3A). In an embodiment, at least one of operations 405 to 425 may be omitted, the order of some operations may be changed, or another operation may be added.

According to one embodiment, the electronic device (301) may display a movable pointer on a first display (e.g., the first display (360) of FIG. 3A). The pointer may be movable on the first display (360) in response to a user input received through at least one input device (e.g., the input device (350) of FIG. 3A) of the electronic device (301). For example, the pointer may be an object for visually indicating a location of a user input on the first display (360). In addition, the input device (e.g., the input device (350) of FIG. 3A) may include, but is not limited to, a mouse, a keyboard, a remote joystick, a game pad, or a remote controller.

According to one embodiment, the electronic device (301) may, in operation 405, identify whether a pointer displayed on the first display (360) reaches one edge of the first display (360). If the electronic device (301) identifies that the pointer does not reach one edge of the first display (360), the electronic device (301) may move and display the pointer according to a user input received on the first display (360).

According to one embodiment, the electronic device (301), in operation 410, may include an operation of identifying whether the direction in which the pointer is to be moved corresponds to the set arrangement direction of the second display (e.g., the second display (361) of FIG. 3A) of an external electronic device (e.g., the external electronic device (302) of FIG. 3A) based on identifying that a pointer displayed on the first display (360) has reached an edge of the first display (360).

According to one embodiment, the electronic device (301), in response to identifying that the direction in which the pointer is to be moved corresponds to the set arrangement direction of the second display (361), may, in operation 425, transmit a signal to the external electronic device (302) to move and display the pointer from the first display (360) to the second display (361). Accordingly, a pointer moving past one edge of the first display (360) toward the second display (361) may be displayed.

According to one embodiment, the electronic device (301), if the direction in which the pointer is to be moved and the set arrangement direction of the second display (361) do not correspond, in operation 415, may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied.

According to one embodiment, the electronic device (301) may, in operation 420, change the set arrangement direction of the second display (361) to correspond to the direction in which the pointer is to be moved, based on the satisfaction of the at least one condition.

According to one embodiment, the electronic device (301) may, in operation 425, transmit a signal to the external electronic device (302) to move the pointer from the first display (360) to the second display (361) and display it.

According to one embodiment, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied may include detecting a designated input for moving past one edge of the first display (360) toward the second display (361) when the direction in which the pointer is to be moved and the set arrangement direction of the second display (361) do not correspond. The electronic device (301) may identify, in response to detection of the designated input, that at least one condition for changing the set arrangement direction of the second display (361) is satisfied.

In one embodiment, the act of detecting the specified input may include detecting the specified input in response to identifying that the pointer has been detected at an edge of the first display (360) a specified number of times and/or for a specified period of time.

According to one embodiment, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied may include an operation of displaying an indicator including one edge of the first display (360) in response to detection of the specified input, and an operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied based on detecting that the pointer is moved onto the indicator.

According to one embodiment, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied may include an operation of identifying, in response to detecting a designated key input for changing the set arrangement direction of the second display (361), that at least one condition for changing the set arrangement direction of the second display (361) is satisfied.

According to one embodiment, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied in response to detecting a user's gaze toward the second display (361) using a camera of the electronic device (301) or the external electronic device (302).

According to one embodiment, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display (361) is satisfied in response to detecting that the pointer is positioned on the indicator for more than a specified time.

According to one embodiment, the method may include counting a time during which the pointer is positioned on the pointer based on detecting that the pointer is moved over the pointer, and displaying the counted time on the first display (360).

FIG. 5 is a flowchart illustrating an operation for identifying a first condition for changing an arrangement direction of an external electronic device according to an embodiment of the present invention. Referring to FIG. 5, an operation method may include operations 505 to 535. Each operation of the operation method of FIG. 5 may be performed by at least one of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, or the electronic device (301) of FIG. 3A), at least one processor of the electronic device (e.g., the processor (120) of FIG. 1 or the processor (320) of FIG. 3A). In an embodiment, at least one of operations 505 to 535 may be omitted, the order of some operations may be changed, or another operation may be added.

To help understand the description of FIG. 5, description will be made with reference to FIGS. 6A to 7. FIG. 6A is an exemplary diagram for explaining a method for identifying a designated input for moving toward an external electronic device past one edge of a display of an electronic device according to an embodiment, FIG. 6B is an exemplary diagram for explaining a first condition for changing the arrangement direction of an external electronic device according to an embodiment, and FIG. 7 is an exemplary diagram for explaining a case where arrangement settings for connected devices and actual arrangements correspond according to an embodiment.

According to one embodiment, the electronic device (301) may, in operation 505, display a movable pointer on the first display (360) of the electronic device (301).

In one embodiment, when the pointer reaches an edge of the first display (360), the electronic device (301) may, in operation 510, identify whether the pointer is detected at the edge of the first display (360) a specified number of times and/or a specified amount of time.

Referring to FIG. 6A, the electronic device (301) can check the current position (e.g., coordinates) of the pointer while the pointer is moved by the input device, and can identify whether it has reached one edge of the first display (360) based on the checked position. For example, it is assumed that the first display (360) is a display having a resolution of 1920 x 1080. When the pointer moves to the right on the first display (360), the electronic device (301) can check whether the x-coordinate value has reached 1920. In addition, when the pointer moves upward on the first display (360), the electronic device (301) can check whether the y-coordinate value has reached 0. In addition, when the pointer moves downward on the first display (360), the electronic device (301) can check whether the y-coordinate value has reached 1080.

For example, if the coordinates of the pointer are (1920, 268) as illustrated in FIG. 6A, the electronic device (301) can recognize that the x-coordinate value has reached 1920 and thus can recognize that the pointer has moved to the right end on the first display (360). For example, it can be determined that the pointer has reached one edge of the first display (360).

According to one embodiment, the electronic device (301) may, in operation 515, display an indicator including an edge of the first display (360) when the pointer is detected at an edge of the first display (360) exceeding a specified number of times and/or a specified time.

For example, if the number of times the x-coordinate value reaches 1920 occurs n or more times in a second, the electronic device (301) may display an indicator (625) on one edge of the first display (360) for a certain period of time (e.g., b seconds), as illustrated in FIG. 6B. For example, if the indicator (625) is generated on one edge (e.g., the right side) of the first display (360), assuming the width of the indicator (625) to be approximately 10px, the coordinate values representing the area of the indicator (625) may have (1910, 0, 1920, 1080). In this way, the indicator (625) may include a portion of one edge of the first display (360).

On the other hand, if the pointer is not detected on one edge of the first display (360) for a specified number of times and/or a specified time, the electronic device (301) may display a movable pointer on the first display (360) in operation 505.

According to one embodiment, the electronic device (301) may, in operation 520, identify whether the pointer is detected to have moved onto the indicator.

According to one embodiment, the electronic device (301), if it is detected that the pointer is moved onto the indicator, in operation 525, it can identify whether a designated key input for changing the set arrangement direction of the second display (361) of the external electronic device (302) is detected. As illustrated in FIG. 6b, when the pointer (620) is moved onto the indicator (625) at one edge of the first display (360), the electronic device (301) can identify whether there is a designated key input. For example, if it is detected that the pointer (620) is continuously positioned on the indicator (625) at one edge of the first display (360), the electronic device (301) can identify that a designated key input (or key combination) is received within a designated time (e.g., within c seconds). On the other hand, if it is not detected that the pointer has moved onto the indicator, the electronic device (301) may, in operation 535, deactivate the display of the indicator. For example, the electronic device (301) may remove the indicator so that it is no longer displayed when the pointer leaves the indicator. Accordingly, the electronic device (301) may perform an operation of checking the location according to the movement of the point.

According to one embodiment, in response to identifying that a designated key input for changing the set arrangement direction of the second display (361) is detected, the electronic device (301) may change the set arrangement direction of the second display (361) in operation 530. Referring to FIG. 7, as the set arrangement direction of the second display (361) is changed, the arrangement relationship of objects (710, 715) representing each device in the setting screen (700) and the arrangement relationship of the actual electronic device (301) and the external electronic device (302) may be changed to match. Accordingly, a pointer (620) may be moved and displayed on the second display (361) past one edge of the first display (360) of the electronic device (301) (720).

According to one embodiment, the electronic device (301) may, in operation 535, disable the display of the indicator after changing the set arrangement direction of the second display (361). Additionally, even if a designated key input for changing the set arrangement direction of the second display (361) is not detected in operation 525, the electronic device (301) may disable the display of the indicator in operation 535.

FIG. 8 is a flowchart illustrating an operation for identifying a second condition for changing an arrangement direction of an external electronic device according to an embodiment of the present invention. Referring to FIG. 8, the operation method may include operations 805 to 835. Each operation of the operation method of FIG. 8 may be performed by at least one of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, or the electronic device (301) of FIG. 3A), at least one processor of the electronic device (e.g., the processor (120) of FIG. 1 or the processor (320) of FIG. 3A). In an embodiment, at least one of operations 805 to 835 may be omitted, the order of some operations may be changed, or another operation may be added.

To help understand the description of Fig. 8, reference will be made to Fig. 9 for explanation. Fig. 9 is an example screen diagram for explaining a second condition for changing the arrangement direction of an external electronic device according to an embodiment of the present invention.

According to one embodiment, operations 805 to 820 of FIG. 8 may be identical to operations 505 to 520 of FIG. 5, and a detailed description thereof will be omitted.

According to one embodiment, the electronic device (301) may, in operation 805, display a movable pointer on the first display (360) of the electronic device (301).

In one embodiment, when the pointer reaches an edge of the first display (360), the electronic device (301) may, in operation 810, identify whether the pointer is detected at an edge of the first display (360) a specified number of times and/or a specified amount of time.

According to one embodiment, the electronic device (301) may, in operation 815, display an indicator including an edge of the first display (360) when the pointer is detected at an edge of the first display (360) exceeding a specified number of times and/or a specified time.

According to one embodiment, the electronic device (301) may, in operation 820, identify whether the pointer is detected to have moved onto the indicator.

According to one embodiment, the electronic device (301), when it is detected that the pointer is moved onto the indicator, in operation 825, may identify whether the user's gaze toward the second display (361) is detected using the camera (380). For example, as illustrated in FIG. 9, the electronic device (301) may capture a user (900) using the camera (380) for a specified time (e.g., c seconds) and analyze the captured image to identify the user's gaze (915). Here, the camera (380) may be positioned at the top center (905) of the display of the electronic device (301) to capture the user (900). In addition, the electronic device (301) may receive an image of the user (900) captured by the camera (380) of the electronic device (301) as well as the camera of an external electronic device (302) connected to the electronic device (301), and confirm the user's gaze (915) from the provided image.

When the electronic device (301) assumes that the line from the center (905) to the user (900) is the reference line (910), the electronic device (301) can identify the display toward which the user's gaze (915) is directed based on the reference line (910). For example, the electronic device (301) can identify whether the user's (900) gaze is directed toward the indicator as a result of analyzing the captured image, and can identify whether the user's (900) gaze corresponds to the direction of the indicator. If the user's (900) gaze corresponds to the direction of the indicator, the electronic device (301) can identify whether the display of the external electronic device (102) is at a position corresponding to the user's gaze (915). For example, it can identify whether the gaze analyzed from the image captured using the camera of the external electronic device (302) corresponds to (or faces) the user's gaze (915).

According to one embodiment, the electronic device (301) may change the set arrangement direction of the second display (361) in response to detecting the user's gaze toward the second display (361) in operation 830. For example, the arrangement direction of the external electronic device (302) is set so that the electronic device (301) controlling the input device through a program for sharing the input device is located on the right side of the external electronic device (302), but as illustrated in FIG. 9, since the electronic device (301) is located on the left side of the external electronic device (302), the stored arrangement relationship and the actual arrangement relationship may be different. Accordingly, when the user moves the pointer to one edge (e.g., the right side) of the first display (360) of the electronic device (301), since the stored arrangement relationship and the actual arrangement relationship are different, the electronic device (301) may generate and display an indicator on one edge (e.g., the right side) of the first display (360). The electronic device (301) can change the information about the stored arrangement relationship so that the stored arrangement relationship matches the actual arrangement relationship by checking the user's gaze (915) while the indicator is displayed. For example, the electronic device (301) can change and store information about the arrangement direction of the external electronic device (302) so that the electronic device (301) is located to the left of the external electronic device (302).

According to one embodiment, the electronic device (301) may disable the display of the indicator in operation 835 after changing the set arrangement direction of the second display (361) or simultaneously with changing the set arrangement direction of the second display (361). On the other hand, if the user's gaze toward the second display (361) is not detected in operation 825, the electronic device (301) may disable the display of the indicator in operation 835.

FIG. 10 is a flowchart illustrating an operation for identifying a third condition for changing an arrangement direction of an external electronic device according to an embodiment of the present invention. Referring to FIG. 10, an operation method may include operations 1005 to 1035. Each operation of the operation method of FIG. 10 may be performed by at least one of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, or the electronic device (301) of FIG. 3A), at least one processor of the electronic device (e.g., the processor (120) of FIG. 1 or the processor (320) of FIG. 3A). In an embodiment, at least one of operations 1005 to 1035 may be omitted, the order of some operations may be changed, or another operation may be added.

To help understand the description of Fig. 10, reference will be made to Fig. 11 for explanation. Fig. 11 is an example screen diagram for explaining a third condition for changing the arrangement direction of an external electronic device according to one embodiment.

According to one embodiment, operations 1005 to 1020 of FIG. 10 may be identical to operations 505 to 520 of FIG. 5, or operations 805 to 820 of FIG. 8, and a detailed description thereof will be omitted.

According to one embodiment, the electronic device (301) may, in operation 1005, display a movable pointer on the first display (360) of the electronic device (301).

In one embodiment, when the pointer reaches an edge of the first display (360), the electronic device (301) may, in operation 1010, identify whether the pointer is detected at an edge of the first display (360) a specified number of times and/or a specified amount of time.

According to one embodiment, the electronic device (301) may, in operation 1015, display an indicator including an edge of the first display (360) when the pointer is detected at an edge of the first display (360) a specified number of times and/or a specified time.

According to one embodiment, the electronic device (301) may, in operation 1020, identify whether the pointer is detected to have moved onto the indicator.

In one embodiment, the electronic device (301), in response to identifying that the pointer is moved onto the indicator, may, in operation 1025, identify whether the pointer is detected to be positioned on the indicator for more than a specified period of time.

For example, referring to FIG. 11, the electronic device (301) may identify whether a specified time has elapsed by using a timer in response to identifying that a pointer (620) is positioned on an indicator (625) displayed on the first display (360). The electronic device (301) may indicate the elapsed time by using an object (1100) representing a timer. For example, the electronic device (301) may display an object (1100) representing a timer for d seconds, and change the time for which the indicator (625) is displayed (e.g., b seconds) to, for example, b seconds - (indicator (625) entry time) + d seconds. If b = 5 and d = 3, the initial indicator (625) display time is 5 seconds, but if the user places the pointer (620) on the indicator (625) after 2 seconds, the indicator (625) display time can be 6 seconds (e.g., 5 - 2 + 3).

If the pointer (620) moves away from the indicator (625) before the expiration of the timer, the electronic device (301) can identify whether the pointer (620) is positioned on the indicator (625) again during the time that the indicator (625) is displayed, and if the pointer (620) re-enters the indicator (625), the operation of recalculating the indicator (625) display time can be repeated.

In one embodiment, the electronic device (301) may, in response to identifying that the pointer is positioned on the indicator for a specified period of time exceeding the specified time, change the set arrangement direction of the second display (361) in operation 1030. For example, the electronic device (301) may, in response to identifying that the pointer (620) continues to be positioned on the indicator (625) until the timer expires, change information about the stored arrangement relationship so that the stored arrangement relationship and the actual arrangement relationship match.

According to one embodiment, after changing the set arrangement direction of the second display (361), the electronic device (301) may, in operation 1035, disable the display of the indicator. On the other hand, even if it is not detected that the pointer is positioned on the indicator for more than a specified time in operation 1025, the electronic device (301) may disable the display of the indicator in operation 1035.

Figure 12 is an example screen for setting up a layout when initially connecting devices according to one embodiment.

According to one embodiment, a user can set up a placement relationship through a program for sharing input devices (e.g., the setting screen (200) of FIG. 2a), but the placement relationship can also be set up at the time of initial connection between each device. In FIG. 12, it can be assumed that an external electronic device (302) is placed on the right side of an electronic device (301).

Referring to 1200a of FIG. 12, when the electronic device (301) is initially connected to an external electronic device (302), the electronic device (301) may set the placement direction (or connection direction) of the external electronic device (302) with which the input device will be shared. For example, the electronic device (301) may receive an input for moving a pointer (1220) displayed through the first display (360) of the electronic device (301) toward one edge of the first display (360). Here, one edge of the first display (360) may correspond to a position where the external electronic device (302) is placed or a direction toward the external electronic device (302).

The electronic device (301) may display an indicator (1225) on a portion of an edge of the first display (360), as illustrated in 1200b, in response to an input of moving the pointer (1220) in a first direction (1205) (e.g., toward the right). The electronic device (301) may display an indicator (1225) in response to an input of moving the pointer (1220) toward an edge of the first display (360), wherein the indicator (1225) may be used to indicate that a user can set a placement direction (or connection direction) of the external electronic device (302).

As illustrated in 1200c, the electronic device (301) may, in response to an input of moving the pointer (1220) toward one edge of the first display (360), output (or display) a notification (1230) including content asking whether to establish a communication connection to share an input device with an external electronic device (302) and/or an item indicating whether to establish the communication connection (e.g., accept the connection or cancel the connection). If an object (1235) indicating accept the connection is selected, the electronic device (301) may display a notification (1240) indicating that the external electronic device (302) can be controlled by sharing the input device, as illustrated in 1200d.

As described above, according to one embodiment, when an electronic device (301) is first connected to an external electronic device (302) with which to share an input device, the user can directly set (or change) the orientation of the external electronic device (302) relative to the electronic device (301) to fit its physical location without using a program for sharing the input device.

FIG. 13 is a screen example diagram showing an object for changing the arrangement direction of an external electronic device according to an embodiment of the present invention.

In one embodiment, a user can set the placement relationship upon initial connection between each device, but even after setting the placement information through a program for sharing input devices, the user can directly set (or change) the placement direction of the external electronic device (302) relative to the electronic device (301) to match the physical location.

As illustrated in 1300a, the electronic device (301) may display an indicator (1325) on a portion of an edge of the first display (360), as illustrated in 1300b, in response to an input of moving the pointer (1320) in a first direction (e.g., toward the right) toward one edge of the first display (360).

According to one embodiment, the pointer (1325) may include an object (1330) for setting the arrangement direction (or connection direction) of the external electronic device (302) based on the electronic device (301). If a selection for the object (1330) is received using the pointer (1320), the electronic device (301) may set (or change) the arrangement direction of the external electronic device (302) in the direction indicated by the object (1330). According to one embodiment, even if the arrangement relationship between the electronic device (301) and the external electronic device (302) set through a program for sharing an input device is different from the actual arrangement, the arrangement direction of the external electronic device (302) may be set (or changed) in the direction indicated by the object (1330), thereby providing a continuous user experience between the electronic device (301) and the external electronic device (302).

Electronic devices according to various embodiments disclosed in this document may be devices of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliance devices. Electronic devices according to embodiments of this document are not limited to the above-described devices.

It should be understood that the various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly dictates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first) is referred to as "coupled" or "connected" to another (e.g., a second) component, with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integrally configured component or a minimum unit of the component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store ^TM ) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities, and some of the multiple entities may be separately arranged in other components. According to various embodiments, one or more of the components or operations of the above-described components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, the multiple components (e.g., a module or a program) may be integrated into one component. In such a case, the integrated component may perform one or more functions of each of the multiple components identically or similarly to those performed by the corresponding component of the multiple components before the integration. According to various embodiments, the operations performed by the module, program, or other component may be executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

According to one embodiment, a storage medium storing at least one computer-readable instruction, wherein the at least one instruction, when executed by at least one processor (320) of an electronic device (301), causes the electronic device to perform at least one operation, wherein the at least one operation may include an operation of identifying, based on set information about an arrangement direction of a second display (361) of an external electronic device (302) with respect to the first display, whether the direction in which the pointer is to be moved corresponds to a set arrangement direction of the second display when a pointer displayed on a first display (360) of the electronic device reaches an edge of the first display.

In one embodiment, the at least one operation may include an operation of identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display.

According to one embodiment, the at least one operation may include, based on satisfaction of the at least one condition, changing the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display.

Claims (15)

In an electronic device (301), First display (360); Communication circuit (390); Memory (330) for storing instructions; and comprising at least one processor (320), The above instructions, when executed by the at least one processor, cause the electronic device to: When the pointer displayed on the first display reaches one edge of the first display, based on the set information about the arrangement direction of the second display (361) of the external electronic device (302) with respect to the first display (360) stored in the memory (330), it is identified whether the direction in which the pointer is to move corresponds to the set arrangement direction of the second display. If the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display, identify whether at least one condition for changing the set arrangement direction of the second display is satisfied, An electronic device, wherein, based on satisfaction of at least one of the above conditions, the set arrangement direction of the second display is changed to correspond to the direction in which the pointer is to be moved, and a signal for moving and displaying the pointer from the first display to the second display is transmitted to the external electronic device through the communication circuit. In the first paragraph, the instructions cause the electronic device to: An electronic device set to transmit a signal to the external electronic device through the communication circuit to move the pointer from the first display to the second display and display it when the direction in which the pointer is to be moved and the set arrangement direction of the second display correspond. In claim 1 or 2, the instructions cause the electronic device to: If the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display, detecting a designated input for moving past one edge of the first display toward the second display, An electronic device configured to identify whether at least one condition for changing the set arrangement direction of the second display is satisfied in response to detection of the specified input. In any one of claims 1 to 3, the instructions cause the electronic device to: An electronic device configured to detect said specified input in response to identifying that said pointer has been detected at an edge of said first display for a specified number of times and/or for a specified period of time. In any one of claims 1 to 4, the instructions cause the electronic device to: In response to detection of the above specified input, displaying an indicator including one edge of the first display, An electronic device configured to identify whether at least one condition for changing the set arrangement direction of the second display is satisfied based on detecting that the pointer is moved onto the indicator. In any one of claims 1 to 5, the instructions cause the electronic device to: An electronic device configured to identify that at least one condition for changing the set layout orientation of the second display is satisfied in response to detecting a designated key input for changing the set layout orientation of the second display. In any one of claims 1 to 6, the instructions cause the electronic device to: An electronic device configured to identify that at least one condition for changing a set arrangement direction of the second display is satisfied in response to detecting a user's gaze toward the second display using a camera of the electronic device or the external electronic device. In any one of claims 1 to 7, the instructions cause the electronic device to: An electronic device configured to identify that at least one condition for changing the set arrangement direction of the second display is satisfied in response to detecting that the pointer has been positioned on the indicator for a period of time exceeding a specified period of time. In any one of claims 1 to 8, the instructions cause the electronic device to: Based on detecting that the above pointer has moved onto the above indicator, the time during which the pointer is positioned on the indicator is counted, An electronic device set to display the counted time on the first display. In any one of claims 1 to 9, the instructions cause the electronic device to: Displaying an object for connection with the external electronic device on at least a portion of an edge of the first display corresponding to a direction toward the external electronic device, An electronic device configured to transmit a signal to the external electronic device through the communication circuit in response to a selection of the object, to move the pointer from the first display to the second display and display it. In a method for identifying the placement direction of an external electronic device (302) sharing an input device (350) in an electronic device (301), An operation of identifying whether the direction in which the pointer is to move corresponds to the set arrangement direction of the second display (361) of the external electronic device (302), based on set information about the arrangement direction of the second display (361) of the external electronic device (302) with respect to the first display, when the pointer displayed on the first display (360) of the electronic device reaches one edge of the first display; An operation for identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display; and A method for identifying a placement direction of an external electronic device sharing an input device, comprising: an operation of changing a set placement direction of the second display to correspond to a direction in which the pointer is to be moved, and transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display, based on satisfaction of at least one of the above conditions. A method for identifying the arrangement direction of an external electronic device sharing an input device, further comprising an action of transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display when the direction in which the pointer is to be moved and the set arrangement direction of the second display correspond to each other in the 11th paragraph. In claim 11 or 12, the operation of identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied is, An operation of detecting a designated input for moving past one edge of the first display toward the second display when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display; and A method for identifying a layout orientation of an external electronic device sharing an input device, comprising: an operation for identifying that at least one condition for changing a set layout orientation of the second display is satisfied in response to detection of the specified input. In any one of claims 11 to 13, the operation of detecting the specified input comprises: A method for identifying an orientation of an external electronic device sharing an input device, the method comprising: detecting the specified input in response to identifying that the pointer has been detected at an edge of the first display for a specified number of times and/or for a specified period of time. A storage medium storing at least one computer-readable instruction, wherein the at least one instruction, when executed by at least one processor (320) of an electronic device (301), causes the electronic device to perform at least one operation, the at least one operation comprising: An operation of identifying whether the direction in which the pointer is to move corresponds to the set arrangement direction of the second display (361) of the external electronic device (302), based on set information about the arrangement direction of the second display (361) of the external electronic device (302) with respect to the first display, when the pointer displayed on the first display (360) of the electronic device reaches one edge of the first display; An operation for identifying whether at least one condition for changing the set arrangement direction of the second display is satisfied when the direction in which the pointer is to be moved does not correspond to the set arrangement direction of the second display; and A storage medium including an operation of changing the set arrangement direction of the second display to correspond to the direction in which the pointer is to be moved, and transmitting a signal to the external electronic device for moving and displaying the pointer from the first display to the second display, based on satisfaction of at least one of the above conditions.

Images