WO2026010115A1 - Electronic device for providing summary information, operating method thereof, and storage medium - Google Patents

Abstract

According to an embodiment, a foldable electronic device may comprise: a foldable housing including a first housing, a second housing, and a third housing, wherein the second housing is disposed between the first housing and the third housing; a flexible display including a first portion supported by the first housing, a second portion supported by the second housing, and a third portion supported by the third housing; at least one processor; and memory storing instructions. The instructions may be configured to cause the foldable electronic device to: display first information on the first portion, second information on the second portion, and third information on the third portion; generate at least one piece of summary information corresponding to the second information and the third information by using a machine learning model, at least partially on the basis of detecting that the third housing is folded to face the second housing; and display the at least one piece of summary information on the first portion.

Description

Electronic device providing summary information, method of operation thereof and storage medium

One embodiment disclosed in this document relates to an electronic device for providing summary information, a method of operation thereof, and a storage medium.

As the variety of services and additional features offered through electronic devices such as smartphones continues to increase, a variety of applications capable of running on these devices are being developed. Furthermore, the hardware and/or software components of these devices are also continuously evolving.

In line with the recent consumer trend of placing importance on design, in the development of electronic devices, the convenience of use is also given importance along with the external design of the electronic device.

For example, electronic devices may adopt novel form factors, such as dual-display electronic devices (e.g., foldable electronic devices). Foldable electronic devices feature a foldable (or bendable) display (e.g., a flexible display) that can be used in a folded or unfolded state. Accordingly, the foldable electronic device can adaptively operate its display to respond to its folded (or folding) state or unfolded (or unfolding) state.

The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

According to one embodiment, a foldable electronic device comprises a foldable housing, the foldable housing including a first housing, a second housing, and a third housing, the second housing being disposed between the first housing and the third housing, a first hinge rotatably connecting the third housing to the second housing, a second hinge rotatably connecting the first housing to the second housing and having a wider width than the first hinge, a flexible display disposed within the first housing, the second housing, and the third housing, the flexible display including a first portion supported by the first housing, a second portion supported by the second housing, and a third portion supported by the third housing, and at least one processor including a processing circuit and a memory storing instructions.

According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may be configured to cause the foldable electronic device to display first information in the first portion, second information in the second portion, and third information in the third portion.

In one embodiment, the instructions may be configured to cause the foldable electronic device to generate at least one summary piece of information corresponding to the second information and the third information using a machine learning model, at least in part based on detecting that the third housing is folded to face the second housing, wherein the machine learning model may include a machine learning model trained to output summarized information based on text or image input.

According to one embodiment, the instructions may be configured to cause the foldable electronic device to display the at least one summary information in the first portion.

According to one embodiment, a foldable electronic device comprises a foldable housing, the foldable housing including a first housing, a second housing, and a third housing, the second housing being disposed between the first housing and the third housing, a first hinge rotatably connecting the third housing to the second housing, a second hinge rotatably connecting the first housing to the second housing and having a wider width than the first hinge, a flexible display disposed within the first housing, the second housing, and the third housing, the flexible display including a first portion supported by the first housing, a second portion supported by the second housing, and a third portion supported by the third housing, and at least one processor including a processing circuit and a memory storing instructions.

According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may be configured to cause the foldable electronic device to display first information in the first portion, second information in the second portion, and third information in the third portion.

In one embodiment, the instructions may be configured to cause the foldable electronic device to generate at least one summary information corresponding to the third information using a generative AI model, at least in part based on detecting that the third housing is folded relative to the second housing, wherein the generative AI model may include a generative AI model that outputs summarized information or a new image based on text or image input.

According to one embodiment, the instructions may be configured to cause the foldable electronic device to display the at least one summary information in the second portion or the third portion.

According to one embodiment, a method for providing summary information in a foldable electronic device may include displaying first information on a first portion of a flexible display disposed within a first housing of a foldable housing, second information on a second portion of a flexible display disposed within a second housing of the foldable housing, and third information on a third portion of the flexible display disposed within a third housing of the foldable housing, wherein the second housing may be disposed between the first housing and the third housing.

According to one embodiment, the method comprises generating at least one summary piece of information corresponding to the second information and the third information using a machine learning model, based at least in part on detecting that the third housing is folded against the second housing, wherein the machine learning model may be trained to output summarized information based on text or image input.

In one embodiment, the method may include an action of displaying the at least one summary information in the first portion.

According to one embodiment, in a non-transitory storage medium storing instructions, the instructions, when executed by at least one processor (120, 220) of an electronic device (101), are set to cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of displaying first information on a first portion of a flexible display disposed within a first housing of a foldable housing, second information on a second portion of a flexible display disposed within a second housing of the foldable housing, and third information on a third portion of the flexible display disposed within a third housing of the foldable housing, wherein the second housing may be disposed between the first housing and the third housing.

In one embodiment, the at least one operation comprises generating at least one summary information corresponding to the second information and the third information using a machine learning model, based at least in part on detecting that the third housing is folded against the second housing, wherein the machine learning model may be trained to output summarized information based on text or image input.

In one embodiment, the at least one action may include an action of displaying the at least one summary information in the first portion.

FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

Figure 2 is an internal block diagram of an electronic device according to one embodiment.

FIG. 3A is a diagram illustrating an unfolded state on the front side of an electronic device according to one embodiment.

FIG. 3b is a drawing illustrating an unfolded state at the rear of an electronic device according to one embodiment.

FIG. 3c is a drawing illustrating a Z-type folding state of an electronic device according to one embodiment.

FIG. 3D is a side view showing a folded state of an electronic device according to one embodiment.

FIG. 3e is a drawing illustrating a G-type folding state of an electronic device according to one embodiment.

FIG. 4 is a flowchart illustrating an operation of an electronic device for providing summary information according to an embodiment.

FIG. 5 is a drawing showing detailed operation of an electronic device according to one embodiment.

FIG. 6 is a drawing showing a first folding state in an unfolded state of an electronic device according to one embodiment.

FIG. 7 is a drawing showing a second folding state in an unfolded state of an electronic device according to one embodiment.

FIG. 8 is a drawing showing a third folding state in a folded state of an electronic device according to one embodiment.

FIG. 9 is a drawing showing a fourth folding state of an electronic device according to an embodiment.

FIG. 10 is a drawing showing a fifth folding state of an electronic device according to an embodiment.

FIG. 11 is a diagram illustrating a method for providing summary information while a first app is running according to one embodiment.

FIG. 12A is a diagram illustrating a method for providing summary information in a first situation while a first app and a second app are running according to one embodiment.

FIG. 12b is a diagram illustrating a method for providing summary information in a second situation while a first app and a second app are running according to an embodiment.

FIG. 12c is a diagram illustrating a method for providing summary information in a third situation while a first app and a second app are running according to one embodiment.

FIG. 13 is a diagram illustrating a method for providing summary information while a first app to a third app are running according to one embodiment.

Figure 14 is a drawing for explaining an example of displaying summary information according to one embodiment.

FIG. 15 is a drawing for explaining an example of summary information displayed differently depending on a folding operation according to one embodiment.

FIG. 16 is a diagram illustrating a case of returning according to a user input for summary information according to one embodiment.

FIG. 17A is a side view of a G-type foldable electronic device according to one embodiment.

FIG. 17b is a drawing showing an unfolded state of a G-type foldable electronic device according to one embodiment.

FIG. 18 is a drawing for explaining an example of displaying summary information in a G-type foldable electronic device according to one embodiment.

FIG. 19 is a side view of an e-type foldable electronic device according to one embodiment.

FIG. 20 is a drawing for explaining an example of displaying summary information in an e-type foldable electronic device according to one embodiment.

FIG. 21 is a drawing for explaining an example of displaying summary information in a slideable electronic device according to one embodiment.

FIG. 22 is a drawing for explaining an example of displaying summary information in a single foldable electronic device according to one embodiment.

FIG. 23 is a diagram illustrating an example of providing additional functions when summarizing two or more apps according to one embodiment.

FIG. 24 is a diagram showing an example of providing summary information when an app is running in a multi-window state in an unfolded state according to one embodiment.

Figure 25 is a drawing for explaining an example of displaying summary information according to one embodiment.

FIG. 26 is a diagram illustrating the execution of on-device AI in response to a selection of summary information according to one embodiment.

FIG. 27 is a drawing for explaining a single display example of summary information by a unfolding action according to one embodiment.

FIG. 28 is a drawing for explaining a plurality of display examples of summary information by an unfolding operation according to one embodiment.

Fig. 29 is a drawing showing a mounting state of an electronic device according to one embodiment.

FIG. 30 is a diagram showing a first screen when utilizing an AI assistant according to an embodiment.

FIG. 31 is a diagram showing a second screen when utilizing an AI assistant according to one embodiment.

Figure 32 is a drawing for explaining a method of displaying summary information according to one embodiment.

FIG. 33 is a diagram for explaining a case where summary information is not displayed according to one embodiment.

FIG. 34 is a diagram for explaining an example of displaying summary information according to the priority of an application according to one embodiment.

In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers. The artificial neural network can be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model can additionally or alternatively include a software structure.

The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and/or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199). The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

In the detailed description below, reference numerals in the drawings may be used interchangeably or omitted for components that can be easily understood through the preceding embodiments, and their detailed descriptions may also be omitted. The electronic device (101) according to one embodiment disclosed in this document may be implemented by selectively combining components of different embodiments, and components of one embodiment may be replaced by components of another embodiment. For example, it should be noted that the present invention is not limited to specific drawings or embodiments.

An electronic device may have a deformable structure capable of adjusting the size of the display to satisfy the portability and usability of the electronic device. An electronic device having a deformable structure may include a foldable electronic device in which at least two housings operate in a manner in which they fold or unfold relative to each other.

The electronic device (101) may have a display portion that is visually visible (or exposed) to the user that changes depending on the transition between folding states. In line with recent consumer trends that prioritize design, the design of graphic objects, such as background images, icons, and widgets displayed on the display, along with the external design of the electronic device, is being given significant consideration in the development of electronic devices.

As electronic devices change the display portion that is activated depending on the transition between folding states, it may be important to provide users with convenience in interacting with the electronic device as well as continuity in using applications.

In one embodiment, an electronic device, a method of operating the same, and a storage medium for providing summary information may be provided, such that when a portion of a display that is activated (or visually exposed to a user) is deactivated (or not visually exposed) in response to a change in a folding state of the electronic device, information including text and/or images that were output (or displayed, reproduced) on the portion of the display may be summarized and displayed on another portion of the activated display.

According to one embodiment, by summarizing information output through a portion of a display that is deactivated according to the switching of the folding state of the electronic device and displaying it on another portion of the display that is activated, the user can recognize (or confirm) the contents of the application being used, and by selecting the displayed summary information, the user can easily switch to the application at a desired time.

In one embodiment, the usability of a multi-foldable electronic device can be improved by displaying a summary of content displayed on a folded and invisible display on a portion of the display visible to the user.

In one embodiment, by utilizing a portion of the display visible to the user when folding a multi-foldable electronic device, the personalization and reproducibility strengths of on-device AI can be highlighted, and new actions can be executed through mechanical triggers such as folding, intuitively confirming from the user's perspective that folding can be utilized as a starting point for a subsequent process rather than simply an end to use.

FIG. 2 is an internal block diagram (200) of an electronic device according to an embodiment. In describing FIG. 2, reference will be made to FIGS. 3A to 3E. FIG. 3A is a diagram illustrating an unfolded state at the front of the electronic device according to an embodiment, FIG. 3B is a diagram illustrating an unfolded state at the rear of the electronic device according to an embodiment, FIG. 3C is a diagram illustrating a Z-type folding state of the electronic device according to an embodiment, FIG. 3D is a side view illustrating a folding state of the electronic device according to an embodiment, and FIG. 3E is a diagram illustrating a G-type folding state of the electronic device according to an embodiment.

Referring to FIG. 2, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may include a display (260) (e.g., the display module (160) of FIG. 1), at least one processor (220) (e.g., the processor (120) of FIG. 1), and a memory (230) (e.g., the memory (130) of FIG. 1). According to an embodiment, the electronic device (201) may further include a sensor circuit (276) (e.g., the sensor module (176) of FIG. 1) and a communication circuit (290) (e.g., the communication module (190) of FIG. 1). The at least one processor (220) may include processing circuitry. Here, not all components illustrated in FIG. 2 are essential components of the electronic device (201), and the electronic device (201) may be implemented by more or fewer components than the components illustrated in FIG. 2.

According to one embodiment, the memory (230) (e.g., the memory (130) of FIG. 1) may store at least one command (or instruction) that causes at least one operation of the electronic device (201). The at least one instruction, when executed by the processor (220), may cause the electronic device (201) to perform a corresponding operation. The memory (230) may store information such as data processing contents of the processor (220) and learning data of an on-device AI model.

According to one embodiment, the processor (220) may detect a transition of the folding state of the electronic device (201) using at least one sensor included in the sensor circuit (276). For example, the processor (220) may detect a transition of the electronic device (201) from a fully unfolded state (e.g., the unfolded state of FIG. 3A) to a folded state along at least one folding axis. Based on detecting the transition of the folding state, the processor (220) may identify an area of the display that is activated (or visually exposed or visible to the user) (e.g., a main display area) and another area of the display that is deactivated (or not visually exposed or invisible) (a sub display area).

According to one embodiment, the processor (220) may drive on-device AI (e.g., machine learning model, generative AI model) to generate summary information upon detecting a transition in the folding state.

According to one embodiment, the processor (220) may generate information (or data) summarizing content (e.g., original content) including text and/or images that were output (or displayed, played) on a portion of the display using a machine learning model when a portion of the activated display is deactivated. For example, the processor (220) may identify a display area that has been folded and become invisible in response to a transition in the folding state, and the machine learning model may transmit information summarizing and organizing the content displayed on the identified display area to the processor (220).

The original content corresponds to the application's execution screen and may include text and/or images. Furthermore, the original content may include audio, video content captured by the user, or video content such as dramas or movies downloaded or streamed from at least one server (e.g., a content provider). Here, the machine learning model may be trained to output summarized text based on text and/or image input.

According to one embodiment, the processor (220) may determine (or identify) a portion to be summarized from the original content using another machine learning model. The processor (220) may determine whether to generate a summary by the other machine learning model, and based on the determination to generate the summary, determine the portion to be summarized and generate summary information. The other machine learning model may learn a user usage pattern for the electronic device (201) and generate new summary information as an inference result when determining a portion to be summarized from the original content including text and/or images output (or displayed, reproduced) on a portion of the display.

According to one embodiment, the processor (220) may generate new information based on original content including text and/or images that were output (or displayed, reproduced) on a portion of the display using a generative artificial intelligence (AI) model. The generative AI model may output (or generate) an answer to a request requested through a prompt based on being provided (or input) with the original content (e.g., text and/or images) and a prompt.

For example, the generated information may be edited and processed portions of the previously output texts and/or images. Alternatively, the generated information may include new text and/or images not included in the original text. For example, text and/or images edited or added based on the original content may be acquired, modified, or generated using a generative AI model.

In one embodiment, the machine learning model may include a generative AI model trained using a neural network structure to generate and output summarized text or images based on a prompt input. For example, an on-device AI model (or AI engine), such as a machine learning model, another machine learning model, and/or a generative AI model, may be stored in memory (230) or stored in an external device (e.g., a server (108)).

Meanwhile, in order to implement the present invention and operate it in an electronic device (201), hardware capable of performing multiple parallel operations, such as a GPU (Graphics Processing Unit) or an NPU (Neural Network Processing Unit), may be required. Since most AI models, as well as generative AI models, must process a large number of arithmetic operations simultaneously, hardware capable of performing parallel operations other than a CPU (Central Processing Unit) may be used, and this may be included in the electronic device (201) as a separate component (or module, circuit). Although a processor (220) is illustrated in FIG. 2, an on-device AI model may be included in the processor (220) in the form of hardware or software blocks.

The above NPU may refer to a structure designed into a chip to perform operations (e.g., convolution, ReLU, Linear, BatchNorm, LayerNorm, Upsample, Transpose) for performing AI models, while the GPU may refer to a structure capable of performing many parallel operations based on a large number of cores. Typically, an NPU may have a structure designed to target specific tasks (e.g., segmentation, tracking).

According to one embodiment, the processor (220) may display the summarized information on another portion of the display that is activated (or visually exposed to the user). In one embodiment, as the portion of the display that is activated changes depending on the transition from the folding state to the active state, the content displayed on the portion of the display before it was switched from the active state to the inactive state is summarized and displayed on the portion of the display that is activated, thereby providing continuity so that the user can continuously check the screen contents of multiple applications.

According to one embodiment, the processor (220) may also control to display the original content of the display area that has become invisible in a summarized form in the visible display area even when the condition is satisfied that the display area shown to the user changes due to a change in the posture (or position) of the electronic device (201), such as a change in the folding state or a rotation or flipping of the electronic device (201).

According to one embodiment, in addition to the posture of the electronic device (201), the display area shown to the user may also change when the user's position changes while at least a portion of the display (260) is placed in a folded state. Therefore, the conditions for changing the display area shown to the user may vary and may not be limited thereto.

According to one embodiment, the sensor circuit (276) may include at least one sensor that generates an electrical signal or data value corresponding to the operating state of the electronic device (201) or an external environmental state. According to one embodiment, the at least one sensor may include at least one of a proximity sensor, an illuminance sensor, a grip sensor, a capacitive touch sensor, a Hall IC, a gyro sensor used to measure the horizontality and direction of the electronic device (201), or an acceleration sensor.

According to one embodiment, the sensor circuit (276) may include a sensor (e.g., a digital Hall sensor, a 9-axis sensor) for determining the operating state (e.g., an unfolded state, a folded state, and/or an intermediate state) and the folded angle of the electronic device (201). The Hall sensor (or Hall IC) according to one embodiment may include one or more magnets and/or one or more magnetic sensors. At least one of the one or more magnets included in the Hall sensor and/or the one or more magnetic sensors may be arranged at different locations within the electronic device (201).

The positional relationship of the one or more magnets and/or the one or more magnetic sensors within the electronic device (201) may change depending on the form of the electronic device (201). The electronic device (201) may measure a change in the positional relationship through the one or more magnetic sensors. The change in the positional relationship may cause a change in the magnetic field formed by the one or more magnets. The electronic device (201) may obtain a power signal indicating a change in the magnetic field using a Hall sensor. For example, the electronic device (201) may distinguish a state (e.g., an unfolded state, a folded state, and/or an intermediate state) of the electronic device (201) using the power signal obtained from the Hall sensor. For example, the Hall sensor may output data indicating a folding state of the display (260), and the electronic device (201) may receive data indicating a state of the electronic device (201) from the Hall sensor.

In one embodiment, the state of the electronic device (201), for example, the posture (or location), may be identified using the state in which the electronic device (201) is held. The display area shown to the user may vary depending on which part of the electronic device (201) the user holds and uses with both hands or one hand. Therefore, a grip sensor may be used to detect the state in which the user holds the electronic device (201), but a touch sensor may also be used to identify which display area the user is holding in contact with. The touch sensor may be included in the display (260) and may provide a signal to the processor (220) regarding a specific position of the display (260) according to the user's holding (for example, at least one of the first display area (or first part) (261), the second display area (or second part) (262), and the third display area (or third part) (263)).

According to one embodiment, the display (260) may be a flexible display including a first display area (or first part) (261), a second display area (or second part) (262), and a third display area (or third part) (263). The display (260) is a display area where an application screen is displayed, and may be divided into a first display area (261), a second display area (262), and a third display area (263) by being partially folded, for example, by a folding axis (e.g., axes B and C in FIG. 3A), as illustrated in FIG. 3A. Although the number of display areas is exemplified as three in FIG. 3A, the number is not limited thereto and may vary depending on the type of the flexible display. For example, the number of areas of the flexible display may increase to a fourth display area or a fifth display area, or may decrease to a first display area or a second display area.

The shape of the display (260) may change as it is folded or unfolded along a folding axis (e.g., axes B and C in FIG. 3A). For example, a Hall sensor may output different data indicating a folding state of the display (260). The data indicating the folding state of the display (260) may include an angle of the display (260) folded along the folding axis. For example, the electronic device (201) may distinguish a partially folded state (e.g., a flex mode) of the display (260) in response to identifying an angle of the display (260) folded along the folding axis that is within a specified angular range (e.g., an angular range between 10° and 160°).

According to one embodiment, the sensor circuit (276) may include a gyro sensor, an acceleration sensor, and/or a magnetometer sensor to detect the state (e.g., attitude) of the electronic device (201). For example, the electronic device (201) may determine a relative azimuth with respect to a reference azimuth using the magnetometer sensor. The electronic device (201) may obtain a final azimuth using UWB by using the relative azimuth with respect to the reference azimuth and the UWB angle of arrival (AOA) using the magnetometer sensor. According to one embodiment, the sensor circuit (276) may be used as a 9-axis motion sensor by using the gyro sensor, the acceleration sensor, and the magnetometer sensor. In addition to the above, the sensor circuit (276) may also include a strain sensor that outputs a strain value used to indirectly measure the folded angle of the electronic device (201).

According to one embodiment, the communication circuit (290) may include an Ultra-wideband (UWB) communication circuit. The UWB communication circuit may measure the distance between the electronic device (201) and the user through UWB communication. The UWB communication circuit may measure the direction (e.g., angle of arrival (AOA)) of the user using a plurality of antennas. For example, when outputting content including text or images in a folded state of at least a portion of the electronic device (201) (or the display (260)), the processor (220) may identify the location of the user through UWB communication, thereby identifying (or determining) a display area (e.g., a first display area (261), a second display area (262), and/or a third display area (263)) on which summary information will be adaptively displayed to correspond to the location of the user.

FIG. 3A is a drawing illustrating a front side of an electronic device (201) according to an embodiment, and FIG. 3B is a drawing illustrating a rear side of an electronic device (201) according to an embodiment. The electronic device (201) may include a first housing (303), a second housing (305), a third housing (307), and a display (260). The electronic device (201) may include at least some of the structures and/or functions of the electronic device (101) of FIG. 1.

In one embodiment, the first housing (303) and the second housing (305) may be arranged on both sides with respect to a folding axis (e.g., C-axis) and may have a shape symmetrical with respect to the folding axis (e.g., C-axis). The second housing (305) and the third housing (307) may be arranged on both sides with respect to a folding axis (e.g., B-axis) and may have a shape symmetrical with respect to the folding axis (e.g., B-axis). The first housing (303) and the second housing (305), and the second housing (305) and the third housing (307) may be configured to fold with respect to each other. The first housing (303), the second housing (305), and the third housing (307) may be folded and overlapped with respect to the folding axis (e.g., B, C-axis) through at least one hinge device (e.g., a first hinge (310), a second hinge (320)).

In one embodiment, referring to FIG. 3C, the first housing (303) and the second housing (305) may be operated in an out-folding manner in which they are rotated relative to each other about the C-axis. The second housing (305) and the third housing (307) may be operated in an in-folding manner in which they are rotated relative to each other about the B-axis. For example, a case in which the first housing (303) and the second housing (305) are operated in an out-folding manner, and the second housing (305) and the third housing (307) are operated in an in-folding manner may be referred to as a triple zigzag foldable or a Z-foldable. In one embodiment, folding may be extended to other foldable electronic devices in addition to the triple zigzag foldable. For example, the arrangement for outfolding and infolding is not limited to the embodiments, and the foldable electronic device may include an electronic device in which the non-display area and display area are increased or expanded in various ways, such as an outfolding method applied and folded inward, a method in which two or more folding areas are formed, or a sliding method.

In one embodiment, the first housing (303) and the second housing (305) are operated in an infolding manner that rotates relative to each other about the C-axis, and the second housing (305) and the third housing (307) are also operated in an infolding manner that rotates relative to each other about the B-axis, which may be referred to as a G-type foldable.

In one embodiment, the first housing (303) may include a first face (321) and a second face (322) facing in an opposite direction to the first face (321), the second housing (305) may include a third face (323) and a fourth face (324) facing in an opposite direction to the third face (323), and the third housing (307) may include a fifth face (325) and a sixth face (326) facing in an opposite direction to the fifth face (325). For example, referring to FIG. 3C, when the display (260) of the Z-type foldable electronic device (201) is folded (e.g., in-folded) about the B-axis, the fifth face (325) of the third housing (307) and the third face (323) of the second housing (305) may face each other. When the display (260) is folded (e.g., out-folded) based on the C-axis, the fourth side (324) of the second housing (305) and the second side (322) of the first housing (303) can be faced to face each other.

For example, referring to FIG. 3e, when the display (260) of the G-type foldable electronic device (201) is folded (e.g., in-folded) about the C-axis, the first surface (321) of the first housing (303) may be faced to face the third surface (323) of the second housing (305). When the display (260) is folded (e.g., in-folded) about the B-axis, the fifth surface (325) of the third housing (307) and the second surface (322) of the first housing (303) may be faced to face each other. As illustrated in FIG. 3e, in the case of the G-type foldable electronic device (201), the first hinge (310) may be an in-foldable hinge, and the second hinge (320) may also be an in-foldable hinge.

According to one embodiment, the unfolded state of the electronic device (201) (or display (260)) may include a state in which the electronic device (201) is fully unfolded, with the first housing (303), the second housing (305), and the third housing (307) arranged side by side.

According to one embodiment, the folded state of the electronic device (201) (or display (260)) may include a state in which the electronic device (201) is fully folded with respect to the B-axis and the C-axis, a state in which the electronic device (201) is folded with respect to the B-axis and not folded with respect to the C-axis, or a state in which the electronic device (201) is not folded with respect to the B-axis and is folded with respect to the C-axis. Here, a state in which the electronic device (201) is fully folded with respect to the B-axis and the C-axis may be referred to as a fully folded state of the electronic device (201).

The display (260) may be disposed within the first housing (303), the second housing (305), and the third housing (307), and may include a first portion (or first side) (321) supported by the first housing (303), a second portion (or third side) (323) supported by the second housing (305), and a third portion (or fourth side) (325) supported by the third housing (307). For example, the first display area (261) may be disposed on the first side (321) of the first housing (303), the second display area (262) may be disposed on the third side (323) of the second housing (305), and the third display area (263) may be disposed on the fifth side (325) of the third housing (307).

Referring to FIGS. 3C and 3D, the electronic device (201) may include a second hinge (320) that can foldably connect one side of the first housing (303) to one side of the second housing (305) and a first hinge (310) that can foldably connect the other side of the second housing (305) to one side of the third housing (307). The second hinge (320) may have a wider width than the first hinge (310). As illustrated in FIG. 3C, the first hinge (310) may be an infoldable hinge, and the second hinge (320) may be an outfoldable hinge.

According to one embodiment, the outfolding may refer to outfolding the display (260) of the electronic device (201) outwardly from the front side of the display (260) when the display (260) is completely unfolded, thereby exposing a portion of the display (260) to the outside. Accordingly, when the second hinge (320) is outfolded, the first display area (261) disposed in the first housing (303) connected to the second hinge (320) may be exposed to the outside.

The above infolding may refer to the display (260) of the electronic device (201) being infolded inwardly from the front of the display (260) when the display (260) is completely unfolded, so that a portion of the display (260) is not exposed to the outside. Accordingly, when the first hinge (310) is infolded, the third display area (263) arranged in the third housing (307) connected to the first hinge (310) and the second display area (262) arranged in the second housing (305) that is not folded may face each other, so that the second display area (262) and the third display area (263) may not be exposed.

According to one embodiment, the first display area (261) disposed in the first housing (303) connected to the second hinge (320) by the outfolding of the second hinge (320) can be positioned to correspond to the outfolding angle of the second hinge (320).

According to one embodiment, the third display area (263) disposed in the third housing (307) connected to the first hinge (310) by the infolding of the first hinge (310) may change its position corresponding to the infolding angle of the first hinge (310), and the third housing (307) whose position has changed according to the infolding of the first hinge (310) may cover the second display area (262) disposed in the second housing (305). As described above, the folding ratio of the electronic device (201) (or the display (260)) may be implemented so that it folds at an equal ratio with respect to the axis, but the folding ratio with respect to the axis may be implemented differently. Although FIGS. 3A to 3E illustrate folding at an equal ratio with respect to the axis, those skilled in the art will readily understand that there is no limitation on the position of the axis.

According to one embodiment, in a foldable electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIGS. 2 to 3d), the foldable housing comprises a first housing (303), a second housing (305), and a third housing (307), the second housing being disposed between the first housing and the third housing, a first hinge (310) rotatably connecting the third housing to the second housing, a second hinge (320) rotatably connecting the first housing to the second housing and having a wider width than the first hinge, a flexible display (260) disposed within the first housing, the second housing, and the third housing, the flexible display comprising a first portion (or first display area) (261) supported by the first housing, a second portion (or second display area) (262) supported by the second housing, and the third It may include a third portion (or third display area) (263) supported by the housing, and may include at least one processor (220) including a processing circuit and a memory (230) storing instructions.

According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may be configured to cause the foldable electronic device to display first information in the first portion, second information in the second portion, and third information in the third portion.

In one embodiment, the instructions may be configured to cause the foldable electronic device to generate at least one summary piece of information corresponding to the second information and the third information using a machine learning model, at least in part based on detecting that the third housing is folded to face the second housing, wherein the machine learning model may be a machine learning model trained to output summarized information based on text or image input.

According to one embodiment, the instructions enable the foldable electronic device to display the at least one summary information in the first portion.

According to one embodiment, at least one more sensor may be included, wherein the at least one sensor may include one of a proximity/light sensor, a grip sensor, a capacitive touch sensor, a Hall IC, an acceleration sensor, or a gyro sensor.

In one embodiment, the instructions may be configured to cause the foldable electronic device to detect, using the at least one sensor, that the third housing is folded to face the second housing.

In one embodiment, the instructions may be configured to cause the foldable electronic device to determine a portion to be summarized using another machine learning model for the second information and the third information based on at least a portion of the detection, wherein the other machine learning model comprises a machine learning model trained to determine a portion to be summarized using a user usage pattern for the foldable electronic device, and may be configured to generate the at least one summary information based on at least a portion of the determination.

In one embodiment, the instructions may be configured to cause the foldable electronic device to determine whether to generate a summary by the other machine learning model, and to generate the at least one summary information based at least in part on the determination to generate the summary.

According to one embodiment, the foldable electronic device may further include another display disposed on a surface opposite to the second portion of the second housing.

In one embodiment, the instructions may be configured to cause the foldable electronic device to detect a fully folded state in which the third housing is folded to face the second housing and the first housing is folded to face the third housing, and to generate, based on at least a portion of the detection, at least one other summary information corresponding to the first information, the second information, and the third information using the machine learning model, and to display the at least one other summary information on the other display.

In one embodiment, the instructions may be configured to cause the foldable electronic device to detect that the first housing is unfolded relative to the third housing, and based on at least a portion of the detection, to stop displaying the other at least one summary information on the other display and to display the other at least one summary information on the first portion.

According to one embodiment, the instructions may be configured to cause the foldable electronic device to display the at least one summary information in a status bar located at the top of the first portion.

According to one embodiment, the at least one summary information may include at least one of text or an image.

According to one embodiment, the instructions may be configured to cause the foldable electronic device to receive a user input for the at least one summary information and to display the second information or the third information in the first portion based on the user input.

In one embodiment, the instructions may be configured to cause the foldable electronic device to stop displaying the at least one summary information in the first portion and display the second information in the second portion and the third information in the third portion based on detecting that the third housing is unfolded relative to the second housing.

In one embodiment, the machine learning model may include a generative AI model trained using a neural network structure to generate and output summarized text or images based on a prompt input.

According to one embodiment, in a foldable electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIGS. 2 to 3d), the foldable housing comprises a first housing (303), a second housing (305), and a third housing (307), the second housing being disposed between the first housing and the third housing, a first hinge (310) rotatably connecting the third housing to the second housing, a second hinge (320) rotatably connecting the first housing to the second housing and having a wider width than the first hinge, a flexible display (260) disposed within the first housing, the second housing, and the third housing, the flexible display comprising a first part (261) supported by the first housing, a second part (262) supported by the second housing, and a third part (263) supported by the third housing. It may include at least one processor (220) including a processing circuit and a memory (230) storing instructions.

According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, cause the foldable electronic device to display first information in the first portion, second information in the second portion, and third information in the third portion, and generate at least one summary information corresponding to the third information using a generative AI model based at least in part on detecting that the third housing is folded relative to the second housing, wherein the generative AI model includes a generative AI model that outputs summarized information or a new image based on text or image input, and may be configured to display the at least one summary information in the second portion or the third portion.

According to one embodiment, the device further comprises at least one sensor, wherein the at least one sensor may include one of a proximity/light sensor, a grip sensor, a capacitive touch sensor, a Hall IC, an acceleration sensor, or a gyro sensor.

In one embodiment, the instructions may be configured to cause the foldable electronic device to detect, using the at least one sensor, that the third housing is folded relative to the second housing.

In one embodiment, the instructions may be configured to determine whether to deactivate the first portion or the third portion and the second portion based on the foldable electronic device detecting that the third housing is folded relative to the second housing using the at least one sensor, using another AI model, wherein the another AI model comprises a machine learning model trained to determine a portion to deactivate using a user usage pattern for the foldable electronic device, and may be configured to deactivate the first portion based on the determination of whether to deactivate the first portion.

According to one embodiment, the instructions may be configured to cause the foldable electronic device to determine whether to deactivate the second portion and the third portion using the other AI model, deactivate the second portion and the third portion based on the determination of whether to deactivate the second portion and the third portion, generate at least one other summary information corresponding to the second information and the third information using the generative AI model, and display the at least one other summary information on the first portion.

FIG. 4 is a flowchart illustrating an operation of an electronic device for providing summary information according to an embodiment. Referring to FIG. 4, the operation method may include operations 410 to 430. Each operation of the operation method of FIG. 4 may be performed by an electronic device (e.g., the electronic device 101 of FIG. 1 and the electronic device 201 of FIGS. 2 to 3D) and at least one processor of the electronic device (e.g., the processor 120 of FIG. 1 and the processor 220 of FIG. 2). In an embodiment, at least one of operations 410 to 430 may be omitted, the order of some operations may be changed, or another operation may be added.

In operation 410, the electronic device (201) can display first information on a first part (261) of a flexible display (260) disposed within a first housing (303) of a foldable housing, second information on a second part (262) of the flexible display disposed within a second housing (305) of the foldable housing, and third information on a third part (263) of the flexible display disposed within a third housing (307) of the foldable housing. Here, the second housing (305) can be disposed between the first housing (303) and the third housing (307).

In operation 420, the electronic device (201) may generate at least one summary piece of information corresponding to the second information and the third information using a machine learning model, at least in part based on detecting that the third housing (307) is folded to face the second housing (305). Here, the machine learning model may be trained to output summarized information (e.g., text, image, etc.) based on text or image input.

According to one embodiment, the electronic device (201) can detect that the third housing (307) is folded to face the second housing (305) using at least one sensor (e.g., the sensor circuit (276) of FIG. 2).

According to one embodiment, the electronic device (201) may determine a portion to be summarized using a different machine learning model for the second information and the third information based on detecting that the third housing (307) is folded to face the second housing (305). Here, the different machine learning model may include a machine learning model trained to determine a portion to be summarized using a user usage pattern for the foldable electronic device. The electronic device (201) may generate the at least one summary information based on determining the portion to be summarized.

According to one embodiment, the at least one summary information generated may be a portion of texts and/or images output in the second and third parts edited and processed. Alternatively, the at least one summary information generated may be a portion of texts and/or images added to texts output in the second and third parts.

According to one embodiment, the electronic device (201) may determine which portions of the second information and the third information to summarize using different machine learning models. Here, the different machine learning models may include machine learning models trained to determine which portions to summarize using user usage patterns for the electronic device (201).

In operation 430, the electronic device (201) can display the at least one summary information on the first portion (261).

For example, referring to FIG. 3C, when the Z-type foldable electronic device (201) detects infolding in which the third housing (307) is folded to face the second housing (305), the second part (e.g., the second display area (262)) and the third part (e.g., the third display area (263)) become invisible from the outside, and only the first part (e.g., the first display area (261)) can be visible from the outside. Accordingly, the electronic device (201) can determine the first part (e.g., the first display area (261)) as the visible area, and control the content (e.g., including text and/or images) displayed in the second and third parts to be summarized information through a machine learning model to be displayed in the determined first part (e.g., the first display area (261)).

According to one embodiment, when the electronic device (201) further includes another display disposed on a side (e.g., the third side (324) of FIG. 3B) opposite to the second part (262) of the second housing (305) (e.g., the third side (323) of FIG. 3B), the electronic device (201) may detect a fully folded state in which the third housing (307) is folded to face the second housing (305) and the first housing (303) is folded to face the third housing (307). The electronic device (201) may generate, based on at least a portion of the detection, at least one other summary information corresponding to the first information, the second information, and the third information using the machine learning model. The electronic device (201) may display the at least one other summary information on the other display.

For example, referring to FIG. 3e, when the G-type foldable electronic device (201) detects an infolding in which the first housing (303) is folded to face the second housing (305) and an infolding in which the third housing (307) is folded to face the first housing (303), all of the first part (e.g., the first display area (261)), the second part (e.g., the second display area (262)), and the third part (e.g., the third display area (263)) may not be visible from the outside. However, when the electronic device (201) further includes another display disposed on a side (e.g., the fourth side (324) of FIG. 3e) opposite to the second part (262) of the second housing (305) (e.g., the third side (323) of FIG. 3e), only the other display may be visible from the outside. Therefore, in the G-type foldable electronic device The device (201) can determine a fourth part (e.g., another display) as a visible area, and control the content (e.g., including text and/or images) displayed in the first to third parts to be displayed on the determined fourth part (e.g., another display) by summarizing the information through a machine learning model.

For example, when the e-type foldable electronic device (201) detects the infolding in which the third housing (307) is folded to face the second housing (305) and the infolding in which the first housing (303) is folded to face the third housing (307), all of the first part (e.g., the first display area (261)), the second part (e.g., the second display area (262)), and the third part (e.g., the third display area (263)) may not be visible from the outside. However, when another display is arranged on the opposite side of the second part (e.g., the second display area (262)), only the other display may be visible from the outside. Therefore, the e-type foldable electronic device (201) determines the fourth part (e.g., another display) as the visible area, and summarizes the content (e.g., including text and/or images) displayed in the first to third parts using a machine learning model to determine the information. It can be controlled to display on a fourth part (e.g. another display).

In one embodiment, the electronic device (201) may detect that the first housing (303) is unfolded relative to the third housing (307). Based on at least a portion of the detection, the electronic device (201) may stop displaying the other at least one summary information on the other display and display the other at least one summary information on the first portion (261). For example, when the electronic device (201) detects that the first housing (303) is unfolded relative to the third housing (307) in a fully folded state, the electronic device (201) may determine that the invisible area is the second part (e.g., the second display area (262), the third part (e.g., the third display area (263)), and the fourth part (e.g., another display). On the other hand, the electronic device (201) may determine that the visible area is the first part (e.g., the first display area (261)). Accordingly, the electronic device (201) may stop displaying at least one other summary information on the fourth part (e.g., the other display) and control the display of the at least one other summary information displayed on the fourth part (e.g., the other display) to the first part (e.g., the first display area (261)), which is the visible area.

FIG. 5 is a diagram illustrating detailed operations of an electronic device according to an embodiment. To facilitate understanding of the description of FIG. 5, reference is made to FIGS. 6 to 10, which illustrate a Z-type foldable electronic device. However, the operation of FIG. 5 can also be applied to other types of foldable electronic devices (e.g., G-type and e-type foldable electronic devices).

Referring to FIG. 5, the electronic device (201) can detect (510) a folding state. Here, the electronic device (201) can assume an unfolded state. The unfolded state can include a state in which the electronic device (201) is completely unfolded, with the first housing (303), the second housing (305), and the third housing (307) being arranged side by side, as illustrated in FIG. 6(a). Here, FIG. 6 is a drawing showing a first folding state in an unfolded state of an electronic device according to one embodiment.

The electronic device (201) may determine a display area (or portion) that is visually visible (or exposed) to the user in response to detection of a folding state (or switching of the folding state). The electronic device (201) may identify at least one condition when performing operation (520) of determining the visible display area. The at least one condition may include at least one of a folding state, a holding state, a posture of the electronic device (201), a running app state, or a user usage pattern, but the condition may not be limited thereto.

In one embodiment, as a first condition, the electronic device (201) can detect (521) a folding state between each display area through at least one sensor (e.g., a sensor circuit (276)). The electronic device (201) can detect a transition from an unfolded state (e.g., FIG. 6(a)) to a folded state along at least one folding axis. For example, detecting the folding state can include detecting a movement of the electronic device (201). The movement can include folding, rotation, flipping, or shaking.

According to one embodiment, the electronic device (201) may generate choices for determining whether an area is visible or invisible based on a folding signal through at least one sensor (e.g., sensor circuit (276)) according to the folding state of each display, and the generated choices may be utilized by the artificial intelligence model to determine whether to display summary information (530).

For example, if outfolding occurs between the first display area (261) and the second display area (262), as a first option, the visible area may be the first display area (261), and the non-visible areas may be the second and third display areas (262, 263). In addition, even if outfolding occurs between the first display area (261) and the second display area (262), as a second option, the visible area may be the second and third display areas (262, 263), and the non-visible area may be the first display area (261).

Additionally, if in-folding occurs between the second display area (262) and the third display area (263), as a third option, the visible area may be the first display area (261), and the invisible areas may be the second and third display areas (262, 263).

According to one embodiment, the artificial intelligence model can pre-generate various options (e.g., first to third options) as described above, and then use the options to determine a visible or invisible area.

In one embodiment, as a second condition, the electronic device (201) can detect (522) the grip state through a touch panel (or touch sensor). For example, as illustrated in FIG. 6(b), when the folding state is switched, the electronic device (201) can confirm the user's grip state through touch on the second display area (262) and the third display area (263).

Meanwhile, if no touch signal is detected after the folding state is switched, it can be determined that the electronic device (201) is being used in a cradle-mounted state, and accordingly, the electronic device (201) can use information about the type or characteristics of the cradle to determine the area of the display to be displayed. For example, if a cradle that uses a magnet to secure the electronic device (201) is used, the display area corresponding to the direction of the magnet may be difficult for the user to see, and therefore, the display area corresponding to the direction opposite to the direction of the magnet may be the display area that is visible to the user. In addition, in the case of a physical grab-type cradle, the display area in a specific direction is covered, and therefore, the display area in the specific direction may be regarded as a display area that is not visible to the user. In addition, if the electronic device (201) is maintained in a specific incline posture without the user grasping it, the electronic device can be regarded as a fixed state, and in this case, the display area that is visible to the user can be identified by identifying the user's direction through UWB communication.

In one embodiment, as a third condition, the electronic device (201) can detect (523) the posture (or position) of the electronic device (201). For example, the electronic device (201) may include an acceleration sensor that senses acceleration in the direction of at least one axis among the three axes (e.g., X-axis, Y-axis, and Z-axis) of each display area (261, 262, 263) (or each housing (303, 305, 307). Alternatively, the electronic device (201) may include a gyro sensor that senses rotation of the electronic device (201) around one axis among rotation about the X-axis (roll), rotation about the Y-axis (pitch), or rotation about the Z-axis (yaw). The electronic device (201) may identify a position in which the electronic device (201) is placed, for example, tent mode, table mode, or book mode, by calculating an angle between each display area through the three-axis values. In addition, the electronic device (201) may determine a display area facing the floor as a display area that is not visible using an inertial sensor. For example, the electronic device The device (201) can determine whether the electronic device (201) is facing the floor based on information from a sensor circuit including an inertial sensor such as a tilt, vector sum, and rotation vector.

In addition, the electronic device (201) can determine a display area in which vibration is relatively small based on the user's grip in an environment where vibration occurs as a display area to be shown. In addition, the electronic device (201) can analyze the user's state based on the measurement values (e.g., IMU (Inertial Measurement Unit) values) of each display area. For example, based on the state in which the electronic device (201) is mounted (or the mounted posture), the electronic device (201) can determine that the electronic device (201) is located in a moving vehicle if the acceleration values in the X, Y, and Z directions, which are perpendicular to each other, have a constant value or at least one of the acceleration values in the X, Y, and Z directions has a pattern in which it increases or decreases at a specific slope. Based on these measurement values, the electronic device (201) can identify that the user is driving with the electronic device (201) mounted. Accordingly, when the electronic device (201) identifies that the user is driving, it can determine that the display area where navigation is running is the display area to be shown by increasing the priority of the display area.

In one embodiment, as a fourth condition, the electronic device (201) can identify (524) the status of a running app (or task). The electronic device (201) can further learn, using a machine learning model, which applications were running before folding, or which applications among the running applications are likely to be continuously used by the user. For example, if the user was sorting photos according to their travel itinerary while simultaneously viewing a calendar and a gallery in a multi-window format through each display area in the unfolded state before folding, the display area where the gallery is displayed may become the main display area after folding. Alternatively, if the user is watching a streaming service or conducting a video conference, the display area where the content being watched or during the video conference is displayed may be the display area shown to the user. Therefore, the electronic device (201) can use this to determine the display area showing the status of the running app.

In one embodiment, as a fifth condition, the electronic device (201) may analyze (525) a user usage pattern for the electronic device (201). The user usage pattern may be learned from an on-device AI model (e.g., a machine learning model). The electronic device (201) may store information about a display type or direction preferred by the user when using a specific application using the on-device AI model, analyze the usage order or connection between applications, or analyze the method or reason for the user's application usage to analyze the user's usage pattern. For example, the electronic device may use the on-device AI model to determine the display area in cases where the user watches a video with only the hinge of the outfolding folded at a set time every day, or where the user watches a game video using an app through two display areas by only the hinge of the outfolding folded, or where the user alternately uses both display areas by rotating the electronic device (201) with the hinge of the outfolding folded, or where the user has completed an operation being performed in a specific application. As such, since the user's usage pattern may vary depending on the user's situation, the electronic device (201) may utilize the learning ability of the on-device AI model to analyze the user's usage pattern. That is, the electronic device (201) may determine the user's usage pattern based on at least one of the display location where each app is displayed, user preference, activation priority according to app characteristics, user status, and the status of the electronic device (201). According to one embodiment, the electronic device (201) may determine the user's usage pattern on a task-by-task basis. Accordingly, the electronic device (201) may determine at least one task being executed in an application and perform a judgment based on the judgment. For example, the electronic device (201) may determine the user's usage pattern by considering the status of the task being executed on the screen, the display location of each app, user preference, activation priority according to app characteristics, user status, electronic device status, etc.

In one embodiment, the electronic device (201) can determine a display area (e.g., a main display area) that is folded and made invisible (or deactivated) and a display area (e.g., a sub display area) that is visible (or activated) based on at least one condition. The electronic device (201) can determine the display area that is invisible/visible and determine (530) whether to display summary information. According to one embodiment, in addition to the at least one condition, the electronic device (201) can comprehensively consider changes in sensor values, touch panel recognition, changes in software (SW), data from previous operations, etc., and train artificial intelligence to learn this information. According to one embodiment, the learned artificial intelligence (e.g., a predictive AI model) can be used to distinguish the main display that the user is using, determine which screen is visible or invisible to the user, and determine whether to display summary information.

In one embodiment, the electronic device (201) may utilize a user's usage pattern for the electronic device (201) based on an artificial intelligence (e.g., on-device AI, external artificial intelligence AI) model to determine whether to display summary information (or summary data). According to one embodiment, the artificial intelligence model may be included not only in the electronic device (201) but also in a server (e.g., server (108) of FIG. 1), and the electronic device (201) may utilize the artificial intelligence model within the server (108) through communication with the server (108). The electronic device (201) may determine whether to display summary information by considering the importance of the content displayed in the display area that is folded and not visible, the progress, or the user's intention to exit. In one embodiment, the electronic device (201) may identify the influence of all variables that the user may perform (e.g., the application being used, the touch operation, or the electronic device (201)) through a regression model in predictive modeling, and infer the subsequent situation. Alternatively, a neural network model can be used to identify common patterns between the user's nonlinear actions and predict subsequent actions.

In one embodiment, a predictive modeling (Predictive AI Model) implemented in artificial intelligence (e.g., on-device AI, external artificial intelligence) may quantify and display the degree corresponding to each of the finally categorized output factors, and may determine a display area (e.g., main display area) that is folded and made invisible (or deactivated) and a display area (e.g., sub display area) that is visible (or activated) by determining that the action corresponding to the highest value is likely to have been performed based on user information. In one embodiment, the use of the artificial intelligence model through an external device (e.g., external server (108)) as well as the on-device AI may also be possible. Here, the input features may include metadata such as 6-axis sensor raw data, digital Hall sensor raw data, touch sensor recognition, analyzed grip status, information on the app the user is running, schedule, weather, 6-axis, location information, health information, preferred videos, photos/messages/call history/call recordings, etc., information about the user or his/her surroundings, lifestyle patterns (e.g., getting on the bus at a specific time every day and getting off at the same location), user feedback, and user usage patterns for electronic devices (201).

According to one embodiment, the electronic device (201) can utilize a predictive AI model to increase the accuracy of the AI model and perform personalization. For example, the predictive AI model can perform probabilistic inference to predict results through past data, statistical results, learning, etc. For example, since a user can use the electronic device (201) in numerous ways and it is impossible to learn all situations in advance, the AI model can act according to a choice with a high probability. According to one embodiment, as the user's frequency of use and the number of uses increase, the accuracy of the AI model increases, and the AI can be personalized based on learning through personal data. For example, the electronic device (201) can learn the user's usage pattern information, the frequency of referencing the display usage content, or the display usage method, and change the structure to match the user.

On the other hand, output features can include the probability of matching unexpected issues such as app termination/aspect ratio change/third party verification/drop, and based on this, the display area that is folded and invisible can be determined and the display area that is visible.

In one embodiment, the electronic device (201) may preferentially consider hardware elements (e.g., sensor data, touch information, and grip status) such as the first and second conditions. However, if the electronic device (201) is unable to analyze the hardware elements in a stationary state, it may utilize software elements (e.g., judgment results of on-device AI). In addition, the electronic device (201) may also prioritize the user's usage patterns (e.g., priority and preference between apps) or repetitive actions at a specific time in determining the display area.

For example, if the importance (e.g., priority) of an application displayed on a display area that is folded out and not visible is lower than that of an application running on a visible display area, the electronic device (201) may not display summary information or may display it in a smaller size than a predetermined size. For example, when watching an online lecture in real time and writing down the lecture content in a notepad, the importance of the real-time lecture may be higher for some users than the memo. If the summary information in the memo obscures the real-time lecture content, it may hinder the user's usability, and therefore, in this case, the electronic device (201) may not perform an operation to generate summary information. Accordingly, considering the user's usage pattern and the importance of the application, the priority between the application displayed on the display area that is folded out and not visible and the application running on the visible display area may be checked, and whether or not to generate summary information may be determined based on the checked priority. In this case, rather than not generating summary information when the priority is low, the summary information may not be generated when the difference in importance between the two applications exceeds a certain level or when it obstructs the field of vision and reduces visibility.

In one embodiment, if the execution of an application is interrupted by a folding action during execution, the electronic device (201) may determine the user's intention based on the user's post-folding action and summarize the operation contents up to the point of interruption, or control the operation to run in the background without generating summary information. In addition, if folding is detected after the execution of an application action is completed, the electronic device (201) may determine that the previous application contents or a summary is necessary based on additional input from the user, thereby generating summary information.

In one embodiment, when the contents of two applications are displayed on a display area where the contents of the first application are visible due to a folding operation, the electronic device (201) may determine whether to summarize the contents of the first application running on the invisible display area by determining the correlation between the two applications. This operation may serve as one of the input features of the on-device AI model, and whether to display summary information may be determined based on the correlation between applications in the same situation.

For example, if the electronic device (201) determines that the two applications are related apps, it may display a summary of the contents of the first application in the display area, and if the two applications are determined to be unrelated, it may not provide a summary function for the contents of the first application. For example, if a user is watching a video using the electronic device (201) and uses a notepad and then folds the display area corresponding to the notepad, the electronic device (201) may not perform an operation for generating and displaying summary information in order not to disturb the immersion of the video. On the other hand, if the video is an online lecture, the electronic device (201) may perform an operation for generating and displaying summary information. In this way, the relationship between applications may be determined based on the user's previous application usage pattern or the properties of the applications.

Meanwhile, according to one embodiment, in order to determine whether to generate summary information by determining whether the user's intention to change the folding state by applying force is to terminate a running application or to use the application in use without interruption, the electronic device (201) may use a machine learning model different from the machine learning model used to determine the display area. If it is determined by the other machine learning model to generate a summary, the electronic device (201) may generate information to be summarized by using a machine learning model trained to output summarized text based on text or image input, based on content displayed in the display area that is folded and not visible.

According to one embodiment, the electronic device (201) may determine (540) information to be displayed by summarizing and organizing the original content of the determined non-visible display area in the visible display area using an AI summary generation model (550) (e.g., a machine learning model, a generative AI model). Accordingly, the electronic device (201) may display (560) summary information of the app (or task) (e.g., original content) that was output in the non-visible display area in the visible display area.

According to one embodiment, in order to determine information to be summarized and displayed in a visible display area, input features input to an artificial intelligence model and output features output from the artificial intelligence model are as follows. For example, the input features may include at least one of an invisible display area, an app being run by the user, user information (e.g., schedule, weather, location, health, user information obtained from photos/messages/call recordings, etc., electronic device usage patterns, etc.), and raw data of a sensor circuit (276). In addition, for example, the output features may include at least one of determination of summarizeable elements among application contents, summary information of a running application, or importance.

Here, the information to be summarized and displayed can be determined based on the type of application, role, and the user's history of using the app. For example, if the user uses a memo app, the electronic device (201) can organize the memo content, and if the user uses a schedule app, it can organize the schedule by date into text. If the user is watching a video, the electronic device (201) can identify the content, title, and viewing information of the video, and if the user is listening to music, the electronic device (201) can provide a list of music the user has previously listened to or generate and provide summary information analyzing the user's preferred music taste. The information (or data) to be summarized and displayed in this manner is determined and generated by the on-device AI, and can be determined by considering various conditions such as the user's purpose of using the application, the type of application, or the user's preferences. When displaying the summary information, information determined to be of high importance among the summarized contents can be preferentially displayed. According to one embodiment, the electronic device (201) can utilize a generative AI model as an AI summary generation model when generating the summary information. In one embodiment, generative AI is an artificial intelligence that creates new results based on analysis of provided data, thereby mimicking human creativity in generating results. For example, generative AI must understand human language and then generate summary information, so a Large Language Model (LLM) based on a transformer architecture can be utilized. Here, since the information to be summarized is not limited to text and can be in various forms such as video, pictures, and sound, a Large Multimodal Model (LMM, = Multimodal LLM) can be used. In one embodiment, the generated summary information is not limited to text and can be changed according to settings or summary content. For example, the electronic device (201) may express the summary of a memo as text, but in the case of an edited photo, it may output the before/after images as a summary.

Meanwhile, when displaying summary information, the format (or output method) of the summary information presented to the user can adaptively change depending on the type of application. Furthermore, the summary information displayed is summarized based on elements corresponding to the main functions (or features) provided by the application, but may also vary depending on the size of the displayed display area or the user's actions.

For example, when playing content such as a video in an unfolded state, the electronic device (201) may display only a minimum display area to the user when it is completely folded by a folding action. The electronic device (201) may display summary information related to content playback, such as application type, content title, viewing time, and paused point in time, on the minimum display area (e.g., the first display area (261)). If the partially folded state causes the visible display area to expand, the electronic device (201) may display additional summary information, such as a thumbnail of the content and comment responses, on the expanded display area (e.g., the second and third display areas (262, 263)). If the device is in a fully unfolded state, the entire display area can be shown to the user, and the electronic device (201) can additionally display information summarizing the content up to the viewing point of the content on the entire display area (e.g., the first to third display areas (261, 262, 263)). In this way, when folding is detected, the summarized content displayed may vary depending on the folding state.

Meanwhile, even when the same application is running, the electronic device (201) can provide different summary information based on the user's last action and user information, thereby improving the accuracy of AI curation. For example, if a folding is detected while the user is watching the same content, summary information about the content scene can be generated. However, if a folding is detected while checking comment responses to the content, information such as a summary of the comment responses or the overall response ratio (e.g., positive/negative) can be generated as summary information. In addition, the electronic device (201) can detect the user's movement using at least one sensor, and can preferentially generate summary information including the determined element by determining whether the user is focusing on a visual element or an auditory element. Here, the use of a sensor to detect the user's status is described as an example, but the means for detecting the user's status is not limited thereto.

Meanwhile, when displaying (560) summary information of an app (or task) (e.g., original content) that was output in an invisible display area in a visible display area, the electronic device (201) can use user information for omitted parts or implied content when generating the summary information, and can be added in the form of a hidden prompt or hidden layer.

Referring to FIG. 6, as shown in FIG. 6(a), the electronic device (201) can be changed from a fully unfolded state to a first folded state as shown in FIG. 6(b). When identifying the first folded state, the electronic device (201) can determine a display area that is folded and invisible and a display area that is visible based on the first condition and the second condition.

According to one embodiment, the electronic device (201) can identify, as a first condition, an outfolding case by a folding axis between the first housing and the second housing through at least one sensor (e.g., sensor circuit (276)). The electronic device (201) can identify a folding angle by the at least one sensor, and accordingly, the electronic device (201) can detect what type of folding state it has changed into. For example, if the folding angle is identified as 120° or more, the electronic device (201) can assume that the first display area (261) or the second and third display areas (262, 263) are the display areas that are shown.

In the grip state as illustrated in FIG. 6(b), the electronic device (201) can, as a second condition, recognize multi-touch in the first display area (261) and detect two or fewer touches in the second and third display areas (262, 263).

Accordingly, the electronic device (201) can determine, based on the first condition and the second condition, that the display area that is folded and becomes invisible is the first display area (261), and that the display areas that are visible are the second and third display areas (262, 263). At this time, the electronic device (201) can further use at least one of the third to fifth conditions described above in addition to the first and second conditions in determining the display area that is folded and becomes invisible, and the display area that is visible.

Referring to FIG. 7, as shown in FIG. 7(a), the electronic device (201) may be changed from a fully unfolded state to a second folded state as shown in FIG. 7(b). FIG. 7 is a diagram illustrating a second folded state in an unfolded state of an electronic device according to an embodiment. When identifying the second folded state, the electronic device (201) may determine a display area that is folded and invisible and a display area that is visible based on the first condition and the second condition.

According to one embodiment, the electronic device (201) can identify, as a first condition, a case of infolding by a folding axis between the second housing and the third housing through at least one sensor (e.g., sensor circuit (276)). For example, if the folding angle is identified as 30° or less, the electronic device (201) can assume that the first display area (261) is the display area that is visible.

In the grip state as illustrated in FIG. 7(b), the electronic device (201) can detect a touch only through the first display area (261) as a second condition. Accordingly, the electronic device (201) can determine, based on the first condition and the second condition, that the display areas that are folded and become invisible are the second and third display areas (262, 263), and that the display area that is visible is the first display area (261).

Referring to FIG. 8, as shown in FIG. 8(a), the electronic device (201) can be changed from a folded state to a third folding state, as shown in FIG. 8(b). FIG. 8 is a drawing showing the third folding state in the folded state of the electronic device according to one embodiment.

According to one embodiment, the electronic device (201) can identify a rotational direction of the electronic device (201) using at least one sensor (e.g., sensor circuit (276)) under a first condition, in which the outfolding is maintained by the folding axis between the first housing and the second housing. When such rotation is detected, the electronic device (201) can determine that the display area shown to the user has changed.

According to one embodiment, in a grip state as illustrated in FIG. 8(b), the electronic device (201) may detect a user touch on the display area as a second condition and use a pattern of the touch, a number of touched areas, a predicted value by the touch, etc. For example, the electronic device (201) may detect two or fewer touches through the first display area (261) and a multi-touch through the second display area (262). Accordingly, the electronic device (201) may determine that the display areas that become invisible due to rotation are the second and third display areas (262, 263), and the display area that becomes visible due to rotation is the first display area (261) based on the first condition and the second condition.

Referring to FIG. 9, as shown in FIG. 9(a), the electronic device (201) can be changed from a folded state to a fourth folding state, as shown in FIG. 9(b). FIG. 9 is a drawing showing the fourth folding state in the folded state of the electronic device according to one embodiment.

The electronic device (201) can detect infolding by a folding axis between the second housing and the third housing through at least one sensor (e.g., sensor circuit (276)) while the outfolding is maintained by the folding axis between the first housing and the second housing, as a first condition.

At this time, the in-folding operation between the second housing and the third housing while the out-folding is maintained by the folding axis between the first housing and the second housing can be broadly divided into a case where the rotation of the electronic device (201) is detected after the third display area (263) is folded to face the second display area (262), and a case where the in-folding between the second display area (262) and the third display area (263) is detected simultaneously with the rotation of the first and second display areas (261, 262).

In the grip state as illustrated in Fig. 9(b), the electronic device (201) can detect a touch only through the first display area (261) as a second condition.

Accordingly, the electronic device (201) can determine that the display area that becomes invisible after folding is the second and third display areas (262, 263) and the display area that is visible is the first display area (261) based on the first condition and the second condition.

Referring to FIG. 10, as shown in FIG. 10(a), the electronic device (201) can be changed from a folded state to a fifth folding state, as shown in FIG. 10(b). FIG. 10 is a drawing showing the fifth folding state in the folded state of the electronic device according to one embodiment.

According to one embodiment, the electronic device (201) can detect an infolding by a folding axis between the second housing and the third housing through at least one sensor (e.g., sensor circuit (276)) while the outfolding is maintained by the folding axis between the first housing and the second housing as a first condition.

At this time, the in-folding operation between the second housing and the third housing while the out-folding is maintained by the folding axis between the first housing and the second housing can be broadly divided into a case where the rotation of the electronic device (201) is detected after the third display area (263) is folded to face the second display area (262), and a case where the in-folding between the second display area (262) and the third display area (263) is detected simultaneously with the rotation of the first and second display areas (261, 262).

In the grip state as illustrated in FIG. 10(b), the electronic device (201) can detect a touch only through the first display area (261) as a second condition.

Accordingly, the electronic device (201) can determine that the display area that becomes invisible after folding is the second and third display areas (262, 263) and the display area that is visible is the first display area (261) based on the first condition and the second condition.

FIG. 11 is a diagram illustrating a method for providing summary information while a first app is running according to one embodiment.

Referring to FIG. 11, if there is only one app running in the expanded (or unfolded) state (e.g., FIG. 11(a)), the electronic device (201) may not provide a summary due to folding. For example, the display screen of the running app may be output across the first display area (261) to the third display area (263). At this time, since the determination of the display area that is not visible/visible due to folding continues, the electronic device (201) may perform an operation such as changing the display ratio or the display screen according to the folding operation as in FIG. 11(b) to FIG. 11(e). For example, FIG. 11(b) and FIG. 11(c) have the same physical state in which the first housing (303) and the second housing (305) are folded out, but the display areas shown to the user may be different. In addition, in FIG. 11(c), the first housing (303) and the second housing (305) are in an outfolded state, in FIG. 11(d), the second housing (305) and the third housing (307) are infolded, and in FIG. 11(e), the first housing (303) and the second housing (305) are in an outfolded state while the second housing (305) and the third housing (307) are infolded. Therefore, although FIGS. 11(c) and 11(e) have different physical states, the display area shown to the user may be the same, for example, the first display area (261). Accordingly, the electronic device (201) can identify which of the first to third display areas (261, 262, 263) is in use (or in an active state, a state shown to the user) when folded. The electronic device (201) can provide notifications by considering the display area in an activated state as the main display area, and can prevent malfunctions by restricting touch input for display areas that are not visible.

FIG. 12a is a diagram for explaining a method for providing summary information in a first situation while a first app and a second app are running according to an embodiment, FIG. 12b is a diagram for explaining a method for providing summary information in a second situation while a first app and a second app are running according to an embodiment, and FIG. 12c is a diagram for explaining a method for providing summary information in a third situation while a first app and a second app are running according to an embodiment.

Referring to FIGS. 12A to 12C, when there are two apps running in an expanded state (or unfolded state), for example, as shown in (a) of FIG. 12A and (a) of FIG. 12B, a case may be considered where content according to the execution of the first app is displayed in the first and second display areas (261, 262), and content according to the execution of the second app is displayed through the third display area (263). In this way, when a folding action is detected in a state where content according to the execution of two apps is displayed, the electronic device (201) can detect folding in various forms, as shown in (b) to (e) of FIG. 12A and (b) to (e) of FIG. 12B. The electronic device (201) can determine which display area is not visible/visible due to folding. Accordingly, the electronic device (201) can generate information summarizing content (e.g., content of the second app) output on a display area (e.g., third display area (263)) that is not to be displayed, as shown in (b) to (e) of FIG. 12a, and output summary information (1210, 1220, 1230) of the second app on at least a portion of a display area (e.g., first display area (261)) that is to be displayed.

For example, as in (b) of FIG. 12a, if the first app is simultaneously included in a display area where it is not visible (e.g., the first display area (261)) and a display area where it is visible (e.g., the second display area (262)), the screen ratio at which the content corresponding to the first app is displayed may be different to correspond to the visible display area (e.g., the second display area (262)), but summary information may not be generated.

On the other hand, as in (b) of FIG. 12B, the screen ratio at which the content corresponding to the first app is displayed may be changed to correspond to the display area (e.g., the second display area (262)) shown depending on the type of the first app, and summary information related to the first app (e.g., the first app summary (1240)) may be generated, and the generated summary information related to the first app (e.g., the first app summary 1240)) may be displayed on a part of the display area (e.g., the second display area (262)) shown. For example, when the type of the first app is an app that has an amount of information that is difficult for the user to check at once due to a change in the screen ratio, such as a PDF, a word document, or a memo (e.g., lecture notes), the electronic device (201) may perform an operation of generating a summary of the content on the display area (e.g., the first display area (261)) that is not shown. In addition, the electronic device (201) can generate information summarizing content (e.g., content of the second app) output on a display area (e.g., third display area (263)) that is not to be displayed, as shown in (c) to (e) of FIG. 12b, and output summary information (1250, 1260, 1270) of the second app on at least a portion of the display area (e.g., first display area (261)) that is to be displayed.

Meanwhile, as illustrated in (a) of FIG. 12c, a case may be considered in which content according to the execution of a first app is displayed in the first display area (261), and content according to the execution of a second app is displayed through the second and third display areas (262, 263). In this case, folding in various forms, as illustrated in (b) to (e) of FIG. 12c, can be detected by the folding operation, and the electronic device (201) can determine which display area is not visible/visible due to folding. For example, since the first display area (261) where content according to the execution of the first app was displayed is not visible from the outside when folded, the electronic device (201) may generate summary information related to the first app (e.g., first app summary (1275)) and display the generated summary information related to the first app (e.g., first app summary (1275)) on a part of the display area (e.g., second display area (262) or third display area (263)). For example, as illustrated in (b) of FIG. 12c, the summary information (1275) related to the first app may be displayed on either the second display area (262) or the third display area (263), or may be displayed across the second display area (262) and the third display area (263). Summary information (1275) related to the first app may be placed within an area that does not interfere with the display of content resulting from the execution of the second app, and may be displayed, for example, in a status bar located at the top of the second display area (262) or the third display area (263).

According to one embodiment, the electronic device (201) may generate information summarizing content (e.g., content resulting from execution of a second app) output on a display area (e.g., a second display area (262), a third display area (263)) that is not to be displayed, as illustrated in (c) to (e) of FIG. 12c (e), and output the information on at least a portion of a display area that is to be displayed (e.g., a first display area (261)).

FIG. 13 is a diagram illustrating a method for providing summary information while a first app to a third app are running according to one embodiment.

Referring to FIG. 13, when the content (or information of the first to third apps) according to the execution of the first to third apps are displayed through each display area (261, 262, 263) as illustrated in FIG. 13(a), when folding is detected, the electronic device (201) may generate information summarizing the content output on the display area that is not visible and output it on at least a portion of the visible display area. For example, when the first display area (261) is covered as illustrated in FIG. 13(b), the electronic device (201) may output information summarizing the content of the first app (e.g., first screen summary (1310)) on at least a portion of one of the second and third display areas (262, 263). If the second and third display areas (262, 263) are covered and not visible as shown in FIGS. 13(c) to (e), the electronic device (201) can output information summarizing the contents of each of the second and third apps (e.g., second screen summary and third screen summary (1320, 1330, 1340)) on a portion of the first display area (261).

For example, in a fully expanded state in which contents of different apps (e.g., information of each of the first app, the second app, and the third app) are displayed through each display area (261, 262, 263) (or the first to third housings (303, 305, 307)) as in FIG. 13(a), when the second display area (262) and the third display area (263) overlap each other so as to face each other as in FIG. 13(d), at least one summary information (e.g., the second screen summary and the third screen summary (1330)) summarizing the contents of each of the second and third apps can be output on a portion of the first display area (261). If the third display area (263) is expanded with respect to the second display area (262), the display of the summary information on the first display area (261) can be stopped (or the display can be canceled), and the content of the second app (or information of the second app) can be displayed on the second display area (262) as in FIG. 13(a), and the content of the third app (or information of the third app) can be displayed on the third display area (263).

As described above, the electronic device (201) can summarize and provide information about all apps in the display area that is covered (or made invisible) by the folding action.

Figure 14 is a drawing for explaining an example of displaying summary information according to one embodiment.

Referring to FIG. 14, the electronic device (201) can summarize and display the content of a first app in an invisible display area on a portion of a visible display area. The method for displaying the summary information may vary in location and size. Furthermore, the method for displaying the summary information may vary depending on the number of apps to be summarized as well as the characteristics of the running apps.

For example, as illustrated in FIG. 14(a), summary information (e.g., video content summary (1410)) may be displayed at the bottom of the displayed display area (e.g., first display area (261)), and may also be displayed at the top (e.g., status bar) as illustrated in FIG. 14(b). If the number of apps to be summarized is two or more, each summary information may be separated and displayed as illustrated in FIG. 14(b) (e.g., second screen summary (1420), third screen summary (1430)). However, as illustrated in FIG. 14(c), separate summary information (e.g., message content summary (1442), schedule content summary (1444)) may also be provided through one area (1440). The method of displaying summary information is not limited to the above, and various methods may be used to display it.

FIG. 15 is a drawing for explaining an example of summary information displayed differently depending on a folding operation according to one embodiment.

Referring to FIG. 15, as shown in FIG. 15(a), in a state where content is displayed according to execution of different apps (e.g., first app to third app) through each display area (261, 262, 263), as shown in FIG. 15(b), the electronic device (201) can determine the first display area (261) as an invisible display area and determine the second and third display areas (262, 263) as visible display areas by detecting an outfolding state and a grip state between the first display area (261) and the second display area (262). Accordingly, if the electronic device (201) determines the first display area (261) as an area that is not to be displayed, it can output information summarizing the content of the first app (e.g., first screen summary (1510)) on at least a portion of one of the second and third display areas (262, 263).

According to one embodiment, in a state where an outfolding between the first display area (261) and the second display area (262) in FIG. 15(b) and a grip state in the second and third display areas (262, 263) are detected, a case may be considered where the rotation of the electronic device (201) is detected as illustrated in FIG. 15(c). If the first display area (261) is turned toward the user due to the rotation of the electronic device (201), the grip state detected through the second and third display areas (262, 263) may not be detected through the third display area (263). Therefore, the electronic device (201) may determine a visible display area and a non-visible display area as the rotation of the electronic device (201) and a change in the grip state are detected.

If, as illustrated in FIG. 15(c), a change in the rotation and grip state of the electronic device (201) is detected, the electronic device (201) may determine the first display area (261) as a visible display area and determine the second and third display areas (262, 263) as invisible display areas. Accordingly, the electronic device (201) may output information summarizing the content of the second app (e.g., second screen summary (1520)) and information summarizing the content of the third app (e.g., third screen summary (1530)) on a portion of the first display area (261).

In the unfolded state as illustrated in FIG. 15(a), as illustrated in FIG. 15(d), the electronic device (201) can determine the first display area (261) as a visible display area and determine the second and third display areas (262, 263) as invisible display areas by detecting the infolding and grasping states between the second display area (262) and the third display area (263). Accordingly, the electronic device (201) can generate information summarizing the contents of each of the second and third apps by detecting the infolding and grasping states. The electronic device (201) can output information summarizing the contents of the second app (e.g., the second screen summary (1540)) and information summarizing the contents of the third app (e.g., the third screen summary (1550)) on the first display area (261), as illustrated in FIG. 15(d).

If, in the in-folding state between the second display area (262) and the third display area (263) in FIG. 15(d), an out-folding between the first display area (261) and the second display area (262) is detected as shown in FIG. 15(e), the electronic device (201) can determine the display area to be shown and the display area to be not shown in response to the change in the folding state.

In addition, when an in-folding between the second display area (262) and the third display area (263) is detected as shown in FIG. 15(e) in the out-folding state between the first display area (261) and the second display area (262) in FIG. 15(c), the electronic device (201) can determine the display area to be shown and the display area to be not shown in response to the change in the folding state.

As illustrated in FIG. 15(e), by detecting the outfolding between the first display area (261) and the second display area (262), and the infolding and gripping state between the second display area (262) and the third display area (263), the electronic device (201) can determine the first display area (261) as a visible display area and determine the second and third display areas (262, 263) as invisible display areas. The electronic device (201) can generate information summarizing the contents of each of the second and third apps, and, as illustrated in FIG. 15(e), can output information summarizing the contents of the second app (e.g., the second screen summary (1560)) and information summarizing the contents of the third app (e.g., the third screen summary (1570)) on the first display area (261).

In one embodiment, the electronic device (201) may determine a display area corresponding to a direction in which the user is looking, as the display area to be shown, based on the movement (e.g., posture) of the electronic device (201), such as rotation as well as a folding motion, but in addition, conditions used to determine the display area to be shown, such as the direction of the user with respect to the electronic device (201) and the user's line of sight, may vary.

FIG. 16 is a diagram illustrating a case of returning according to a user input for summary information according to one embodiment.

Referring to FIG. 16, FIG. 16(a) illustrates a case where content (1610) is output through a display area (e.g., a first display area (261)) that is displayed according to app execution, and summary information (e.g., a message content summary, a schedule content summary) (1620) is displayed on at least a portion of the display area (e.g., the first display area (261)). The summary information (1620) may be generated based on content output through display areas (e.g., a second display area (262) and a third display area (263)) that are not displayed due to folding. If a user input (e.g., touch input) for summary information (1620) is received, the electronic device (201) may display content that was being output through display areas that are no longer visible (e.g., the second display area (262) and the third display area (263)) so that it occupies the entire visible display area (e.g., the first display area (261)). In this way, the electronic device (201) may output previous content through the currently visible display area (e.g., the first display area (261)) in response to a selection for summary information (1620). For example, in an unfolded state in which first, second, and third information are displayed through the first display area (261), the second display area (262), and the third display area (263), respectively, due to an infolding operation between the second display area (262) and the third display area (263), the second display area (262) and the third display area (263) overlap each other and become invisible, so that summary information (1620) can be displayed on the first display area (261). If a user input (e.g., a touch input) for the summary information (1620) is received, the electronic device (201) can also display the second information or the third information on the first display area (261) without an unfolding operation by the user.

According to one embodiment, when a user input (e.g., touch input) for the summary information (1620) is received, the electronic device (201) may stop displaying the content (1610) according to the app execution without a user's unfolding action and display the second information or the third information on the first display area (261).

According to one embodiment, when a user input (e.g., touch input) for the summary information (1620) is received, the electronic device (201) may display second information or third information by overlapping it on the content (1610) according to the app execution.

Meanwhile, in one embodiment, the electronic device (201) may, in response to a selection of summary information (1620) as illustrated in FIG. 16(b), drive the conversational AI of the on-device AI to display information (1630) including a detailed description of the summary information (1620) by the on-device AI through the currently visible display area (e.g., the first display area (261)). The detailed description of the summary information (1620) may be an interactive description of the summarized content based on the content of the display areas that have become invisible (e.g., the second display area (262) and the third display area (263)). The user may request additional analysis of the summarized content through the conversational AI.

FIG. 17a is a side view of a G-type foldable electronic device according to one embodiment, and FIG. 17b is a drawing showing an unfolded state of the G-type foldable electronic device according to one embodiment.

Referring to FIGS. 17a and 17b, for example, in the case of a G-type foldable electronic device (201), the first display area (261) (or the first housing (303)) and the second display area (262) (or the second housing (305)) may be infolded based on the folding axis, and the second display area (262) (or the second housing (305)) and the third display area (263) (or the third housing (307)) may be infolded based on the folding axis. Accordingly, the first display area (261) and the second display area (262) may be faced so as to face each other. Here, the switching of the folded/unfolded state may be detected based on a signal detected by a sensor circuit (1173b, 1173c), for example, a Hall sensor (or Hall IC), and one or more magnets (1173a, 1173d).

In the case of FIG. 17A, another display (1730) may be further included on a side (1720) opposite to the side (1710) of the second display area (262), and since the other display (1730) can be externally visible in the fully folded state, the electronic device (201) may determine the other display (1730) as a display to display summary information. The electronic device (201) may display summarized content generated based on the content of the display areas (e.g., the first display area (261), the second display area (262), and the third display area (263)) that have become invisible as the folding state changes to the fully folded state on at least a portion of the other display (1730). Accordingly, the user can check summary information (e.g., summary information generated based on content displayed in the first display area (261), the second display area (262), and the third display area (263)) through the other display (1730).

According to one embodiment, FIG. 17A illustrates a case where another display (1730) is further included on a side (1720) opposite to a side (1710) of the second display area (262), but the side on which the other display is arranged may not be limited thereto. For example, another display may be further included on a side (e.g., the sixth side (326)) opposite to a side (e.g., the fifth side (325)) of the third display area (263). If, unlike FIG. 17A, another display is further included on a side opposite to the first display area (261), the user can check summary information through the other display even in a fully folded state in which the third display area (263) is folded to face the second display area (262) and the first display area (261) is folded to face the third display area (263).

FIG. 18 is a drawing for explaining an example of displaying summary information in a G-type foldable electronic device according to one embodiment.

Referring to FIG. 18, as illustrated in FIG. 18(a), in the case of the G-type foldable electronic device (201), when the contents of different apps (e.g., the first app to the third app) are displayed through each of the first to third display areas (261, 262, 263), when the third display area (263) is folded to face the second display area (262), as illustrated in FIG. 18(b), summary information (e.g., the second screen summary (1810), the third screen summary (1820)) related to the second and third display areas (262, 263) can be displayed on one area of the first display area (261).

If the electronic device (201) is rotated, another display (1730) (e.g., the fourth display area (264)) arranged on the opposite side to the second display area (262) may be determined as the display area shown to the user. Accordingly, as illustrated in FIG. 18(c), information summarizing the contents of different apps (e.g., the first app to the third app) output through each of the first to third display areas (261, 262, 263) may be displayed on a portion of the fourth display area (264) (e.g., the first screen summary (1860), the second screen summary (1870), the third screen summary (1880)). The electronic device (201) can generate at least one summary information corresponding to each content of different apps (e.g., first to third apps) using a machine learning model, and can display the generated summary information (e.g., first screen summary (1860), second screen summary (1870), third screen summary (1880)) through the other display (1730) (e.g., fourth display area (264)).

If a fully folded state in which the third display area (263) is folded to face the second display area (262) and the first display area (261) is folded to face the third display area (263) is detected, the display of the summarized information (e.g., the first screen summary (1830), the second screen summary (1840), the third screen summary (1850)) can be maintained on a part of the fourth display area (264), as shown in FIG. 18(d).

Meanwhile, when the first display area (261) is detected to be unfolded with respect to the third display area (263), the electronic device (201) may deactivate the fourth display area (264). For example, the electronic device (201) may stop displaying the summarized information (e.g., the first screen summary (1830), the second screen summary (1840), the third screen summary (1850)) through the fourth display area (264) and may display at least one summary information (e.g., the second screen summary (1810), the third screen summary (1820)) on a part of the first display area (261) that is shown as unfolded, as illustrated in FIG. 18(b).

On the other hand, when a fully expanded state is detected while the operations of three apps are summarized and displayed in the fourth display area (264), the electronic device (201) can switch the app running in the fourth display area (264) to the background and display the summary of the three apps in detail through each display area (261, 262, 263).

Meanwhile, in addition to the aforementioned G-type foldable electronic device, the present invention may be applied to various multiple foldable electronic devices, such as the e-type foldable electronic device illustrated in FIG. 19. FIG. 19 is a side view of an e-type foldable electronic device according to one embodiment.

As illustrated in FIG. 19, in the case of an e-type foldable electronic device, when the first display area (1961) is folded to face the second display area (1962) and the third display area (1963) is folded to face the first display area (1961), the electronic device (201) can detect a fully folded state.

According to one embodiment, as illustrated in FIG. 19, if another display (1930) is further included on a side (1920) opposite to the side (1910) on which the second display area (262) is arranged, the user can check summary information through the other display (1930) even in a fully folded state. According to one embodiment, FIG. 19 exemplifies a case in which another display (1930) is further included on a side (1920) opposite to the side (1910) on which the second display area (262) is arranged, but the side on which the other display is arranged may not be limited thereto. For example, another display may be further included on a side opposite to the side of the third display area (263).

FIG. 20 is a drawing for explaining an example of displaying summary information in an e-type foldable electronic device according to one embodiment.

Referring to FIG. 20, in a state where content of different apps (e.g., the first app to the third app) is displayed through each of the first to third display areas (261, 262, 263) as illustrated in FIG. 20(a), the number of areas of the display that are shown in a partially folded or fully folded state by a folding action may be 1, and the number of areas of the display that are shown in a fully unfolded state may be 3. If a fourth display area is further included, the display areas shown may be different as in FIG. 20(b) or FIG. 20(c) due to rotation. For example, as illustrated in FIG. 20(b), when the first display area (261) is folded to face the second display area (262), the electronic device (201) can display summary information (e.g., first screen summary (2010), second screen summary (2020)) related to the first and second display areas (261, 262) on one area of the third display area (263).

If the electronic device (201) is rotated, as illustrated in FIG. 20(c), even if the folding state is maintained, the electronic device (201) may determine another display (1930) (e.g., the fourth display area (264)) arranged on the opposite side from the second display area (262) as the display area to be shown. Accordingly, as illustrated in FIG. 20(c), information summarizing the contents of different apps (e.g., the first app to the third app) that were output through each of the first to third display areas (261, 262, 263) (e.g., the first screen summary (1060), the second screen summary (2070), the third screen summary (2080)) may be displayed on a portion of the fourth display area (264).

If a fully folded state in which the third display area (263) is folded to face the second display area (262) and the first display area (261) is folded to face the third display area (263) is detected, the display of the summarized information (e.g., the first screen summary (2030), the second screen summary (2040), the third screen summary (2050)) can be maintained on a part of the fourth display area (264), as shown in FIG. 20(d).

If folding of all folding axes (or hinges) is detected in the Z-type foldable electronic device, the display of content output through the first display area (261) is maintained when folding out, but in the case of the E-type foldable electronic device, if folding of all folding axes (or hinges) is detected, an operation of additionally generating summary information for content of different apps (e.g., the first app to the third app) through the first to third display areas (261, 262, 263) as in FIG. 20(d) may be required.

FIG. 21 is a drawing for explaining an example of displaying summary information in a slideable electronic device according to one embodiment, and FIG. 22 is a drawing for explaining an example of displaying summary information in a single foldable electronic device according to one embodiment.

Referring to FIG. 21, when the electronic device (201) displays the content of the first app in the existing display area (2101) and the content of the second app in the expanded (or externally exposed) display area (2102) in a fully expanded state (or slide-out state) as in FIG. 21(a), the electronic device (201) can determine the display area to be shown and the display area not to be shown by the slide-in operation. Since the display area (2102) is no longer externally exposed by the slide-in operation, the electronic device (201) can generate summary information based on the content of the second app that was displayed on the display area (2102). The electronic device (201) can display the summary information (2110) on a portion of the display area (2101). In addition, in a state where the contents of different apps are output through each of the first and second display areas (2210, 2220) separated by the folding axis as in FIG. 22(a), when folding is detected, the electronic device (201) may generate information summarizing the contents (e.g., contents of the first app and contents of the second app) of the display areas (e.g., the first and second display areas (2210, 2220)) that have been folded and become invisible, and display the generated summarized information (e.g., first screen summary (2240), second screen summary (2250)) on another display area (2230).

As described above, even if the form of the electronic device is different, according to one embodiment, the visible/invisible display area can be determined based on movements such as folding or sliding of the electronic device (201). Therefore, if movements such as folding or sliding of the electronic device (201) can be detected, the visible/invisible display area can be determined, and thus, it is possible to provide summary information through the visible display area.

FIG. 23 is a diagram illustrating an example of providing additional functions when summarizing two or more apps according to one embodiment.

Referring to FIG. 23, when different applications (e.g., an interactive application, a calendar application) are running as in FIG. 23(a) and FIG. 23(b), if a movement such as folding is detected in the electronic device (201), each application can be individually analyzed to generate different summary information, but two applications running in a display area that is not visible in response to a user's setting or request or a suggestion from an on-device AI may not be summarized separately, and summary information can be generated and output based on the annual relationship between the two applications as in FIG. 21(c).

For example, if a user is messaging a friend about their travel itinerary through an unseen display area and simultaneously checking their schedule through a calendar application, on-device AI can generate a summary that links the two applications.

FIG. 24 is a diagram showing an example of providing summary information when an app is running in a multi-window state in an unfolded state according to one embodiment.

As shown in FIGS. 24(a) to 24(c), when operating in a multi-window state in an unfolded state, the electronic device (201) may perform an operation of generating and outputting summary information in a similar manner to that described above. Referring to FIGS. 24(a) to 24(c), summary content for a display area that is not visible may be displayed in the same location regardless of the multi-window, but the arrangement of the display areas may vary depending on the priority of the application. If summary information is displayed in a visible display area, summary information for an application located at the top within the multi-window may be preferentially displayed. For example, referring to FIG. 24(a), in a state where the content (2410) of the first app and the content (2420) of the second app are displayed in a multi-window format on the first display area (261), the content of the third app is displayed on the second display area (262), and the content of the fourth app is displayed on the third display area (263), the first display area (261) can be folded to face the second display area (262). The electronic device (201) can determine the third display area (263) as a visible area and determine the first and second display areas (261, 262) as invisible areas. Accordingly, the electronic device (201) can display summarized information on at least a portion of the third display area (263). The summarized information may be summarized based on the content of the first app, the content of the second app, and the content of the third app. Based on the priority of the apps, the summary information for the content (2410) of the first app located at the top in the multi-window may be preferentially displayed on at least a portion of the third display area (263). In addition, the electronic device (201) may display the summarized information in the following order: summary information for the content (2410) of the first app, summary information for the content (2420) of the second app, and summary information for the content of the third app on at least a portion of the third display area (263).

Referring to FIG. 24(b), when the second display area (262) is not displayed due to a folding operation, summary information based on the content of the second app (2430) and the content of the third app (2440) with a higher priority positioned at the top of the window may be displayed on the display area (e.g., the first display area (261) or the third display area (263)).

Referring to FIG. 24(c), when the third display area (263) is not displayed due to a folding operation, summary information based on the content of the third app with a higher priority among the content of the third app (2450) and the content of the fourth app (2460) may be displayed on the displayed display area (e.g., the first display area (261)). FIG. 25 is a drawing for explaining an example of displaying summary information according to one embodiment.

In a state where content is output (or displayed, played) according to execution of different apps (e.g., video app, memo app) (2505, 2510) through the first and second display areas (2501, 2502) in the unfolded state as shown in FIG. 25(a), when a folding operation is detected, the electronic device (201) can give priority to the operation of the app (e.g., video app (2505)) through the displayed display area (e.g., first display area (2501)). In this case, in order to prevent the visibility of the app through the displayed display area (2501) from being lowered, the summary information (2515) generated based on the memo app may be displayed on at least a portion of the display area (2501) where the app (e.g., the video app (2505)) is displayed, as in Fig. 25(b), or the summary information may be displayed in the form of an object (2520) in a status bar located at the top of the displayed display area (e.g., the first display area (2501)), as in Fig. 25(c). In this way, the display method (e.g., display location, size) of the summary information may be determined based on importance, such as the priority of the app running through the displayed display area.

Even if content for the same app rather than different apps is being output, the electronic device (201) can display summarized information about the app on a portion of the visible display area, taking into account various circumstances such as the user's level of task performance, the importance of the memo content, or the visibility of the display area.

FIG. 26 is a diagram illustrating the execution of on-device AI in response to a selection of summary information according to one embodiment.

As illustrated in FIG. 26(a), in response to a user selection (e.g., double-click, click, slide, or drag input) for summary information (2610), the electronic device (201) may output a screen (2620) that drives the conversational AI, as illustrated in FIG. 26(b). As illustrated in FIG. 26(b), the conversational AI may convey (or provide) summarized content or data and wait for the user's next response.

FIG. 27 is a drawing for explaining a single display example of summary information by a unfolding action according to one embodiment, and FIG. 28 is a drawing for explaining a plurality of display examples of summary information by a unfolding action according to one embodiment.

As illustrated in FIGS. 27(a) to 27(d), when switching from a folded state or a partially folded state to an unfolded state, the electronic device (201) can output a screen corresponding to the unfolding action. Here, a detailed description of the summary information can be displayed on the display area where the summary information is displayed with just the unfolding action. For example, when a change from a fully folded state in FIG. 27(a) to a partially folded state as illustrated in FIG. 27(b) or FIG. 27(c) is detected, the electronic device (201) can determine a display area to be shown/not shown. In FIGS. 27(b) and 27(c), the summary information output through the visible area can remain the same even if the folding action is changed. On the other hand, if a change from a partially folded state as in FIG. 27(b) or FIG. 27(c) to a fully unfolded state as in FIG. 27(d) is detected, the electronic device (201) can determine a display area to be shown/not shown. In FIG. 27(d), the first to third display areas (2701, 2702, 2703) can all be determined as visible display areas. Accordingly, the electronic device (201) can display the content of the first app through an area extended to the first and second display areas (2701, 2702) as in FIG. 27(d), corresponding to the fully unfolded state, rather than through the first display area (2701) as in FIG. 27(a) to FIG. 27(c). Additionally, the electronic device (201) may, in response to the fully unfolded state, display summary information (e.g., second screen summary (2710)) based on the content of the second app as in FIGS. 27(a) to 27(c) as the content of the original second app through the third display area (2703). Here, the content of the original second app may represent the content of the second app that was displayed through the third display area (2703) before the folding operation.

As illustrated in FIGS. 28(a) to 28(d), if two or more summary information (e.g., second screen summary (2810), third screen summary (2820)) are displayed through the first display area (2801) in the folded state, the electronic device (201) can determine the areas that are visible/not visible due to the unfolding action. In FIG. 28(d), the first to third display areas (2801, 2802, 2803) can all be determined as visible display areas. Accordingly, the electronic device (201) can display the content of the first app through the first display area (2801), the content of the second app corresponding to the summarized information (e.g., the second screen summary (2810)) through the second display area (2802), and the content of the third app corresponding to the summarized information (e.g., the third screen summary (2820)) through the third display area (2803), due to the full unfolding operation, as in FIG. 28(d).

As described above, the electronic device (2010) can maintain a screen on which a first app is executed through the first display area (2801), and can display content according to the execution of the second and third apps corresponding to summarized information (e.g., second screen summary (2810), third screen summary (2820)) through the remaining second and third display areas (2802, 2803), respectively.

Fig. 29 is a drawing showing a mounting state of an electronic device according to one embodiment.

Referring to FIG. 29, an electronic device (201) according to an embodiment can determine a display area to be shown/not shown based on a movement of the electronic device (201), such as rotation, in addition to folding. An example will be described where the first display area (2901) is not shown due to outfolding, while the second and third display areas (2902, 2903) are placed in a state where they are visually exposed (or shown) to the user.

As illustrated in FIG. 29, in order to determine the display area to be shown/invisible when there is no user input for a certain period of time in a stationary state, the electronic device (201) may utilize a running app to be used as a trigger. For example, after a certain period of time has elapsed in the stationary state, the point in time when video playback ends and the next video begins may be determined as the trigger point, thereby determining the display area to be shown/invisible even without separate input from the user.

FIG. 30 is a drawing showing a first screen when utilizing an AI assistant according to one embodiment, and FIG. 31 is a drawing showing a second screen when utilizing an AI assistant according to one embodiment.

Referring to FIGS. 30 and 31 , an AI assistant may be utilized to detect a folding state and a visible/invisible display area. For example, as illustrated in FIG. 30 (a), a situation may occur where the first display area (3001) is not visible due to outfolding, while the second and third display areas (3002, 3003) are visually exposed (or visible) to the user and remain in a positioned state for a certain period of time. In such a case, the electronic device (201) may have low accuracy in determining the visible/invisible display area. In such a case, when a user requests execution of a specific app through conversational AI, the electronic device (201) may determine a display method in response to the requested app. For example, a first display area (3001) may request the conversational AI to execute a specific app (3030) while the content (3010, 3020) of different apps (e.g., second and third apps) is displayed through the second display area (3002) and the third display area (3003), respectively.

Accordingly, the electronic device (201) can output content (3040) guiding the next operation while explaining the optimized screen ratio when executing a specific app (e.g., a third app) through conversational AI. If a user input for executing a specific app with the above screen ratio is received, the electronic device (201) can display the execution screen of the app through the entire second and third display areas (3002, 3003) as in FIG. 30(b), while displaying the previous apps (e.g., the second and third apps) displayed through each of the second and third display areas (3002, 3003) in FIG. 30(a) in the form of summary information (e.g., second screen summary (3050), third screen summary (3060)) on a portion of the current display areas (3002, 3003). At this time, the electronic device (201) can determine based on the display area (e.g., the second display area (3002) and the third display area (3003)) that shows the execution screen of the app (e.g., the third app) through conversational AI.

By utilizing conversational AI in this way, it is possible to provide summarized information (e.g., second screen summary (3050), third screen summary (3060)) through areas of the display (e.g., second display area (3002) and third display area (3003)) without physical triggering (or change in folding state).

Meanwhile, the electronic device (201) may prioritize the act of determining whether a display area is visible/invisible based on a user's request over the judgment based on learned user usage patterns. For example, the basis for the judgment may be intuitively apparent in the user's request, but may also be implicit or metaphorically included, and thus the analysis results of a large language model (LLM) may be referenced in the judgment.

Referring to FIG. 31, an example is shown in which a user plays music by executing a music app (e.g., album) (3110) through a first display area (3101) and simultaneously executes a memo app (3120) through the second and third display areas (3102, 3103). If the user usage pattern is referred to, since the album is highly utilized, the AI model can consider that the display area (e.g., the first display area (3101)) that outputs content according to the execution of the album is likely to be the display that is shown (e.g., the main display area).

However, when a user input, for example, a voice command (3130) such as 'turn to the next page' is input through conversational AI, the electronic device (201) may set a higher priority for the operation of the memo app than the album operation based on the requirements included in the voice command (3130). According to one embodiment, the electronic device (201) may determine that the priority of the user request is higher than the user usage pattern. Therefore, if predictive modeling is used, the electronic device (201) may determine that the operation of the memo app (3120) that operates according to a voice command such as 'turn to the next page' is higher than the priority of the operation of the memo app (3120) that operates according to a voice command such as 'turn to the next page', as illustrated in FIG. 31, and that the second and third display areas (3102, 3103) are likely to be visible display areas.

Figure 32 is a drawing for explaining a method of displaying summary information according to one embodiment.

Referring to FIG. 32(a), an example is shown in which, in an unfolded state, content according to the execution of a first app (e.g., a memo app) (3210) is displayed through a first display area (3201), and content according to the execution of a second app (e.g., a video app) (3220) is simultaneously displayed through a second and third display areas (3202, 3203).

If the display area where the content (3220) of the second app (e.g., video app) is displayed is folded and not visible, as illustrated in FIGS. 32(b) to 32(e), the electronic device (201) may generate summarized information based on the content of the second app (e.g., video app). For example, the electronic device (201) may display summarized information (3222) in the form of a square at least in a portion of the first display area (3201) where the content according to the execution of the first app (e.g., memo app) (3210) is displayed, as illustrated in FIG. 32(b). For example, the electronic device (201) may display summarized information (3224) in the form of a bar at the bottom of the first display area (3201) so that the content according to the execution of the first app (e.g., memo app) (3210) is not obscured, as illustrated in FIG. 32(c). For example, the electronic device (201) may display summarized information (3226) in the form of a bar at the status bar location of the first display area (3201) so that the content resulting from the execution of the first app (e.g., memo app) (3210) is not obscured, as illustrated in FIG. 32(d). For example, the electronic device (201) may display summarized information (3228) in the form of a long bar along the boundary axis of the first display area (3201) and the second display area (3202) so that the content resulting from the execution of the first app (e.g., memo app) (3210) is not obscured, as illustrated in FIG. 32(e).

According to one embodiment, the summarized information (3222, 3224, 3226, 3228) as shown in FIGS. 32(b) to 32(e) is summarized based on the content (3220) of a second app (e.g., a video app) and may include video summary content.

Here, the summarized information (3222, 3224, 3226, 3228) as shown in FIGS. 32(b) to 32(e) may be output differently according to various display methods such as display method, position, size, method, color, or font, and the display method is not limited thereto.

Meanwhile, the electronic device (201) can determine whether the on-device AI is operating when determining the visible/invisible display area. Referring to FIG. 33, if the on-device AI is operating in the invisible display area, the electronic device (201) can summarize and organize the contents of the invisible display area and output them on a portion of the visible display area. In one embodiment, the function for displaying summary information can be activated or deactivated through the settings menu.

Figure 33 is a drawing for explaining a case where summary information is not displayed according to one embodiment.

Referring to FIG. 33, FIG. 33(a) illustrates an example in which an electronic device (201) displays content of a first app (3310) through a first display area (3301), content of a second app (3320) through a second display area (3302), and content of a third app (3330) through a third display area (3303). If a folding action is detected, as illustrated in FIG. 33(b), the electronic device (201) can determine whether a display area is visible/invisible.

According to one embodiment, the electronic device (201) can determine whether the on-device AI is running in response to detection of a folding action. For example, it is necessary to determine whether the folding action is an attempt to control the operation of the on-device AI or an intention to terminate apps running in the second and third display areas (3302, 3303) that have become invisible. Accordingly, the electronic device (201) can determine whether the on-device AI is running in the second and third display areas (3302, 3303), and if it is running, can generate summarized information based on the content of the second app (3320) and the content of the third app (3330), and display the generated summarized information through the first display area (33010).

On the other hand, if the on-device AI is determined not to be running in response to detection of a folding action of the electronic device (201), the electronic device (201) can control the apps (e.g., the second app (3320) and the third app (3330)) in the non-visible display area (e.g., the second and third display areas (3302, 3303)) to move to the background of the app (e.g., the first app (3310)) running in the visible display area (e.g., the first display area (3301)) and run.

For example, when folding is detected while the first to third apps (3310, 3320, 3330) (e.g., gallery, camera, or memo app) are displayed through the respective display areas (3301, 3302, 3303) as in FIG. 33(a), it is possible to identify whether the on-device AI is running. If the on-device AI is not running, the electronic device (201) may move the second and third apps (3320, 3330) (e.g., gallery and camera apps) running in the second and third display areas (3302, 3303) to the background without displaying summary information about the second and third apps (3320, 3330) (e.g., gallery and camera apps) as in FIG. 33(b).

FIG. 34 is a diagram illustrating an example of displaying summary information according to the priority of an application according to an embodiment. Referring to FIG. 34, summarized content based on an executing app in a display area that is not visible due to a folding action is output through the visible display area. However, the folding action may indicate a user's intention to turn off the display area that is folded and not visible and use only the visible display area. Alternatively, the folding action may be intended to change the screen ratio. In this case, the electronic device (201) may summarize and display the content of the visible display area and assume that the priority of the executing app through the display area that is not visible is higher than that of other apps.

In one embodiment, when the electronic device (201) detects an outfolding between the first and second display areas (3401, 3402) as illustrated in FIG. 34(a) and determines that the second and third display areas (3402, 3403) are the visible displays, the electronic device (201) may determine an app to be displayed through the second and third display areas (3402, 3403) based on the priority of the app. For example, the priority of the first app (3410) (e.g., a video app) may be higher than the priority of the second app (3420).

In one embodiment, the electronic device (201) may not control the second app (3420) (e.g., memo app) running in the second and third display areas (3402, 3403) to operate as the main app, but may control the first app (3410) (e.g., video app) with a higher priority to operate through the display area (e.g., the second and third display areas) shown as shown in FIG. 34(b), and may generate and output summary information (e.g., memo content summary (3430)) about the second app (3420) (e.g., memo app) running in the second and third display areas (3402, 3403).

As described above, the electronic device (201) can adaptively determine the display area to be shown/not shown by considering various conditions such as priority and importance between apps and whether the operation being performed has been completed.

In one embodiment, an electronic device, a method of operating the same, and a storage medium for providing summary information can be provided, such that when a part of a display that is activated (or visually exposed to a user) is deactivated (or not visually exposed) in response to a movement such as a change in a folding state of the electronic device, information including text and/or images that were output (or displayed, reproduced) on the part of the display can be summarized and displayed on another part of the activated display.

According to one embodiment, by summarizing information output through a portion of a display that is deactivated according to the switching of the folding state of the electronic device and displaying it on another portion of the display that is activated, the user can recognize (or confirm) the contents of the application being used, and by selecting the displayed summary information, the user can easily switch to the application at a desired time.

Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store ^™ ) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

According to one embodiment, a non-transitory storage medium storing instructions, wherein the instructions, when executed by at least one processor (120, 220) of an electronic device (101), are configured to cause the electronic device to perform at least one operation, wherein the at least one operation comprises: displaying first information on a first portion of a flexible display disposed within a first housing of a foldable housing, second information on a second portion of a flexible display disposed within a second housing of the foldable housing, and third information on a third portion of the flexible display disposed within a third housing of the foldable housing, the second housing being disposed between the first housing and the third housing; and generating at least one summary information corresponding to the second information and the third information using a machine learning model, based at least in part on detecting that the third housing is folded to face the second housing, the machine learning model being trained to output summarized information based on a text or image input; and may include an action of displaying at least one summary information in the first portion.

Claims (15)

In a foldable electronic device (101, 201), A foldable housing, wherein the foldable housing is It comprises a first housing (303), a second housing (305), and a third housing (307), wherein the second housing is disposed between the first housing and the third housing; A first hinge (310) that rotatably connects the third housing to the second housing; A second hinge (320) that rotatably connects the first housing to the second housing and has a wider width than the first hinge; A flexible display (260) disposed within the first housing, the second housing, and the third housing, the flexible display including a first part (261) supported by the first housing, a second part (262) supported by the second housing, and a third part (263) supported by the third housing; At least one processor (220) comprising a processing circuit; and Includes a memory (230) for storing instructions, The above instructions, when individually or collectively executed by the at least one processor, cause the foldable electronic device to: Displaying first information in the first part, second information in the second part, and third information in the third part; At least partially based on detecting that the third housing is folded to face the second housing, generating at least one summary information corresponding to the second information and the third information using a machine learning model, wherein the machine learning model is a machine learning model trained to output summarized information based on text or image input; and A foldable electronic device configured to display at least one summary information in the first portion. In the first paragraph, at least one sensor is further included, The at least one sensor includes one of a proximity/light sensor, a grip sensor, a capacitive touch sensor, a Hall IC, an acceleration sensor, or a gyro sensor, The above instructions cause the foldable electronic device to: A foldable electronic device configured to detect that the third housing is folded to face the second housing using at least one sensor. In the second paragraph, the instructions, the foldable electronic device, Based on at least a portion of the above detection, a different machine learning model is used to determine a portion to be summarized for the second information and the third information, and the different machine learning model comprises a machine learning model trained to determine a portion to be summarized using a user usage pattern for the foldable electronic device; and A foldable electronic device configured to generate at least one summary information based on at least a portion of the above judgment. In the third paragraph, the instructions, the foldable electronic device, Determining whether to generate a summary by the above other machine learning model; and A foldable electronic device configured to generate at least one summary information based at least in part on determining to generate said summary. In any one of claims 1 to 4, further comprising another display arranged on a surface opposite to the second part of the second housing, The above instructions cause the foldable electronic device to: Detecting a fully folded state in which the third housing is folded to face the second housing and the first housing is folded to face the third housing; Based on at least a portion of the above detection, generating at least one summary information corresponding to the first information, the second information, and the third information using the machine learning model; and An electronic device configured to display at least one summary information on the other display. In any one of claims 1 to 5, the instructions cause the foldable electronic device to: Detecting that the first housing is unfolded relative to the third housing; Based on at least a portion of said detection, stopping displaying said at least one other summary information on said other display; and An electronic device set to display at least one summary information in the first section. In any one of claims 1 to 6, the instructions cause the foldable electronic device to: The at least one summary information is set to be displayed in a status bar located at the top of the first part, An electronic device, wherein said at least one summary information comprises at least one of text and an image. In any one of claims 1 to 7, the instructions cause the foldable electronic device to: Receiving user input for at least one summary piece of information; and An electronic device set to display the second information or the third information in the first part based on the user input. In any one of claims 1 to 8, the instructions cause the foldable electronic device to: An electronic device configured to stop displaying the at least one summary information in the first portion and display the second information in the second portion and the third information in the third portion based on detecting that the third housing is unfolded relative to the second housing. An electronic device according to any one of claims 1 to 9, wherein the machine learning model comprises a generative AI model trained using a neural network structure to generate and output summarized text or images based on a prompt input. A method for providing summary information in a foldable electronic device, An operation of displaying first information on a first portion of a flexible display disposed within a first housing of a foldable housing, second information on a second portion of a flexible display disposed within a second housing of the foldable housing, and third information on a third portion of the flexible display disposed within a third housing of the foldable housing, wherein the second housing is disposed between the first housing and the third housing; An operation of generating at least one summary information corresponding to the second information and the third information using a machine learning model, at least in part based on detecting that the third housing is folded to face the second housing, wherein the machine learning model is trained to output summarized information based on text or image input; and A method for providing summary information, comprising an action of displaying at least one summary information in the first portion. In Article 11, A method for providing summary information, comprising an action of detecting that the third housing is folded to face the second housing using at least one sensor of the foldable electronic device. In claim 11 or 12, An operation of determining a portion to be summarized using a different machine learning model for the second information and the third information based on detecting that the third housing is folded to face the second housing, wherein the different machine learning model includes a machine learning model trained to determine a portion to be summarized using a user usage pattern for the foldable electronic device; and A method for providing summary information, comprising an operation of generating at least one summary information based on determining a portion to be displayed as a summary. In any one of claims 11 to 13, if the foldable electronic device further includes another display arranged on a surface opposite to the second portion of the second housing, An operation of detecting a fully folded state in which the third housing is folded to face the second housing and the first housing is folded to face the third housing; An operation of generating at least one summary information corresponding to the first information, the second information, and the third information using the machine learning model; and A method for providing summary information, comprising the action of displaying at least one summary information on the other display. In a non-transitory storage medium storing instructions, the instructions, when executed by at least one processor (120, 220) of an electronic device (101, 201), are configured to cause the electronic device to perform at least one operation, wherein the at least one operation is: An operation of displaying first information on a first portion of a flexible display disposed within a first housing of a foldable housing, second information on a second portion of a flexible display disposed within a second housing of the foldable housing, and third information on a third portion of the flexible display disposed within a third housing of the foldable housing, wherein the second housing is disposed between the first housing and the third housing; An operation of generating at least one summary information corresponding to the second information and the third information using a machine learning model, at least in part based on detecting that the third housing is folded to face the second housing, wherein the machine learning model is trained to output summarized information based on text or image input; and A storage medium comprising an operation of displaying at least one summary information in the first portion.