WO2025116284A1 - Electronic device and operation method thereof - Google Patents

Abstract

An electronic device according to one embodiment comprises: a display; a memory for storing one or more instructions; and at least one processor for executing the one or more instructions. The at least one processor may execute the one or more instructions to: control the display to display a first execution screen of game content; control the display to display a second execution screen on the basis of a request for a menu through which a plurality of functions related to the game content can be set; and control the display to display the menu including only at least one function, which can be provided on the second execution screen, among the plurality of functions on the basis of the first execution screen and the second execution screen.

Description

Electronic device and method of operation thereof

Various embodiments relate to an electronic device capable of streaming a running screen of content running on a server and a method of operating the same.

Due to technological developments in display devices, display devices capable of implementing various functions are being developed.

A typical example of a display device is a TV. Conventional TVs simply received broadcast signals and played corresponding broadcast content (e.g., news, dramas, entertainment programs, etc.).

Recently, due to the development of display device technology, various applications or programs can be stored and installed on TVs to perform various functions, and various functions and services can be provided in addition to the broadcast content playback function through the installed applications. For example, a game application can be stored and installed on a TV, and game content can be executed through the game application. In addition, as the number of game users increases, the demand to enjoy game content on a large-screen TV for game immersion is also increasing.

Cloud gaming is a game service that performs major calculations, such as graphic processing, required for the game on a cloud server, transmits user operation inputs from user devices such as smartphones or smart TVs to the cloud server, and displays the rendered game screen by receiving it as IP streaming on the user device.

An electronic device according to one embodiment may include a display, a memory storing one or more instructions, and at least one processor executing the one or more instructions.

The at least one processor can control the display to display a first execution screen of game content by executing the one or more instructions.

The at least one processor can control the display to display a second execution screen based on a menu request that can set a plurality of functions related to the game content by executing the one or more instructions.

The at least one processor can control the display to display the menu including only at least one function that can be provided on the second execution screen among the plurality of functions, based on the first execution screen and the second execution screen, by executing the one or more instructions.

A method of operating an electronic device according to one embodiment may include a step of displaying a first execution screen of game content.

A method of operating an electronic device according to one embodiment of the present invention may receive a menu request capable of setting a plurality of functions related to the game content.

A method of operating an electronic device according to one embodiment may include a step of displaying a changed second execution screen based on the menu request.

A method of operating an electronic device according to one embodiment may include a step of displaying a menu including only at least one function that can be provided on the second execution screen among the plurality of functions, based on the first execution screen and the second execution screen.

FIG. 1 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to display a content screen running on a server.

FIGS. 2 to 5 are drawings showing examples of how a displayed game screen changes when an electronic device according to one embodiment receives a quick panel execution request.

FIG. 6 is a flowchart illustrating a method of operating an electronic device according to one embodiment.

FIG. 7 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to analyze a changed game screen in response to a quick panel execution request.

FIG. 8 is a diagram illustrating an operation of an electronic device collecting game pause screen data according to one embodiment.

FIG. 9 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

FIG. 10 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to collect information on a sub-window corresponding to game content.

FIG. 11 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

FIG. 12 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

FIG. 13 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

FIG. 14 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to store information on functions that can be provided for each game content.

FIG. 15 is a flowchart illustrating a method of operating an electronic device according to one embodiment.

FIG. 16 is a drawing showing an operation of an electronic device according to one embodiment of the present invention to output a notification message when setting a location to apply a function to a game screen.

FIG. 17 is a diagram illustrating a system including an electronic device and a server according to one embodiment.

Figure 18 is a block diagram showing the configuration of a server according to one embodiment.

FIG. 19 is a block diagram showing the configuration of an electronic device according to one embodiment.

Fig. 20 is a block diagram showing the configuration of an electronic device according to one embodiment.

The terms used in this specification will be briefly explained, and the present invention will be described in detail.

The terms used in the present invention are selected from the most widely used general terms possible while considering the functions of the present invention, but they may vary depending on the intention of engineers working in the field, precedents, the emergence of new technologies, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meanings thereof will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the overall contents of the present invention, rather than simply the names of the terms.

When a part of the specification is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated. In addition, terms such as "part", "module", etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software.

Hereinafter, with reference to the attached drawings, embodiments are described in detail so that those skilled in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are assigned similar drawing reference numerals throughout the specification.

The term "user" in the embodiments of this specification means a person who controls a system, function or operation, and may include a developer, an administrator or an installer.

Additionally, in the embodiments of the present specification, 'image' or 'picture' may represent a still image, a moving image composed of a plurality of consecutive still images (or frames), or a video.

FIG. 1 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to display a content screen running on a server.

Referring to FIG. 1, an electronic device (100) according to one embodiment may provide a gaming hub service, i.e., a sub-service for a game that provides connection and control of cloud games, game consoles, etc., optimized settings of picture quality or sound suitable for games, and operating functions of related applications. The electronic device (100) may provide a gaming hub service by executing an application corresponding to the gaming hub service.

An electronic device (100) according to one embodiment of the present invention can search for and execute games available on cloud games and console game devices based on a gaming hub service, and perform various game-related settings, external device control, etc. Cloud gaming refers to a game service in which a user's operation input from a user terminal is transmitted to a cloud server, and the cloud server executes major operations such as graphic processing according to the user's operation input, thereby rendering a game screen, and IP-streaming the rendered game screen to the user terminal.

An electronic device (100) according to one embodiment may be connected to a server (200) via wireless communication. An electronic device (100) according to one embodiment may be a fixed or portable device. For example, the electronic device (100) may include an electronic device capable of displaying image content, video content, game content, graphic content, etc., including a display. Alternatively, the electronic device (100) may include various types of electronic devices capable of receiving and outputting content, such as a television such as a network TV, a smart TV, an Internet TV, a web TV, an IPTV, a computer such as a desktop, a laptop, a tablet, a smartphone, a cellular phone, a game player, a music player, a video player, medical equipment, home appliances, etc.

Alternatively, the electronic device (100) according to one embodiment may include an output port for outputting video/audio signals to an external display device so as to output various contents through an external display device including a display. At this time, the output port may include ports for transmitting video and audio signals together and ports for outputting video signals and audio signals separately. In addition, the electronic device (100) may transmit video and audio signals corresponding to various contents to the external display device through wireless communication. However, the present invention is not limited thereto.

The server (200) according to one embodiment may be a cloud server or a game server, but is not limited thereto. The electronic device (100) according to one embodiment may transmit a request for executing game content to the server (200). When the server (200) receives a request for executing game content from the electronic device (100), it may execute the game content and transmit a result image or result screen rendered according to the execution of the game content to the electronic device (100) through a communication network. The electronic device (100) may control the result screen (30) received from the server (200) to be displayed on the display.

The controller (300) according to one embodiment may include various electronic devices such as a remote controller, a game controller, a smartphone, etc. The controller (300) may transmit an operation command for controlling content corresponding to a content execution screen displayed on the electronic device (100) or an application providing the content to the server (200) or to the electronic device (100) through a communication network.

A user can operate a controller (300) to control content or an application providing content while viewing a content execution screen displayed on an electronic device (100), and the controller (300) can transmit an operation command according to such user operation to the electronic device (100) or server (200).

An electronic device (100) according to one embodiment may provide a quick panel while game content is being executed. For example, the quick panel may include a menu that provides frequently used functions within the game, convenient functions provided in relation to the game, various settings related to the game, etc. The quick panel may allow the user to quickly and conveniently access necessary menus without having to move to a complicated menu screen or navigate the screen.

An electronic device (100) according to one embodiment may receive a quick panel execution request while game content is being executed, and control the display of the quick panel. When a quick panel execution request is received, the execution screen of the game content displayed on the electronic device (100) may be changed. This will be described in detail with reference to FIGS. 2 to 5.

As the execution screen of the game content according to one embodiment is changed, there may be functions among the multiple functions provided in the quick panel that cannot be provided in the changed execution screen. If all the multiple functions are provided in the quick panel regardless of whether they can be provided in the changed execution screen, the user may experience inconvenience in setting the corresponding functions.

Accordingly, the electronic device (100) according to one embodiment can identify functions that can be provided on a changed execution screen and provide a quick panel configured to provide only the identified functions.

FIGS. 2 to 5 are drawings showing examples of how a displayed game screen changes when an electronic device according to one embodiment receives a quick panel execution request.

Referring to FIGS. 2 to 5, an electronic device (100) according to one embodiment may receive a quick panel execution request from a controller (300). At this time, a user input for executing a quick panel may include a button input included in the controller (300) (for example, an input of pressing a button briefly or long), a touch input, a voice input, etc. However, the present invention is not limited thereto. In the case of a voice input, a voice signal may be received through a microphone included in the controller (300) or a voice signal may be received through a microphone included in the electronic device (100). However, the present invention is not limited thereto.

When receiving a request to execute a quick panel, the electronic device (100) may notify the server (200) that the input of the controller (300) cannot be transmitted to the server (200). For example, the electronic device (100) may transmit data corresponding to a pause event caused by executing a quick panel to the server (200).

When a server (200) according to one embodiment receives data corresponding to a stop event due to execution of a quick panel from an electronic device (100), the server (200) can perform a preset operation and perform different operations depending on the game content being executed.

For example, referring to FIG. 2, when the server (200) receives data corresponding to a stop event during execution of the first game content, it may transmit a stop menu screen (201) corresponding to the first game content to the electronic device (100). The electronic device (100) may display the stop menu screen (201) received from the server (200), and the stop menu screen (201) may be displayed in full screen. When the stop menu screen (201) is displayed in full screen, the game progress screen may not be displayed on the display. In addition, the electronic device (100) may display a quick panel (202).

In this way, when the stop menu screen (201) is displayed in full screen, some of the functions available in the quick panel (202) that require setting on the game progress screen may not be available.

In addition, referring to FIG. 3, when the server (200) receives data corresponding to a stop event during execution of the second game content, the server (200) may pause the game and transmit the paused game screen to the electronic device (100). Alternatively, the server (200) may transmit the game screen in progress to the electronic device (100) while continuing the game without pausing the game.

The electronic device (100) can be controlled to display a game progress screen (302) received from the server (200). In this way, when the quick panel (301) is executed, the game progress screen (302) is displayed as is, so functions that require settings on the game progress screen (302) can be used.

Alternatively, referring to FIG. 4, if the server (200) receives data corresponding to a stop event during execution of the third game content, the server (200) may stop the game progress and execute a stop menu (410) corresponding to the third game content. In addition, the server (200) may transmit a game screen including the stop menu (410) to the electronic device (100). The electronic device (100) may be controlled to display a game screen (420) received from the server (200). At this time, the game screen (420) may be a screen including the stop menu (410) in a portion of the screen.

Accordingly, some areas of the game screen (420) may be covered by the stop menu (410), and some of the functions available in the quick panel (430) that require settings in the covered area may not be available.

In addition, referring to FIG. 5, when the server (200) receives data corresponding to a stop event during execution of the fourth game content, the server (200) may stop the game progress and change the color, transparency, shade, etc. of the stopped game screen. However, the present invention is not limited thereto.

In addition, the server (200) can transmit the changed game screen to the electronic device (100). The electronic device (100) can be controlled to display the game screen (510) received from the server (200). At this time, the game screen (510) can be a screen with changed color, transparency, shade, etc.

Accordingly, some areas of the game progress screen may not be identifiable due to the color, transparency, shading, etc. of the game screen (510), and some functions that require settings in some areas that are not identifiable among the functions available in the quick panel may not be used.

As illustrated and described in FIGS. 2 to 5, the changed game screen displayed on the display after a quick panel request according to one embodiment may vary depending on the game content, and the functions that can be provided may vary depending on the changed game screen.

Accordingly, the electronic device (100) according to one embodiment can identify functions that can be provided in the changed game screen by analyzing the changed game screen, and provide a quick panel that includes only the identified functions.

Hereinafter, with reference to the drawings, a method for identifying functions that an electronic device (100) can provide according to one embodiment and providing a quick panel configured to provide only the identified functions will be described in detail.

FIG. 6 is a flowchart illustrating a method of operating an electronic device according to one embodiment.

An electronic device (100) according to one embodiment may request execution of game content to a server (200) based on a gaming hub service. In response to the request for execution of game content, the server (200) may execute the game content and transmit a result image or result screen rendered according to the execution of the game content to the electronic device (100).

Accordingly, the electronic device (100) can display an execution screen (game screen) of game content.

Referring to FIG. 6, an electronic device (100) according to one embodiment may receive a quick panel execution request (S610). For example, the electronic device (100) may receive a quick panel execution request while displaying a game screen. The quick panel may include a menu that provides frequently used functions in the game, convenient functions provided in relation to the game, various settings related to the game, etc.

The electronic device (100) can receive user input to execute a quick panel including button input, touch input, voice input, etc. included in the controller (300), but is not limited thereto.

According to one embodiment, when a quick panel execution request is received, the electronic device (100) can obtain a first game screen (S620).

For example, the electronic device (100) may capture a first game screen displayed within a preset time range based on a first point in time or a first point in time when a quick panel execution request is received. The first game screen may include a screen on which a game is in progress.

An electronic device (100) according to one embodiment can transmit data corresponding to a pause event according to a quick panel execution request to a server (200) (S630).

For example, when receiving a quick panel execution request, the electronic device (100) can notify the server (200) through a stop event that the input of the controller (300) cannot be transmitted to the server (200).

According to one embodiment, when a server (200) receives a stop event, it can perform a preset operation, and as the preset operation is performed, the execution screen of the game content can be changed.

Alternatively, even if the server (200) receives a stop event, it may control the game to continue without stopping the game, depending on the game content.

The server (200) can transmit the changed game screen to the electronic device (100), and the electronic device (100) can display the changed game screen (e.g., a second game screen).

An electronic device (100) according to one embodiment can analyze a second game screen (S640) based on the first game screen acquired in step 620 (S620). By analyzing the second game screen, the electronic device (100) can identify whether the entire second game screen is a menu screen or whether at least a part of the second game screen includes a game progress screen (S650).

The electronic device (100) may not provide functions related to the game progress screen in the quick panel when the entire second game screen is a menu screen (when the second game screen does not include a game progress screen) (S660). For example, the electronic device (100) may not provide functions that need to be set in the game progress screen.

On the other hand, if the entire second game screen is not a menu screen (at least a part of the second game screen includes a game progress screen), the electronic device (100) can identify functions that can be provided on the second game screen among functions related to the game progress screen (S670).

For example, the electronic device (100) can identify functions that can be provided on the second game screen based on the first game screen and the second game screen. The electronic device (100) can set the game progress screen among the second game screens as a candidate area by comparing the first game screen and the second game screen, and can detect a sub-window in the candidate area. The electronic device (100) can obtain sharpness information of a sub-window included in the second game screen based on a sub-window included in the first game screen. The electronic device (100) can identify whether a function corresponding to the sub-window can be provided based on the sharpness information. This will be described in detail with reference to FIGS. 9 and 11.

In addition, the electronic device (100) can obtain similarity information between the sub-window included in the second game screen and the sub-window included in the first game screen. Based on the similarity information, the electronic device (100) can identify whether a function corresponding to the sub-window can be provided. This will be described in detail with reference to FIG. 12.

In addition, the electronic device (100) can identify a function that can be provided in a remaining area excluding an area corresponding to a sub-window included in the first game screen from a candidate area (e.g., a game progress screen) included in the second game screen. For example, the electronic device (100) can identify a function that can be provided in the remaining area based on at least one of the size, position, shape, and clarity information of the remaining area. This will be described in detail with reference to FIG. 13.

In addition, the electronic device (100) can recognize a sub-window (e.g., a mini-map, etc.) included in the second game screen by comparing and analyzing the first game screen and the second game screen when the second game screen includes a screen where the game continues to progress. The electronic device (100) can identify functions corresponding to the sub-window recognized in the second game screen as being capable of being provided.

However, the method by which the electronic device (100) identifies functions that can be provided on the second game screen among the functions related to the game progress screen is not limited to the examples described above.

An electronic device (100) according to one embodiment can provide functions identified in step 670 (S670) and store analysis data corresponding to game content (S680).

For example, the electronic device (100) can be controlled to display a quick panel configured to provide only the functions identified in step 670 (S670) among the functions related to the game progress screen that can be provided in the quick panel.

In addition, the electronic device (100) may display the location and size of the quick panel differently based on the analysis results of the second game screen. For example, the electronic device (100) may display the quick panel in the menu screen area of the second game screen, not the game progress screen, so as not to disturb the user from viewing the game progress screen.

Alternatively, the position and size of the quick panel may be set or changed based on user input, but is not limited thereto.

Additionally, the electronic device (100) can store information about the functions identified in step 670 (S670) together with information about the game content and information about the sub-window of the game screen corresponding to the game content.

FIG. 7 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to analyze a changed game screen in response to a quick panel execution request.

Referring to FIG. 7, an electronic device (100) according to one embodiment can analyze whether the entire second game screen is a menu screen or whether at least a portion of the second game screen includes a game progress screen based on a first game screen (710) acquired before a quick panel execution request and a second game screen (720) changed in response to the quick panel execution request.

An electronic device (100) according to one embodiment may include a screen analysis unit (730). The screen analysis unit (730) may include appropriate logic, circuitry, interfaces, and/or codes that operate to identify whether the entire second game screen is a menu screen or whether at least a portion of the second game screen includes a game progress screen.

For example, the screen analysis unit (730) can identify whether the entire second game screen is a menu screen or includes a portion of the game progress screen by using artificial intelligence (AI) technology based on a neural network.

Artificial intelligence technology may include visual understanding technology. Visual understanding is a technology that recognizes and processes objects like human vision, and may include object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image enhancement.

An electronic device (100) according to one embodiment may input information about a first game screen (710), a second game screen (720), and game content into a pre-trained machine learning model to identify whether the entire second game screen (720) is a menu screen or whether at least a portion of it includes a game progress screen. The information about the game content may include a game content name, genre information, etc.

In addition, if the screen analysis unit (730) identifies that the entire second game screen is a menu screen, it can re-verify whether the entire second game screen is a menu screen using the game pause screen database (DB) (740). The game pause screen DB (740) will be described in detail with reference to FIG. 8.

FIG. 8 is a diagram illustrating an operation of an electronic device collecting game pause screen data according to one embodiment.

Referring to FIG. 8, an electronic device (100) according to one embodiment can obtain information (e.g., game progress screen data) on a game progress screen (810) during game progress.

In addition, when a pause input occurs during game play, the electronic device (100) can obtain information (e.g., pause screen data) about a screen (820) displayed after a preset amount of time has elapsed based on the time at which the pause input occurs.

The electronic device (100) can store information about game content (e.g., game content name, genre information, etc.), game progress screen data, and still screen data in a game still screen DB (740).

In addition, the electronic device (100) can transmit information about game content, game progress screen data, and still screen data to an external server. The external server can receive information about game content, game progress screen data, and still screen data from not only the electronic device (100) according to one embodiment, but also other electronic devices. The external server can store game progress screen data and still screen data for each game content in a game still screen DB, and the external server can transmit the game still screen DB stored in the external server to the electronic device (100) according to one embodiment.

Again, referring to FIG. 8, the electronic device (100) can re-verify whether the entire second game screen (720) is a menu screen by checking, based on the game pause screen DB (740), whether the second game screen (720) is a pause screen corresponding to game content and whether the entire pause screen corresponding to game content is a menu screen. However, the present invention is not limited thereto.

According to one embodiment, an electronic device (100) may not provide functions related to the game progress screen in the quick panel when the entire second game screen (720) is a menu screen.

FIG. 9 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

An electronic device according to one embodiment may provide a mini-map related function through a quick panel. The mini-map may include an image that displays a small map that represents the entire space where the game is played. The mini-map related function may include a function for enlarging the mini-map area or sharing it with an external device. However, the present invention is not limited thereto.

In order to use the mini-map related function according to one embodiment, the mini-map area must be set on the game screen. In order for the mini-map area to be set on the game screen, the mini-map area must be identifiable on the game screen.

Referring to FIG. 9, an electronic device (100) according to one embodiment may identify whether a mini-map area can be identified in a changed second game screen in response to a quick panel execution request.

An electronic device (100) according to one embodiment may obtain a mini-map candidate area corresponding to the game content based on information about the game content. For example, the electronic device (100) may obtain corner areas of a second game screen (910) as mini-map candidate areas based on genre information of the game content (e.g., shooting game). Alternatively, the electronic device (100) may obtain a mini-map candidate area based on information about a mini-map area corresponding to the game content stored in a game sub-window DB.

The game sub window DB will be described in detail with reference to Figure 10.

FIG. 10 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to collect information on a sub-window corresponding to game content.

Referring to FIG. 10, an electronic device (100) according to one embodiment can collect information about a sub-window from a game progress screen displayed during game progress.

The game progress screen may include a separate sub-window that is distinct from the game play video. For example, the sub-window may be displayed in a partial area of the game play video. However, this is not limited thereto.

Sub-windows may include information related to the progress of the game, services for game participants (e.g., chat windows, etc.). Sub-windows may include mini maps, skill windows, item-related windows, character-related windows, chat windows, control-related windows, etc. In addition, the types of sub-windows included in the game video may vary depending on the type of game.

Specifically, the mini-map may include a small image that displays a map that represents the entire space where the game is played. The skill window may be a status window that indicates the cool down time of a skill. The item-related window may be a window that provides information about items used in the game. For example, the item-related window may include an item inventory window. The item inventory window may display the type of items owned by the player and/or their cooldown time. The character-related window may be a window that provides information about a character playing the game. For example, the character-related window may include a character status window. The character status window may be a window that provides HP (Health points) and/or MP (Mana points), resurrection time, etc. of each of the allies and/or enemies in a game such as a battle game where allies and enemies exist. The chat window may be a window that displays chat contents of multiple people participating in the game or watching the game. Alternatively, the chat window may be a window that displays messages related to the game.

A control-related window may include a window for providing information related to a cockpit or driver's seat in a game where a game user controls an object (e.g., a flying object, a car, etc.) used in the game. For example, a control-related window may include a flight simulation cockpit window for providing information required or provided during flight or operation. A flight simulation cockpit window may display airspeed, engine status, fuel, flaps, trim, altitude status, etc.

A sub-window according to one embodiment may be displayed in a separately partitioned area on the game image. Alternatively, the sub-window may be displayed in an opaque or semi-transparent form on a partial area of the game image.

Referring to FIG. 10, an electronic device (100) according to an embodiment may store frames (1010, 1020, 1030, 1040, 1050) displayed during a game, and may extract sub-window candidate regions by performing image analysis on each of the frames (1010, 1020, 1030, 1040, 1050). According to an embodiment, the electronic device (100) may extract sub-window candidate regions by using artificial intelligence (AI) technology based on a neural network. In addition, the electronic device (100) may extract sub-window candidate regions from a plurality of frames based on features such as position, size, and shape of the sub-window.

The electronic device (100) can increase the score for each region as a sub-window candidate region is extracted.

For example, referring to FIG. 10, the electronic device (100) may extract a first region (1061), a second region (1062), and a third region (1063) as sub-window candidate regions in a first frame (1010), and increase scores of the first region (1061), the second region (1062), and the third region (1063). In addition, the electronic device may extract a first region (1061), a third region (1063), and a fourth region (1064) as sub-window candidate regions in a second frame (1020), and increase scores of the first region (1061), the third region (1063), and the fourth region (1064). The electronic device (100) can extract sub-window candidate regions from the third frame (1030), the fourth frame (1040), and the fifth frame (1050) in the same manner, and increase the scores of the corresponding regions. The electronic device (100) can analyze a plurality of frames including the first to fifth frames (1010, 1020, 1030, 1040, 1050), and identify sub-window candidate regions having a score higher than a preset value as sub-windows. The electronic device (100) can store information about the sub-window together with information about the game content in the game sub-window DB (1070). For example, the information about the game content can include the game content name, genre information, and the like. The information about the sub-window can include the type, size, location, shape, and the like of the sub-window. However, the present invention is not limited thereto.

Referring again to FIG. 9, the electronic device (100) may exclude areas other than the game progress screen in the second game screen (910) from the mini-map candidate areas based on the first game screen (920) acquired before the change.

Accordingly, the electronic device (100) can obtain the first candidate area (931) and the second candidate area (932) as the final mini-map candidate areas on the second game screen (920).

The electronic device (100) can identify a mini-map in a mini-map candidate area (e.g., a first candidate area (931) and a second candidate area (932)) based on the game sub-window DB (1070).

For example, the electronic device (100) can detect a mini-map area (940) from a mini-map candidate area based on information such as the location, size, and shape of the mini-map included in the game sub-window DB (1070). However, the present invention is not limited thereto.

When the mini-map area (940) is detected, the electronic device (100) can obtain sharpness information for the mini-map area (940) detected on the second game screen (910). For example, the electronic device (100) can compare the sharpness of the mini-map area (940, second mini-map area) of the second game screen (910) and the mini-map area (950, first mini-map area) of the first game screen (920). The electronic device (100) can calculate a ratio of the sharpness of the second mini-map area (940) and the sharpness of the first mini-map area (950). For example, the sharpness of the second mini-map area (940) can be 90% of the sharpness of the first mini-map area (950).

The electronic device (100) can obtain a sharpness score based on the sharpness information of the second mini-map area (940), and if the sharpness score is equal to or higher than a preset value, it can be identified as capable of providing a mini-map related function.

Accordingly, the electronic device (100) can provide a mini-map related function to the quick panel (960). For example, as illustrated in FIG. 9, the electronic device (100) can display an item representing a mini-map related function on the quick panel (960). However, the present invention is not limited thereto.

On the other hand, the electronic device (100) may identify that it cannot provide a mini-map related function if the sharpness score of the second mini-map area (940) is below a preset value.

Accordingly, the electronic device (100) may not provide a mini-map related function in the quick panel. For example, the electronic device (100) may not display an item indicating a mini-map related function in the quick panel or may display it in a disabled state. However, the present invention is not limited thereto.

FIG. 11 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

An electronic device according to one embodiment may provide mini-map related functions through a quick panel.

Referring to FIG. 11, an electronic device (100) according to one embodiment can identify whether a mini map area can be identified in a changed second game screen (1110) in response to a quick panel execution request.

An electronic device (100) according to one embodiment can acquire a game progress screen area (1130) in a second game screen (1110) based on a first game screen (1120) acquired before being changed.

The electronic device (100) can obtain a mini-map candidate area in the game progress screen area (1130). For example, the electronic device (100) can obtain a mini-map candidate area included in the game progress screen area (1130) based on information about the genre information of the game content (e.g., sports game). Alternatively, the electronic device (100) can obtain a mini-map candidate area included in the game progress screen area (1130) based on information about a mini-map corresponding to the game content, which is stored in the game sub-window DB (1070).

Accordingly, the electronic device (100) can obtain the first candidate area (1141) and the second candidate area (1142) as mini map candidate areas.

The electronic device (100) can identify a mini-map in a mini-map candidate area (e.g., a first candidate area (1141) and a second candidate area (1142)) based on the game sub-window DB (1070).

For example, the electronic device (100) can detect a mini-map area (1150) from mini-map candidate areas (1141, 1142) based on information about the location, shape, size, etc. of the mini-map included in the game sub-window area DB. However, the present invention is not limited thereto.

When the mini-map area (1150) is detected, the electronic device (100) can obtain sharpness information for the mini-map area (1150) detected on the second game screen (1110). For example, the electronic device (100) can compare the sharpness of the mini-map area (1150, second mini-map area) of the second game screen (1110) and the mini-map area (1160, first mini-map area) of the first game screen (1120). The electronic device (100) can calculate a ratio of the sharpness of the second mini-map area (1150) and the sharpness of the first mini-map area (1160). For example, the sharpness of the second mini-map area (1150) can be 85% of the sharpness of the first mini-map area.

The electronic device (100) can obtain a sharpness score based on the sharpness information of the second mini-map area (1150), and if the sharpness score is equal to or higher than a preset value, it can be identified as capable of providing a mini-map related function.

Accordingly, the electronic device (100) can provide a mini-map related function to the quick panel (1170). For example, as illustrated in FIG. 11, the electronic device (100) can control the quick panel (1170) to display an item representing a mini-map related function. However, the present invention is not limited thereto.

On the other hand, the electronic device (100) may identify that it cannot provide a mini-map related function if the sharpness score of the second mini-map area (1150) is below a preset value.

Accordingly, the electronic device (100) may not provide a mini-map related function in the quick panel. For example, the electronic device (100) may control not to display an item indicating a mini-map related function in the quick panel or to display it in a deactivated manner. However, the present invention is not limited thereto.

FIG. 12 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

An electronic device according to one embodiment may provide a targeting point related function through a quick panel. For example, the targeting point related function may include a function of displaying a targeting point.

The aiming point can be set in the aiming point area, and in order to set the aiming point, the aiming point area must be identifiable on the game screen.

An electronic device (100) according to one embodiment may, in response to a quick panel execution request, determine whether the game content being executed is a game that requires an aiming point related function based on genre information of the game content being executed and genre information of the game content that provides an aiming point function (e.g., a game DB (1220) that supports a virtual aiming point (VAP)).

The electronic device (100) may not provide the aiming point related function to the quick panel if the genre of the game content is a genre that does not require the aiming point related function. For example, the electronic device (100) may control the quick panel to not display an item indicating the aiming point related function or to display it in a deactivated manner. However, the present invention is not limited thereto.

On the other hand, if the genre of the game content being executed is a genre that requires a targeting point related function, the electronic device (100) can determine whether the targeting point area can be identified in the changed second game screen (1210) in response to the quick panel execution request.

The electronic device (100) can obtain an aiming point area (e.g., an aiming point area (1230)) on the second game screen (1210) based on information about the aiming point area. The electronic device (100) can obtain similarity information indicating the degree to which the aiming point area (1230) on the second game screen (1210) is similar to the corresponding area (1350) on the first game screen (1240). The electronic device (100) can calculate the degree to which the aiming point area (1230) and the corresponding area (1250) of the first game screen (1240) match. For example, the aiming point area (1230) can match the corresponding area (1250) of the first game screen (1240) by 70%.

The electronic device (100) may obtain a similarity score based on similarity information of the targeting point area (1230), and if the similarity score is greater than a preset value, determine that a targeting point-related function can be provided.

Accordingly, the electronic device (100) can provide a targeting point related function to the quick panel (1260). For example, the electronic device (100) can control the quick panel (1260) to display an item representing a targeting point related function, as shown in FIG. 12. However, the present invention is not limited thereto.

On the other hand, the electronic device (100) may determine that it cannot provide a function related to the aiming point if the similarity score of the aiming point area is less than a preset value.

Accordingly, the electronic device (100) may not provide a targeting point related function in the quick panel. For example, the electronic device (100) may control not to display an item indicating a targeting point related function in the quick panel or to display it in a deactivated manner. However, the present invention is not limited thereto.

FIG. 13 is a diagram illustrating an operation for identifying functions that can be provided in a quick panel by an electronic device according to one embodiment.

An electronic device according to one embodiment may provide game-related functions through a quick panel. For example, the game-related functions may include an information display function including an FPS (Frame Per Second) display function, a timer display function, and the like, in addition to the mini-map-related function and the aiming point-related function described in FIGS. 10 to 12. However, the present invention is not limited thereto.

Referring to FIG. 13, an electronic device (100) according to one embodiment may identify whether a timer display function can be provided on a changed second game screen (1310) in response to a quick panel execution request.

An electronic device (100) according to one embodiment may obtain a candidate area where a timer may be displayed on a second game screen (1310). For example, the electronic device (100) may obtain a border area of the second game screen (1310) as a candidate area (1315) for displaying a timer so as not to interfere with the game progress.

The electronic device (100) can exclude an area of the second game screen (1310) that is not a game progress screen from the candidate area (1315) based on the first game screen (1320) acquired before being changed.

Accordingly, the electronic device (100) can obtain the first candidate area (1330) as the final candidate area on the second game screen (1310).

The electronic device (100) can exclude an area where a sub-window is already displayed from the first candidate area (1330). For example, the electronic device (100) can detect a game sub-window from the first game screen (1320) using the game sub-window DB (1070).

Referring to FIG. 13, the electronic device (100) can exclude an area (1341) where an item window is displayed and an area (1342) where a mini map is displayed.

The electronic device (100) can identify whether the timer object (1360) can be displayed in the remaining candidate areas (1350, 1370) by comparing the size and shape of the remaining candidate areas (1350) with the size and shape of the timer object (1360). The electronic device (100) can determine an area among the remaining candidate areas (1350) where the timer object (1360) can be displayed as a final area, and can obtain sharpness information of the final area. For example, the electronic device (100) can compare the sharpness of the final area (1370) of the second game screen (1310) and the corresponding area of the first game screen (1320). The electronic device (199) can calculate a ratio of the sharpness of the final area (1370) of the second game screen and the sharpness of the first game screen. For example, the sharpness of the final area (1370) may be 70% of the sharpness of the corresponding area of the first game screen (1320).

The electronic device (100) may determine to obtain a sharpness score based on the sharpness information of the final area (1370) and provide a timer display function if the sharpness score is greater than or equal to a preset value.

Accordingly, the electronic device (100) can provide a timer display function on the quick panel. For example, the electronic device (100) can display an item indicating the timer display function on the quick panel. However, the present invention is not limited thereto.

On the other hand, the electronic device (100) may determine not to provide the timer display function if the sharpness score of the final area (1370) is less than a preset value.

Accordingly, the electronic device (100) may not provide a timer display function on the quick panel. For example, the electronic device (100) may not display an item indicating a mini-map related function on the quick panel, or may display it in a disabled state. However, the present invention is not limited thereto.

FIG. 14 is a diagram illustrating an operation of an electronic device according to one embodiment of the present invention to store information on functions that can be provided for each game content.

Referring to FIG. 14, an electronic device (100) according to one embodiment can identify functions that can be provided in a quick panel while executing game content. Since the operations of the electronic device (100) to identify functions that can be provided in a quick panel while executing game content have been described in detail in FIGS. 6 to 13, a detailed description thereof will be omitted.

The electronic device (100) can store the results of identifying functions that can be provided in the quick panel for each game content in a database (DB, 1440).

For example, the electronic device (100) can store information about game content (1410), information about a sub-window of the game content (1420), and information about at least one identified provideable function (1430).

At this time, information about game content (1410) may include identification information that can identify the game content (e.g., game content name), genre information, etc. In addition, information about a sub-window corresponding to the game content (1420) may include information about the type, location, size, shape, etc. of a sub-window displayed on a game screen (first game screen) acquired before the screen is changed in response to a quick panel execution request or a game screen collected when there is no quick panel execution request.

In addition, the function information (1430) that can be provided in the game content can indicate whether each of the functions that can be provided in the quick panel can be provided in the second game screen (the game screen changed by the quick panel execution request). For example, the function information (1430) that can be provided in the game content can include information that, in the case of the first game content, the aiming window display function cannot be provided, the mini-map related function can be provided, and the FPS or timer display function can be provided. Alternatively, the function information (1430) that can be provided in the game content can include information that, in the case of the second game content, the aiming window display function can be provided, the mini-map related function cannot be provided, and the FPS or timer display function can be provided. However, the present invention is not limited thereto.

According to one embodiment, when an electronic device (100) receives a request for executing a quick panel while the first game content is being executed, it can identify functions that can be provided in the quick panel. At this time, if the result data for the first game content is already stored in the DB (1440) of information on functions that can be provided for each game, the electronic device (100) can identify functions that can be provided in the quick panel using the result data for the first game content. This will be described in detail with reference to FIG. 15.

FIG. 15 is a flowchart illustrating a method of operating an electronic device according to one embodiment.

An electronic device (100) according to one embodiment may identify at least one function that can be provided in a quick panel during execution of the game content by using analysis data corresponding to the game content. The analysis data corresponding to the game content may include result data of analyzing at least one function that can be provided in a quick panel during execution of the game content. The analysis data according to one embodiment may include data analyzed and stored in the electronic device (100). Alternatively, the analysis data may include data analyzed in other electronic devices. The electronic device (100) according to one embodiment may receive analysis data from other electronic devices or a server. However, the present invention is not limited thereto.

Referring to FIG. 15, an electronic device (100) according to one embodiment may receive a quick panel execution request when executing game content (S1510). For example, the electronic device (100) may receive a quick panel execution request while displaying a game screen. The quick panel may include a menu that provides frequently used functions in the game, convenient functions provided in relation to the game, various settings related to the game, etc.

The electronic device (100) can receive user input to execute a quick panel including button input, touch input, voice input, etc. included in the controller (300), but is not limited thereto.

According to one embodiment, when a quick panel execution request is received, the electronic device (100) can obtain a first game screen (S1520).

For example, the electronic device (100) may capture a first game screen displayed within a preset time range based on a first point in time or a first point in time when a quick panel execution request is received. The first game screen may include a screen on which a game is in progress.

An electronic device (100) according to one embodiment can determine whether there is first analysis data (e.g., data on at least one function that can be provided in a quick panel) corresponding to a running game content (first game content) (S1530).

If the first analysis data does not exist, the electronic device (100) can perform steps 630 (S630) to 680 (S680) described in FIG. 6. For example, the electronic device (100) can transmit data corresponding to a stop event according to a quick panel execution request to the server (200), and if the game screen is changed, can analyze the changed second game screen based on the first game screen acquired in step 1520 (S1520). In addition, the electronic device (100) can identify at least one function that can be provided on the second game screen among the functions that can be provided on the quick panel based on the analysis result. Since this has been described in detail with reference to FIGS. 6 to 14, a detailed description thereof will be omitted.

On the other hand, the electronic device (100) according to one embodiment can determine whether the sub-window included in the first game screen acquired in step 1520 (S1520) matches the sub-window corresponding to the first game content included in the first analysis data if the first analysis data exists (S1540).

For example, the electronic device (100) can determine whether the type, position, size, shape, etc. of the sub-window included in the first game screen matches the type, position, size, shape, etc. of the sub-window corresponding to the first game content included in the first analysis data. The type, position, size, shape, etc. of the sub-window displayed on the game content execution screen can be changed by the user. Therefore, even if it is the same first game content, the type, position, size, shape, etc. of the sub-window can be changed. If the type, position, size, shape, etc. of the sub-window are different from the type, position, size, shape, etc. of the sub-window included in the first analysis data, at least one function that can be provided using the quick panel cannot be identified using the first analysis data.

Accordingly, if the sub-window included in the first game screen does not match the sub-window corresponding to the first game content included in the first analysis data, the electronic device (100) can perform steps 630 (S630) to 680 (S680) described in FIG. 6.

On the other hand, if the sub-window included in the first game screen matches the sub-window corresponding to the first game content included in the first analysis data, the electronic device (100) can provide at least one function that can be provided through the quick panel based on information about at least one function that can be provided included in the first analysis data (S1550).

FIG. 16 is a drawing showing an operation of an electronic device according to one embodiment of the present invention to output a notification message when setting a location to apply a function to a game screen.

Referring to FIG. 16, an electronic device (100) according to one embodiment may receive a quick panel execution request while a game is in progress. For example, the electronic device (100) may receive a quick panel execution request while displaying an execution screen (game screen, 1610) of game content. The electronic device (100) may receive a user input for executing a quick panel including a button input, a touch input, a voice input, etc. included in the controller (300). However, the present invention is not limited thereto.

The electronic device (100) may control to display a quick panel (1620) on the game screen in response to a quick panel execution request. The quick panel (1620) according to one embodiment may include a menu that provides frequently used functions in the game, convenient functions provided in relation to the game, various settings related to the game, etc.

The electronic device (100) can receive an input for selecting a first item (1630) that displays an aiming point among the menus included in the quick panel (1620). In response to the first item selection input, the electronic device (100) can control an aiming window object (1640) to be displayed on the screen. The user can move and set the aiming window object (1640) on the game screen through the controller (300).

The electronic device (100) according to one embodiment may output a notification message if the location set by the user is not a suitable location for setting the targeting window object (1640). For example, if the targeting window object (1640) is set in an area (1650) where information notifications provided by the electronic device (100) or information notifications provided during execution of game content are displayed, the user's game progress may be interrupted due to the displayed notifications.

Accordingly, when the electronic device (100) receives an input for setting the targeting window object (1640) in the corresponding area (1650), it can output a notification message (1660) notifying that the corresponding area is an area where a notification, etc. can appear. Accordingly, the user can set the targeting window object (1640) in a different location.

FIG. 17 is a diagram illustrating a system including an electronic device and a server according to one embodiment.

Referring to FIG. 17, the electronic device (100) may be connected to a server (1700) via a network or short-range communication. The server (1700) according to one embodiment may be the server (200) of FIG. 1 or another separate server. The electronic device (100) according to one embodiment may transmit a first game screen and a second game screen to the server (1700). Since the first game screen and the second game screen have been described in detail in FIG. 6, a detailed description thereof will be omitted.

Alternatively, when the server (1700) is implemented as the server (200) of FIG. 1, the server (1700) can obtain a first game screen before receiving data corresponding to a stop event according to a quick panel execution request and a second game screen after receiving data corresponding to the stop event.

According to one embodiment, a server (1700) can identify whether the entire second game screen is a menu screen or whether at least a portion of the second game screen includes a game progress screen by analyzing a second game screen based on a first game screen. If the entire second game screen is a menu screen (if the second game screen does not include a game progress screen), the server (1700) can transmit data indicating that functions related to the game progress screen cannot be provided in the quick panel to the electronic device (100).

The electronic device (100) receives the data from the server (1700) and may not include functions that can be provided on the game progress screen in the quick panel.

Additionally, the server (1700) according to one embodiment can identify functions that can be provided on the second game screen based on the first game screen and the second game screen when the entire second game screen is not a menu screen (when at least a part of the second game screen includes a game progress screen).

The server (1700) can receive information about functions related to the game progress screen provided in the quick panel from the electronic device (100).

In addition, the server (1700) can identify functions that can be provided on the second game screen based on the first game screen and the second game screen. By comparing the first game screen and the second game screen, the server (1700) can set the game progress screen among the second game screens as a candidate area and detect a sub-window in the candidate area. The server (1700) can obtain sharpness information of a sub-window included in the second game screen based on a sub-window included in the first game screen. The server (1700) can identify whether a function corresponding to the sub-window can be provided based on the sharpness information.

In addition, the server (1700) can obtain similarity information between the sub-window included in the second game screen and the sub-window included in the first game screen. Based on the similarity information, the server (1700) can identify whether a function corresponding to the sub-window can be provided.

In addition, the server (1700) can identify functions that can be provided in the remaining areas except for the areas corresponding to the sub-windows included in the first game screen among the candidate areas (e.g., the game progress screen) included in the second game screen. For example, the server (1700) can identify functions that can be provided in the remaining areas based on at least one of the size, position, shape, and clarity information of the remaining areas.

The server (1700) can transmit information about functions identified as being available on the second game screen among functions related to the game progress screen provided on the quick panel to the electronic device (100).

The electronic device (100) can control to display a quick panel including only identified functions among functions related to the game progress screen that can be provided in the quick panel based on the received information.

Figure 18 is a block diagram showing the configuration of a server according to one embodiment.

Referring to FIG. 18, a server (200) according to one embodiment may include a communication unit (230), a processor (240), and a memory (250). The server (200) may be implemented by more components than the components illustrated in FIG. 17. For example, the server (200) may be equipped with a separate image processing unit for image processing of game content images executed on the server (200).

The communication unit (230) according to one embodiment may include one or more modules that enable wireless communication between the server (200) and a wireless communication system or between the server (200) and another device. For example, the communication unit (230) may perform communication with the electronic device (100) or the controller (300) according to the Internet protocol according to one embodiment. However, the present invention is not limited thereto.

The communication unit (230) according to one embodiment may receive data corresponding to a stop event according to a quick panel execution request from the electronic device (100). The memory (250) according to one embodiment may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, an SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

A processor (240) according to one embodiment may control the overall operation of the server (200). For example, the processor (240) may perform an operation or function of the server (200) described in the embodiment of the present disclosure by executing one or more instructions or programs stored in the memory (250).

According to one embodiment, a processor (240) may receive a game content execution request from an electronic device (100), load and execute the requested game content, and transmit an image of the execution result to the electronic device (100) by executing one or more instructions stored in a memory (250).

According to one embodiment, the processor (240) may receive data corresponding to a stop event in response to a quick panel execution request from the electronic device (100). When data corresponding to a stop event in response to a quick panel execution request is received, a preset operation may be performed for each game content.

The block diagram of the server (200) illustrated in FIG. 18 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the server (200) actually implemented. That is, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

FIG. 19 is a block diagram showing the configuration of an electronic device according to one embodiment.

Referring to FIG. 19, an electronic device (100) according to one embodiment may include a communication unit (110), a processor (120), a memory (130), and a display (140).

The communication unit (110) according to one embodiment can transmit and receive data or signals with an external device (e.g., a server (200) or a controller (300)). For example, the communication unit (110) can include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, etc. In this case, each communication module can be implemented in the form of at least one hardware chip.

The Wi-Fi module and the Bluetooth module perform communication in the Wi-Fi and Bluetooth modes, respectively. When using the Wi-Fi module or Bluetooth module, various connection information such as the SSID and session key are first transmitted and received, and then communication is established using these, and various information can be transmitted and received. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

The communication unit (110) according to one embodiment can request the server (200) to execute game content. The server (200) can execute the game content in response to the request for execution of the game content. The communication unit (110) can receive a result image or result screen rendered according to the execution of the game content by the server (200).

According to one embodiment, the communication unit (110) can receive a control signal or control command corresponding to a user input (e.g., input of a key or button of a control device) from the controller (300).

According to one embodiment, a processor (120) controls the overall operation of the electronic device (100) and the signal flow between internal components of the electronic device (100), and performs a function of processing data.

The processor (120) may include a single core, a dual core, a triple core, a quad core, and a multiple thereof. In addition, the processor (120) may include a plurality of processors. For example, the processor (120) may be implemented as a main processor (not shown) and a sub processor (not shown).

In addition, the processor (120) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a VPU (Video Processing Unit). Or, according to an embodiment, it may be implemented in the form of a SoC (System On Chip) that integrates at least one of a CPU, a GPU, and a VPU. Or, the processor (120) may further include an NPU (Neural Processing Unit).

According to one embodiment, the memory (130) can store various data, programs or applications for driving and controlling the image processing device (200).

Additionally, the program stored in the memory (130) may include one or more instructions. The program (one or more instructions) or application stored in the memory (130) may be executed by the processor (120).

According to one embodiment, the processor (120) can control the display (140) to display an execution screen of image content received from the server (200) by executing one or more instructions stored in the memory (130).

According to one embodiment, the video content may be game content, movie content, music content, home training service content, broadcast content, etc. For example, the processor (120) may control to display an execution screen (game screen) of the game content.

According to one embodiment, the processor (120) may receive a quick panel execution request while displaying a game screen. The quick panel may include a menu that provides frequently used functions within the game, convenient functions provided in relation to the game, and various settings related to the game.

For example, the processor (120) can receive user input to execute a quick panel including button input, touch input, voice input, etc. included in the controller (300).

According to one embodiment, the processor (120) may obtain a first game screen when a quick panel execution request is received by executing one or more instructions stored in the memory (130). For example, the processor (120) may capture a first game screen displayed at a first time point when the quick panel execution request is received or within a preset time range based on the first time point. The first game screen may include a screen on which a game is in progress.

According to one embodiment, the processor (120) may control the communication unit (110) to transmit a pause event to the server (200) in response to a quick panel execution request by executing one or more instructions stored in the memory (130). For example, when receiving a quick panel execution request, the processor (120) may notify the server (200) that the input of the controller (300) cannot be transmitted to the server (200) through the pause event.

When a server (200) according to one embodiment receives data corresponding to a stop event, it can perform a preset operation, and as the preset operation is performed, the execution screen of the game content can be changed.

According to one embodiment, the processor (120) can receive a changed game screen from the server (200) through the communication unit (110). The processor (120) can control the display (140) to display the changed game screen (e.g., a second game screen).

According to one embodiment, the processor (120) can analyze a second game screen based on the acquired first game screen by executing one or more instructions stored in the memory (130).

According to one embodiment, the processor (120) may analyze the second game screen by executing one or more instructions stored in the memory (130), thereby identifying whether the entire second game screen is a menu screen or whether at least a portion of the second game screen includes a game progress screen. If the entire second game screen is a menu screen (if the second game screen does not include a game progress screen), the processor (120) may not provide functions related to the game progress screen in the quick panel. For example, the processor (120) may not provide functions that need to be set in the game progress screen.

According to one embodiment, the processor (120) can identify functions that can be provided on the second game screen among functions related to the game progress screen when the entire second game screen is not a menu screen (when at least a part of the second game screen includes a game progress screen) by executing one or more instructions stored in the memory (130).

For example, the processor (120) can identify functions that can be provided on the second game screen based on the first game screen and the second game screen. The processor (120) can set the game progress screen among the second game screens as a candidate area by comparing the first game screen and the second game screen, and can detect a sub-window in the candidate area. The processor (120) can obtain sharpness information of the sub-window included in the second game screen based on the sub-window included in the first game screen. The processor (120) can identify whether a function corresponding to the sub-window can be provided based on the sharpness information. Since this has been described in detail with reference to FIGS. 9 and 11, a detailed description thereof will be omitted.

In addition, the processor (120) can obtain similarity information between the sub-window included in the second game screen and the sub-window included in the first game screen. Based on the similarity information, the processor (120) can identify whether a function corresponding to the sub-window can be provided. Since this has been described in detail with reference to FIG. 12, a detailed description will be omitted.

In addition, the processor (120) can identify functions that can be provided in the remaining areas except for the areas corresponding to the sub-windows included in the first game screen among the candidate areas (e.g., the game progress screen) included in the second game screen. For example, the processor (120) can identify functions that can be provided in the remaining areas based on at least one of the size, position, shape, and clarity information of the remaining areas. Since this has been described in detail with reference to FIG. 13, a detailed description thereof will be omitted.

However, the method by which the processor (120) identifies functions that can be provided on the second game screen among functions related to the game progress screen is not limited to the examples described above.

According to one embodiment, the processor (120) may provide functions identified as being available on the second game screen through a quick panel by executing one or more instructions stored in the memory (130), and may store the identified functions as analysis data corresponding to game content.

For example, the processor (120) may control to display a quick panel configured to provide only identified functions among functions related to a game progress screen that can be provided in the quick panel. In addition, the processor (120) may store information about the identified functions together with information about the game content and information about a sub-window of the game screen corresponding to the game content.

A display (140) according to one embodiment converts image signals, data signals, OSD signals, control signals, etc. processed by a processor (120) to generate a driving signal. The display (240) may be implemented as a PDP, LCD, OLED, flexible display, etc., and may also be implemented as a three-dimensional display (3D display). In addition, the display (140) may be configured as a touch screen and may be used as an input device in addition to an output device.

According to one embodiment, the display (140) can display an execution screen (game screen) of game content received from the server (200). In addition, the display (140) can display a quick panel configured to provide only functions identified as being available on the second game screen.

In addition, the electronic device (100) according to one embodiment may not include a display (140), and may be controlled to display an execution screen (game screen) or a quick panel of game content received from a server (200) through a separate display device. However, the present invention is not limited thereto.

FIG. 20 is a block diagram showing the configuration of an electronic device according to one embodiment.

The electronic device (1900) of FIG. 20 may be an embodiment of the electronic device (100) described with reference to FIGS. 1 to 16.

Referring to FIG. 20, an electronic device (2000) according to one embodiment may include a tuner unit (2040), a processor (2010), a display unit (2020), a communication unit (2050), a detection unit (2030), an input/output unit (2070), a video processing unit (2080), an audio processing unit (2085), an audio output unit (2060), a memory (2090), and a power supply unit (2095).

The communication unit (2050) of Fig. 20 may correspond to the communication unit (110) of Fig. 19, the processor (2010) of Fig. 20 may correspond to the processor (120) of Fig. 19, the memory (2090) of Fig. 20 may correspond to the memory (130) of Fig. 19, and the display unit (2020) of Fig. 20 may correspond to the display (140) of Fig. 19. Therefore, the same content as described above will be omitted.

A tuner unit (2040) according to one embodiment can select only the frequency of a channel to be received by a display device (2000) from among many radio wave components by amplifying, mixing, resonating, etc. a broadcast signal received wired or wirelessly. The broadcast signal includes audio, video, and additional information (e.g., EPG (Electronic Program Guide)).

The tuner unit (2040) can receive broadcast signals from various sources, such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner unit (2040) can also receive broadcast signals from sources, such as analog broadcasting or digital broadcasting.

The communication unit (2050) according to one embodiment may receive a control signal or a control command, etc. from an external control device. For example, the communication unit (2050) may include an IR module capable of transmitting and receiving signals with the external control device according to the IR communication standard. Specifically, the communication unit (2050) may receive a control signal or a control command, etc. corresponding to a user input (e.g., a key or button input of the control device, etc.) from the control device.

A detection unit (2030) according to one embodiment detects a user's voice, a user's image, or a user's interaction, and may include a microphone (2031), a camera unit (2032), and a light receiving unit (2033).

The microphone (2031) receives a user's uttered voice. The microphone (2031) can convert the received voice into an electric signal and output it to the processor (2010). The user's voice may include, for example, a voice corresponding to a menu or function of the electronic device (2000). For example, the microphone (2031) can receive a user's voice corresponding to a display rotation command, convert the received voice into an electric signal, and output it to the processor (2010).

The camera unit (2032) can receive an image (e.g., continuous frames) corresponding to a user's motion including a gesture within the camera recognition range. The processor (2010) can use the recognition result of the received motion to select a menu displayed on the electronic device (2000) or perform a control corresponding to the motion recognition result. For example, the processor (2010) can receive an image from the camera unit (2032), recognize a user's motion corresponding to the rotation of the display from the received image, and rotate the display corresponding to the recognition result.

The optical receiver (2033) receives an optical signal (including a control signal) from an external control device through an optical window (not shown) of a bezel of the display unit (2020). The optical receiver (2033) can receive an optical signal corresponding to a user input (e.g., touch, pressing, touch gesture, voice, or motion) from the control device. A control signal can be extracted from the received optical signal under the control of the processor (2010).

The input/output unit (2070) according to one embodiment can receive video (e.g., moving images, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the electronic device (2000). The input/output unit (2070) can include any one of a High-Definition Multimedia Interface (HDMI), a Mobile High-Definition Link (MHL), a Universal Serial Bus (USB), a Display Port (DP), a Thunderbolt, a Video Graphics Array (VGA) port, an RGB port, a D-subminiature (D-SUB), a Digital Visual Interface (DVI), a component jack, and a PC port.

The processor (2010) controls the overall operation of the electronic device (2000) and the signal flow between the internal components of the electronic device (2000), and performs the function of processing data. The processor (2010) can execute an OS (Operation System) and various applications stored in the memory (2090) when there is a user input or a preset and stored condition is satisfied.

The processor (2010) may include a RAM that stores signals or data input from the outside of the electronic device (2000) or is used as a storage area corresponding to various tasks performed in the electronic device (2000), a ROM that stores a control program for controlling the electronic device (2000), and a processor.

The video processing unit (2080) performs processing on video data received by the electronic device (2000). The video processing unit (2080) can perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion on the video data.

The audio processing unit (2085) performs processing on audio data. The audio processing unit (2085) may perform various processing such as decoding, amplification, and noise filtering on audio data. Meanwhile, the audio processing unit (2085) may be equipped with a plurality of audio processing modules to process audio corresponding to a plurality of contents.

The audio output unit (2060) outputs audio included in a broadcast signal received through the tuner unit (2040) under the control of the processor (2010). The audio output unit (2060) can output audio (e.g., voice, sound) input through the communication unit (2050) or the input/output unit (2070). In addition, the audio output unit (2060) can output audio stored in the memory (2090) under the control of the processor (2010). The audio output unit (2060) can include at least one of a speaker, a headphone output terminal, or a S/PDIF (Sony/Philips Digital Interface: output terminal).

The power supply unit (20695) supplies power input from an external power source to components inside the electronic device (2000) under the control of the processor (2010). In addition, the power supply unit (2095) can supply power output from one or more batteries (not shown) located inside the electronic device (2000) to the internal components under the control of the processor (2010).

The memory (2090) can store various data, programs or applications for driving and controlling the electronic device (2000) under the control of the processor (2010). The memory (2090) can include a broadcast reception module (not shown), a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, an optical reception module, a display control module, an audio control module, an external input control module, a power control module, a power control module for an external device connected wirelessly (for example, Bluetooth), a voice database (DB), or a motion database (DB). The modules and database of the memory (2090) not shown can be implemented in the form of software to perform a broadcast reception control function, a channel control function, a volume control function, a communication control function, a voice recognition function, a motion recognition function, an optical reception control function, a display control function, an audio control function, an external input control function, a power control function or a power control function for an external device connected wirelessly (for example, Bluetooth). The processor (2010) can perform each function using the software stored in the memory (2090).

Meanwhile, the block diagrams of the electronic device (100, 2000) illustrated in FIGS. 19 and 20 are block diagrams for one embodiment. Each component of the block diagrams may be integrated, added, or omitted according to the specifications of the electronic device (100, 2000) actually implemented. That is, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

An electronic device according to one embodiment may include a display, a memory storing one or more instructions, and at least one processor executing the one or more instructions.

The at least one processor can control the display to display a first execution screen of game content by executing the one or more instructions.

The at least one processor can control the display to display a second execution screen based on a menu request that can set a plurality of functions related to the game content by executing the one or more instructions.

The at least one processor may execute the menu, which is configured to provide only at least one function that can be provided in the second execution screen among the plurality of functions, based on the first execution screen and the second execution screen, by executing the one or more instructions.

The at least one processor can control the display to display a quick panel including the menu by executing the one or more instructions.

The above multiple functions may include at least one of a mini-map related function, an aiming point display function, an FPS display function, and a timer display function.

An electronic device according to one embodiment may further include a communication unit that receives an execution screen of the content from an external device.

The at least one processor may control the communication unit to transmit data corresponding to a pause event to the external device based on receiving the menu execution request by executing the one or more instructions.

The at least one processor may control the display to display a menu that does not include at least one function that can be provided in the progress screen, if the second execution screen is identified as not including a progress screen based on the first execution screen and the second execution screen obtained by executing the one or more instructions based on receiving the menu request.

The at least one processor can identify whether the second execution screen does not include a progress screen based on the still screen information of the game content by executing the one or more instructions.

The at least one processor can obtain information on at least one of the type, position, size, and shape of the sub-window of the game content by comparing the progress screens of the game content by executing the one or more instructions.

The at least one processor can compare the acquired first execution screen with the second execution screen by executing the one or more instructions to identify whether the second execution screen does not include a progress screen.

The at least one processor may identify the progress screen among the second execution screens as a candidate area when the second execution screen is identified as including the progress screen by executing the one or more instructions.

The at least one processor can identify at least one function that can be provided on the second execution screen based on the candidate region, information about the sub-window of the game content, and the sub-window included in the first execution screen by executing the one or more instructions.

The at least one processor can obtain sharpness information of a sub-window included in the candidate area based on the sub-window included in the first execution screen by executing the one or more instructions.

The at least one processor can identify whether a function corresponding to the sub-window is available based on the clarity information by executing the one or more instructions.

The at least one processor can obtain similarity information between a sub-window included in the first execution screen and a sub-window included in the candidate area by executing the one or more instructions.

The at least one processor can identify whether a function corresponding to the sub-window is available based on the similarity information by executing the one or more instructions.

The at least one processor may identify whether the plurality of functions can be provided based on at least one of size, position, shape, and sharpness information of an area other than an area corresponding to a sub-window included in the first execution screen in the candidate area by executing the one or more instructions.

The at least one processor may store function information that can be provided for each content, including the content information, information about the at least one identified function, and information about the first execution screen, by executing the one or more instructions.

The at least one processor can identify at least one function that can be provided on the second execution screen based on the information on the functions that can be provided for each content by executing the one or more instructions.

A method of operating an electronic device according to one embodiment may include a step of displaying a first execution screen of game content.

A method of operating an electronic device according to one embodiment may include a step of displaying a second execution screen based on a menu request capable of setting a plurality of functions related to the game content.

A method of operating an electronic device according to one embodiment may include a step of displaying a menu including only at least one function that can be provided on the second execution screen among the plurality of functions, based on the first execution screen and the second execution screen.

The step of displaying the above menu may include the step of displaying a quick panel including the above menu.

The above multiple functions may include at least one of a mini-map related function, an aiming point display function, an FPS display function, and a timer display function.

The step of displaying the above menu may include a step of displaying the menu that does not include at least one function that can be provided in the progress screen, if it is determined that the second execution screen does not include a progress screen based on the first execution screen and the second execution screen acquired based on the menu request.

The operating method of an electronic device according to one embodiment may further include a step of obtaining information on at least one of a type, position, size, and shape of a sub-window of the game content by comparing progress screens of the game content.

The step of displaying the above menu may include a step of comparing the acquired first execution screen with the second execution screen to identify whether the second execution screen does not include a progress screen.

The step of displaying the above menu may include a step of identifying the progress screen among the second execution screens as a candidate area if the second execution screen is identified as including a progress screen.

The step of displaying the above menu may include a step of identifying at least one function that can be provided on the second execution screen based on information about the candidate area, the sub-window of the game content, and the sub-window included in the first execution screen.

The step of displaying the above menu may include a step of obtaining sharpness information of a sub-window included in the candidate area based on a sub-window included in the first execution screen.

The step of displaying the above menu may include a step of identifying whether a function corresponding to the sub-window can be provided based on the clarity information.

The step of displaying the above menu may include a step of obtaining similarity information between a sub-window included in the first execution screen and a sub-window included in the candidate area.

The step of displaying the above menu may include a step of identifying whether a function corresponding to the sub-window can be provided based on the similarity information.

An electronic device according to one embodiment provides, through a quick panel, only the functions that can be provided on a game screen that has changed due to a quick panel execution request, so that a user can easily set the functions and may not cause confusion to the user.

The operating method of the display device according to one embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention or may be those known to and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

In addition, the operating method of the display device according to the disclosed embodiments may be provided as included in a computer program product. The computer program product may be traded between sellers and buyers as a product.

A computer program product may include a S/W program, a computer-readable storage medium on which the S/W program is stored. For example, the computer program product may include a product in the form of a S/W program (e.g., a downloadable app) distributed electronically by a manufacturer of an electronic device or through an electronic market (e.g., Google Play Store, App Store). For electronic distribution, at least a part of the S/W program may be stored in a storage medium or temporarily created. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic market's server, or a relay server that temporarily stores the SW program.

The computer program product may include a storage medium of the server or a storage medium of the client device in a system comprising a server and a client device. Or, if there is a third device (e.g., a smartphone) that is communicatively connected to the server or the client device, the computer program product may include a storage medium of the third device. Or, the computer program product may include a S/W program itself that is transmitted from the server to the client device or the third device, or transmitted from the third device to the client device.

In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) may execute a computer program product stored on the server to control a client device in communication with the server to perform a method according to the disclosed embodiments.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims (15)

In an electronic device (100), display (140); A memory (130) storing one or more instructions; and At least one processor (120) for executing one or more of the above instructions, The at least one processor (120) executes the one or more instructions, Control the display to display the first execution screen of the game content, Controlling the display to display a second execution screen based on a menu request that can set multiple functions related to the above game content, An electronic device that controls the display to display the menu including only at least one function that can be provided on the second execution screen among the plurality of functions based on the first execution screen and the second execution screen. In the first paragraph, The at least one processor (120) executes the one or more instructions, Control the display to display a quick panel containing the above menu, An electronic device wherein the above multiple functions include at least one of a mini-map related function, an aiming point display function, an FPS display function, and a timer display function. In paragraph 1 or 2, The above electronic device, It further includes a communication unit (110) that receives the execution screen of the content from an external device, The at least one processor (120) executes the one or more instructions, An electronic device that controls the communication unit (110) to transmit data corresponding to a pause event to the external device based on the above menu request. In the second or third paragraph, The at least one processor (120) executes the one or more instructions, An electronic device that controls the display to display the menu that does not include at least one function that can be provided on the progress screen, when the second execution screen is identified as not including a progress screen based on the first execution screen and the second execution screen obtained based on the menu request. In paragraph 4, The at least one processor (120) executes the one or more instructions, An electronic device that identifies whether the second execution screen does not include a progress screen based on still screen information of the above game content. In any one of the provisions of paragraphs 2 to 5, The at least one processor (120) executes the one or more instructions, An electronic device that obtains information on at least one of the type, position, size, and shape of a sub-window of the game content by comparing the progress screens of the game content. In any one of the provisions of paragraphs 2 to 6, The at least one processor (120) executes the one or more instructions, By comparing the first execution screen obtained above with the second execution screen, it is identified whether the second execution screen does not include a progress screen, If the second execution screen is identified as including a progress screen, the progress screen among the second execution screens is identified as a candidate area, An electronic device that identifies at least one function that can be provided on the second execution screen based on the candidate area, information about the sub-window of the game content, and the sub-window included in the first execution screen. In Article 7, The at least one processor (120) executes the one or more instructions, Based on the sub-window included in the first execution screen, the sharpness information of the sub-window included in the candidate area is obtained, An electronic device that identifies whether a function corresponding to the sub-window is available based on the clarity information. In clause 7 or 8, The at least one processor (120) executes the one or more instructions, Obtaining similarity information between a sub-window included in the first execution screen and a sub-window included in the candidate area, An electronic device that identifies whether a function corresponding to the sub-window can be provided based on the above similarity information. In any one of Articles 7 to 9, The at least one processor (120) executes the one or more instructions, An electronic device that identifies whether the plurality of functions can be provided based on at least one of size, position, shape, and clarity information of an area remaining in the candidate area excluding an area corresponding to a sub-window included in the first execution screen. In any one of claims 1 to 10, The at least one processor (120) executes the one or more instructions, An electronic device storing function information that can be provided for each content, including the content information, information about at least one identified function, and information about the first execution screen. In Article 11, The at least one processor (120) executes the one or more instructions, An electronic device that identifies at least one function that can be provided on the second execution screen based on the function information that can be provided for each content. In a method of operating an electronic device, A step for displaying the first execution screen of game content; A step of receiving a menu request capable of setting multiple functions related to the above game content; A step of displaying a second execution screen based on the above menu request; and An operating method of an electronic device, comprising the step of displaying a menu including only at least one function that can be provided on the second execution screen among the plurality of functions based on the first execution screen and the second execution screen. In Article 13, The steps to display the above menu are: Comprising a step of displaying a quick panel including the above menu, A method of operating an electronic device, wherein the above multiple functions include at least one of a mini-map related function, an aiming point display function, an FPS display function, and a timer display function. One or more computer-readable recording media storing a program for performing the method of claim 13 or 14.

Images