TWI909203B - Electronic apparatus, providing translation information method and non-transitory computer readable recording medium comprising a computer program for performing the method - Google Patents

Abstract

This invention provides a method for providing translated information in an electronic device. The method may include the following steps: confirming whether information about an image corresponding to a first application has been obtained; if such image information has been obtained, obtaining translation information about the image; and providing the obtained translation information.

Description

Electronic devices, methods for providing translated information, and non-transitory computer-readable recording media, including programs for performing those methods.

This invention relates to an electronic device and its control method that translates text contained in an image based on an image corresponding to an application, so that users can more conveniently use applications that do not provide multilingual services.

With the widespread use of the Internet and the proliferation of mobile devices, users can obtain information through various applications. Depending on the type of application used, users not only receive information written in their native language, but also frequently receive information written in foreign languages that they cannot understand.

However, when users encounter foreign language texts they cannot understand, translating them manually can be troublesome and inconvenient. Therefore, to improve user convenience, a method is needed where users can more easily obtain translation information for foreign language texts included in the application.

For further information, see previous references such as KR 10-2258004 B1.

[Problem Solved by the Invention] The disclosed embodiments provide an electronic device and a method for translating information. More specifically, the object is to provide an electronic device and a method for controlling it that translates text contained in an image based on an image corresponding to an application, so that users can more easily use applications that do not provide multilingual services.

The technical problem to be solved in this embodiment is not limited to the technical problem described above; other technical problems can be deduced by analogy from the following embodiments. [Technical means to solve the problem]

One aspect of this invention provides a method for providing translation information, which includes the following steps: confirming whether information about an image corresponding to a first application has been obtained; if the information about the image has been obtained, obtaining translation information about the image; and providing the obtained translation information.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, wherein the steps for obtaining the aforementioned translation information include the following steps: while the aforementioned first application is active, a second application driven in the background is used to obtain the aforementioned translation information based on the information of the aforementioned image.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, wherein the translated information includes a second text in a second language obtained based on a first text in a first language included in the aforementioned image.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, wherein the step of confirming whether the information of the aforementioned image has been obtained includes the following steps: confirming whether a new image file has been generated in the set storage library.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, wherein the step of confirming whether the information of the aforementioned image has been obtained includes the following steps: confirming whether the image capture function has been activated.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, wherein the steps for obtaining the translation information of the aforementioned image include the following steps: providing a user interface (UI) that asks whether the aforementioned image should be translated.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, wherein the steps for obtaining the translation information of the aforementioned image include the following steps: providing a user interface that queries the translation language of the aforementioned image.

Furthermore, in one embodiment of the present invention, a method for providing translation information can be provided, wherein the steps for obtaining the translation information of the aforementioned image include the following steps: confirming whether a Type 1 input for requesting to capture an image and translating the captured image has been obtained; and the aforementioned Type 1 input is different from the Type 2 input for requesting to capture an image.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, which further includes the following steps: obtaining a translation request for an object area within the aforementioned image; and the aforementioned translation information includes the translation information of the aforementioned object area.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, which further includes the following steps: identifying time-varying areas within the aforementioned image.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, which further includes the following steps: confirming whether the aforementioned area, which varies over time, needs to be translated; and obtaining a translation request for the aforementioned area, which varies over time; and the aforementioned translation information includes translation information for the aforementioned area, which varies over time.

Furthermore, in one embodiment of the present invention, a method for providing translation information may be provided, wherein translation information of the aforementioned area that changes over time is displayed by means of a set method.

Furthermore, in one embodiment of the present invention, a method for providing translation information can be provided, wherein the aforementioned image includes an image obtained by scrolling and capturing a scrollable page, and the steps for providing translation information of the aforementioned image include the following steps: providing translation information corresponding to the area within the aforementioned page displayed on the screen as the aforementioned page is scrolled, together with the aforementioned page.

Furthermore, in one embodiment of the present invention, a method for providing translated information can be provided, wherein the aforementioned image includes an image obtained by capturing a page of the aforementioned first application. The method for providing translated information includes the following steps: confirming whether the aforementioned page has changed due to an update of the aforementioned first application; upon confirming that the aforementioned page has changed, confirming whether to translate the image corresponding to the changed aforementioned page; upon confirming that the image corresponding to the changed aforementioned page has been translated, obtaining information about the image corresponding to the changed aforementioned page; obtaining translation information about the image corresponding to the changed aforementioned page; and providing the translation information about the image corresponding to the changed aforementioned page.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, wherein the obtained translated information is displayed on the content of the first application.

Furthermore, in one embodiment of the present invention, a method for providing translated information may be provided, which further includes the following steps: obtaining input of settings for the first area in the translated information; confirming the second area in the first application corresponding to the first area based on the input of the settings; and providing a page corresponding to the second area.

Furthermore, in one embodiment of the present invention, a method for providing translated information can be provided, wherein when the size of the information of the aforementioned image differs from that of the aforementioned translated information, the setting elements within the aforementioned translated information are displayed by a setting method.

Another embodiment of the present invention provides an electronic device including a communication unit, a memory unit, and a controller unit, wherein the controller unit performs the following functions: confirming whether information corresponding to an image for a first application has been obtained; when the information of the image has been obtained, obtaining translation information of the image; and providing the obtained translation information.

Another aspect of the present invention provides a computer-readable recording medium that records programs for implementing methods carried out by an electronic device.

Specific details of other embodiments are provided in the detailed description and diagrams. [Effects of the Invention]

Based on the proposed implementation examples, one or more of the following effects can be expected.

According to the embodiments of this manual, the following technical effects can be achieved: automatically sensing whether an image has been acquired and providing translation information, thereby making it easier for users to obtain translation information of the image corresponding to the application.

Furthermore, according to the embodiments of this specification, the following technical effects can be achieved: providing translation information of areas within the image displayed on the application screen that change over time, thereby making it easier for users to obtain translation information of the image corresponding to the application.

Furthermore, according to the embodiments of this specification, the following technical effects can be achieved: capturing images and providing translation information based on the input of the type of captured image requested for translation, thereby making it easier for users to obtain translation information of images corresponding to the application.

Furthermore, according to the embodiments of this specification, the following technical effects can be achieved: providing translation information of images obtained by scrolling and capturing scrollable pages, thereby making it easier for users to obtain translation information of images corresponding to the application.

Furthermore, according to the embodiments of this specification, the following technical effects can be achieved: providing translation information of images simultaneously on the application screen, thereby making it easier for users to obtain translation information of images corresponding to the application.

The effects of this invention are not limited to those mentioned above. Those skilled in the art can clearly understand other effects not mentioned based on the description in the scope of the invention application.

The terminology used in the embodiments is selected as widely used common terms as possible with consideration for the function of this invention, but may vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies. Furthermore, in specific cases, there may be terms arbitrarily chosen by the applicant; in such cases, their meanings will be described in detail in the corresponding description. Therefore, the terminology used in this invention should be defined based on its meaning and the overall content of this invention, and not simply on its name.

Throughout this specification, when a part is referred to as "including" a certain constituent element, unless otherwise stated, it means that other constituent elements may be included, rather than excluded.

The expression "at least one of a, b and c" in the entire instruction manual may include "a alone", "b alone", "c alone", "a and b", "a and c", "b and c" or "a, b and c".

The term "terminal" as used below can be implemented by a computer or portable terminal that can be connected to a server or other terminal via a network. Here, computers include, for example, laptops, desktops, and laptops equipped with web browsers. Portable terminals are wireless communication devices that ensure portability and mobility. These can include communication terminals such as IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), as well as all types of handheld wireless communication devices such as smartphones and tablets.

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those skilled in the art to which the present invention pertains can easily implement it. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

The embodiments of the present invention will now be described in detail with reference to the figures.

Figure 1 illustrates an embodiment of the system.

Referring to FIG1, the system may include at least one of electronic device 100, first server 120, second server 140, and network 160. On the other hand, the system shown in FIG1 only illustrates the components relevant to this embodiment. Therefore, those skilled in the art related to this embodiment will understand that other common components may be included in addition to those shown in FIG1.

Electronic device 100 is a device that provides various types of information. Electronic device 100 may provide the information through web pages or application screens, or provide the information in a form that can be displayed through web pages or application screens at the receiving terminal.

According to one embodiment, electronic device 100 is a terminal used by various users, who can use their respective electronic devices 100 to connect to services provided via network 160. For example, electronic device 100 can confirm whether it has obtained information about an image corresponding to an application, and if it has obtained image information, it can obtain and provide translation information for the image.

The first server 120, as a device for providing services through applications or web pages, can provide services based on requests received from the electronic device 100, using information that can be displayed through web pages or application screens. For example, the first server 120 can provide services such as applications that allow ordering and delivery of items, or applications that allow viewing various types of content.

The second server 140, as a device that provides translation services, can generate translation information based on the image information received from the electronic device 100 and send it to the electronic device 100. For example, the second server 140 can obtain an image from the electronic device 100, translate the text in the first language included in the image into the second language, and then send the translation information including the text translated into the second language to the electronic device 100.

On the other hand, the application related to the services provided by the first server 120 may be referred to as the first application, and the application related to the services provided by the second server 140 may be referred to as the second application.

The first server 120 and the second server 140 can communicate with the electronic device 100 within the network 160. The network 160 may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof. The network is a data communication network in a comprehensive sense that enables the constituent entities of the networks shown in FIG1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile wireless communication network. Wireless communication may include, but is not limited to, wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra-wideband), infrared communication (IrDA), and NFC (Near Field Communication).

Figures 2a to 2g are diagrams illustrating the process by which an electronic device 100 of an embodiment provides translated information.

According to one embodiment, electronic device 100 can execute a first application and display the screen 200 of the first application. For example, referring to FIG2a, electronic device 100 can display a portion of the search results page when searching for "vitamin D" in the screen 200 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, referring to FIG2b, the electronic device 100 can confirm whether information such as the address of the image 210 captured from the screen 200 of the first application, or the image 210 corresponding to the screen 200 of the first application, has been acquired. At this time, the electronic device 100 can confirm whether image information has been acquired by various methods.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library. For example, when a new image file is generated in the storage library for captured images, the electronic device 100 can confirm that image information has been acquired.

According to one embodiment, electronic device 100 can confirm whether image information has been acquired based on whether the image capture function has been activated. For example, electronic device 100 can confirm whether the image capture function has been activated by hooking the image capture input. When the image capture function is confirmed to be activated, electronic device 100 can confirm that image information has been acquired.

According to one embodiment, electronic device 100 can confirm whether to translate image 210 by providing a user interface (UI) 220 that asks whether to translate image 210. For example, referring to FIG2c, when information about image 210 is obtained, electronic device 100 can display the user interface 220 asking whether to translate image 210 on the screen. Subsequently, when electronic device 100 receives user input with a "Yes" icon, electronic device 100 can obtain translation information 240 of image 210.

According to one embodiment, whether to translate the image 210 can be confirmed based on whether the input for requesting to capture an image and translate the captured image has been received. At this time, the input types for requesting to capture an image and the input types for requesting to capture an image and translate the captured image can be set differently.

For example, the input requesting to capture an image can be set to simultaneously pressing the power button and the volume down button of the electronic device 100, and the input requesting to capture an image and translate the captured image can be set to simultaneously pressing the power button and the volume up button of the electronic device 100. By obtaining the input of the setting type of requesting to capture an image and translate the captured image, the electronic device 100 can obtain the translation information 240 of the image 210 without asking the user whether to translate the image 210.

According to one embodiment, electronic device 100 can provide a user interface 230 for querying the translation language of image 210. For example, referring to FIG2d, after receiving user input that the user has selected an icon for user interface 220, electronic device 100 can display the user interface 230 for querying the translation language of image 210 on the screen. Thereafter, after receiving user input that the user has selected a language, electronic device 100 can obtain translation information 240 including the image with text translated into the selected language.

Alternatively, regardless of whether image information has been obtained, the electronic device 100 can display a user interface 230 on the screen that queries the translation language of the image. In this way, when image 210 information is obtained, the electronic device 100 can omit the process of querying the translation language and obtain translation information 240 of the image 210, including text translated into the set language.

According to one embodiment, while the first application is active, the electronic device 100 can use a second application driven in the background to obtain translation information 240 of the image 210 based on the information of the image 210. For example, the electronic device 100 uses the second application to directly translate the text in the first language included in the image 210 into the second language, thereby obtaining the translation information 240 of the image 210. Alternatively, the electronic device 100 uses the second application to send a translation request for the image 210 to an external server and then receives the translation information of the image 210 from the external server, thereby obtaining the translation information 240 of the image 210.

According to one embodiment, the translation information 240 of image 210 may include a second text in a second language obtained based on a first text in a first language included in image 210. For example, referring to FIG2e, the translation information 240 of image 210 may include English text obtained by translating Korean text included in image 210.

According to one embodiment, electronic device 100 can provide the acquired translation information 240. For example, referring to FIG2f, electronic device 100 can drive a second application in the foreground and display the translation information 240 of image 210 on the screen 250 of the second application. Alternatively, electronic device 100 can transmit the translation information 240 of image 210 to a set email address. However, this is only one embodiment, and the method by which electronic device 100 provides the acquired translation information is not limited to the above.

According to one embodiment, when the dimensions of the information in image 210 and the translated information 240 of image 210 differ, the electronic device 100 can display the set elements within the translated information 240 in a configured manner. For example, referring to Figure 2f, during the process of translating Korean text into English text, the amount of text changes, therefore the dimensions of the information in image 210 and the translated information 240 of image 210 will differ. In this case, the electronic device 100 can display the translated information 240 of image 210 on the screen 250 of the second application and highlight the bar icon 260 within the translated information 240 for user convenience. However, the elements displayed in the translated information 240 and the method used to display these elements can be set differently and can be changed according to user input, etc.

On the other hand, according to one embodiment, the electronic device 100 can confirm whether to provide the acquired translation information. For example, referring to FIG2g, the electronic device 100 can display a user interface 270 on the screen asking whether to load the acquired translation information. Subsequently, when the electronic device 100 receives user input with the "yes" icon selected, the translation information 240 of the image 210 can be displayed on the screen 250 of the second application.

Figures 3a and 3b are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, electronic device 100 can execute a first application and display the screen of the first application. Afterwards, electronic device 100 can confirm whether it has obtained information about image 300. If information about image 300 has been obtained, electronic device 100 can confirm whether to translate image 300.

According to one embodiment, electronic device 100 can obtain a translation request for an object area 320 within an image 300. For example, referring to FIG3a, when confirming the translation of image 300, electronic device 100 can obtain input selecting the area 320 within image 300 that the user wishes to translate.

According to one embodiment, electronic device 100 can obtain translation information 340 of image 300, which includes translation information of object area 320. For example, referring to Figure 3b, electronic device 100 can obtain translation information 340, which includes "Related products that other customers have bought a lot" obtained from translating "related products that other customers have bought a lot" included in object area 320. In this case, text included in areas outside object area 320 may not be translated.

According to one embodiment, electronic device 100 can provide the acquired translation information 340. For example, electronic device 100 can provide translation information 340 in various forms for only the text included in the translation target area 320.

Figures 4a to 4g are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, a specific area within screen 400 of the first application may change over time. For example, referring to Figure 4a, in area 410 within screen 400 of the first application, a coupon icon may be displayed along with the text "Get a surprise coupon!" However, referring to Figure 4b, over time, area 410 may display the text "Out of Stock" and "1-Day Special". As described above, when a specific area within screen 400 of the first application changes over time, additional translation of the corresponding area may be necessary.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, electronic device 100 can identify areas in the screen 400 of the first application that change over time. More specifically, electronic device 100 can identify time-varying areas by comparing captured images before and after the area changes, or by identifying time-varying areas based on user input.

For example, referring to Figures 4c and 4d, the electronic device 100 can acquire an image 420 obtained from the image 400 before the captured area 410 changes, and an image 430 obtained from the image 400 after the captured area 410 changes. The electronic device 100 can then compare the images 420 and 430 to identify different areas, recognizing the corresponding area as the area 410 that changes over time.

Alternatively, the electronic device 100 can identify areas that change over time by monitoring the screen 400 of the first application. More specifically, the electronic device 100 can monitor the screen 400 of the first application after acquiring the image 420 corresponding to the screen 400 of the first application. The electronic device 100 can identify areas that differ from the screen 400 when the image 420 was acquired, and identify the corresponding areas as areas that change over time 410.

Alternatively, the electronic device 100 may confirm the area that changes over time based on user input. More specifically, the electronic device 100 may obtain user input in the screen 400 of the first application or the image 420 corresponding to the screen 400 of the first application to select an area that changes over time, and confirm the selected area as the area 410 that changes over time.

According to one embodiment, electronic device 100 can determine whether to translate an area that changes over time. For example, referring to FIG4e, electronic device 100 can provide a user interface 440 that asks whether to translate an area that changes over time. Subsequently, when electronic device 100 receives user input indicating that an icon has been selected, electronic device 100 can obtain translation information 460 of image 430.

According to one embodiment, electronic device 100 can use a second application driven in the background to obtain translation information of an image based on the image information while the first application is active. For example, referring to FIG4f, electronic device 100 can use the second application to obtain translation information 450 of image 420. Also, referring to FIG4g, electronic device 100 can use the second application to obtain translation information 460 of image 430.

According to one embodiment, electronic device 100 can provide the acquired translation information. For example, electronic device 100 can drive a second application in the foreground and display translation information 450 of the image 420 captured before the change of area 410 on the screen of the second application. Alternatively, electronic device 100 can drive a second application in the foreground and display translation information 460 of the image 430 captured after the change of area 410 on the screen of the second application.

Figures 5a to 5e are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that overlaps with Figures 2a to 2g is simplified or omitted.

According to one embodiment, a specific area within screen 500 of the first application may change over time. For example, referring to Figure 5a, in area 520 within screen 500 of the first application, a coupon icon may be displayed along with the text "Get a surprise coupon!" However, referring to Figure 5b, over time, area 520 may display the text "Out of Stock" and "1-Day Special". As described above, when a specific area within screen 500 of the first application changes over time, additional translation of the corresponding area may be necessary.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, electronic device 100 can identify an area in screen 500 of the first application that changes over time. The specific action of electronic device 100 in identifying the area that changes over time is the same as that described in Figures 4a to 4g, and therefore is omitted.

According to one embodiment, electronic device 100 can use a second application driven in the background to obtain translation information 560 of image 540 based on information from image 540 while the first application is active. For example, referring to FIG5d, electronic device 100 can use the second application to obtain translation information 560 of image 540.

According to one embodiment, electronic device 100 can provide the acquired translation information. In this case, electronic device 100 can display the translation information of a region that changes over time in a configured manner. For example, referring to FIG5e, electronic device 100 can drive a second application in the foreground and display the translation information 560 of image 540 in the screen 580 of the second application, highlighting the region 520 that changes over time. However, the method by which electronic device 100 displays the translation information of a region that changes over time is not limited to the above.

Figures 6a to 6f are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, the electronic device 100 can display a portion of a scrollable page on the screen 600 of the first application. For example, referring to FIG6a, the electronic device 100 can display the upper portion of the search results page when searching for "vitamin D" on the screen 600 of the first application. Also, referring to FIG6b, by obtaining user input via a drop-down scroll, the electronic device 100 can display the lower portion of the search results page when searching for "vitamin D" on the screen 600 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in the set storage library or whether the image capture function has been activated.

According to one embodiment, the electronic device 100 can acquire an image obtained by scrolling a scrollable page. Here, scroll capture, as a method of acquiring an image of the page while moving the scroll, can be used to capture pages larger than the screen of the terminal.

For example, referring to Figure 6c, the electronic device 100 can acquire an image 620 obtained by scrolling and capturing the search results page when searching for "vitamin D". In this way, the electronic device 100 can acquire an image 620 larger than the screen 600 of the first application.

According to one embodiment, while the first application is active, the electronic device 100 can use a second application driven in the background to obtain translation information 640 of the image 620 based on the information of the image 620. For example, referring to FIG6d, the electronic device 100 can use the second application to obtain translation information 640 of the image 620.

According to one embodiment, electronic device 100 can provide the acquired translation information. At this time, electronic device 100 can provide the translation information corresponding to the area of the page displayed on the screen 600 of the first application to the screen 660 of the second application as the page is scrolled.

For example, referring to Figure 6e, the electronic device 100 can display translation information on the screen 660 of the second application that corresponds to the upper part of the search results page when searching for "vitamin D" displayed on the screen 600 of the first application. Also, referring to Figure 6f, by obtaining user input from the scroll included in the drop-down menu of the screen 600 of the first application, the electronic device 100 can display translation information on the screen 660 of the second application that corresponds to the lower part of the search results page when searching for "vitamin D" displayed on the screen 600 of the first application.

At this time, the electronic device 100 highlights the bar icon 680 within the translated information for user convenience. However, the elements displayed within the translated information and the method used to display these elements can be set differently and can be changed according to user input, etc.

Figures 7a to 7e are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, electronic device 100 can execute a first application and display the screen 700 of the first application. For example, referring to FIG7a, electronic device 100 can display the homepage of the first application in the screen 700 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, electronic device 100 can confirm whether the page corresponding to the acquired image has changed due to an update of the first application. For example, referring to Figure 7b, after confirming that information corresponding to the image of the first application's homepage has been acquired, electronic device 100 can confirm whether the homepage has changed due to an update of the first application. With the update of the first application, electronic device 100 can confirm that the text recorded in area 720 of the homepage changes from "Mid-Autumn Festival up to 70%" to "New members get a half-price coupon" and at the same time add a coupon icon.

According to one embodiment, as the page corresponding to the confirmed and acquired image changes, the electronic device 100 can confirm whether to translate the image corresponding to the changed page. For example, referring to FIG7c, the electronic device 100 can display a user interface 740 that informs that a part of the homepage has changed due to an update of the first application and asks whether to translate the image accordingly.

According to one embodiment, electronic device 100 can obtain an image 760 corresponding to the changed page. For example, referring to FIG7d, when electronic device 100 obtains user input in the user interface 740 where the icon is selected, electronic device 100 can obtain an image 760 captured from the changed homepage.

According to one embodiment, electronic device 100 can use a second application driven in the background to obtain translation information of image 760 based on information of image 760 while the first application is active. For example, electronic device 100 can use the second application to obtain translation information of image 760.

According to one embodiment, electronic device 100 can provide the acquired translation information. In this case, electronic device 100 can display the translation information of areas that have changed due to the update of the first application in a configured manner. For example, referring to Figure 7e, electronic device 100 can drive a second application in the foreground and display the translation information of image 760 on the screen 780 of the second application, highlighting the area 720 that has changed due to the update of the first application.

Figures 8a and 8b are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, electronic device 100 can execute a first application and display the screen 800 of the first application. For example, referring to FIG8a, electronic device 100 can display the homepage of the first application on the screen 800 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, upon confirming that the image has been acquired, the electronic device 100 can determine whether to translate the image. For example, the electronic device 100 can determine whether to translate the image by providing a user interface that asks whether to translate the image, or by providing an input indicating whether a request to capture an image and translate the captured image has been received.

According to one embodiment, electronic device 100 can obtain image translation information based on image information using a second application driven in the background while the first application is active. For example, electronic device 100 can obtain image translation information by directly translating the text in the first language included in the image into the second language; or by sending an image translation request to an external server and then receiving the image translation information from the external server.

According to one embodiment, electronic device 100 can provide the acquired translation information along with the content of the first application. For example, referring to Figure 8b, electronic device 100 can display the homepage on the screen 800 of the first application and display the text translated into the second language on top of the corresponding text in the first language. At this time, electronic device 100 can adjust the transparency of the text translated into the second language so that the user can confirm the content of the text in the first language corresponding to the text translated into the second language.

Figures 9a to 9c are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that overlaps with Figures 2a to 2g is simplified or omitted.

According to one embodiment, electronic device 100 can execute a first application and display the screen 900 of the first application. For example, referring to FIG9a, electronic device 100 can display the homepage of the first application on the screen 900 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, upon confirming that the image has been acquired, the electronic device 100 can determine whether to translate the image. For example, the electronic device 100 can determine whether to translate the image by providing a user interface that asks whether to translate the image, or by providing an input indicating whether a request to capture an image has been received and the captured image has been translated.

According to one embodiment, electronic device 100 can obtain image translation information based on image information using a second application driven in the background while the first application is active. For example, electronic device 100 can obtain image translation information by directly translating the text in the first language included in the image into the second language; or by sending an image translation request to an external server and then receiving the image translation information from the external server.

According to one embodiment, electronic device 100 may provide the acquired translation information together with the content of the first application. For example, referring to FIG9b, electronic device 100 may display a homepage on the screen 900 of the first application and display a bubble icon 920 next to the text in the first language. However, the appearance of the icon displayed next to the text in the first language is only an embodiment and is not limited to the above content.

According to one embodiment, when user input is received indicating that a bubble icon 920 has been selected, the electronic device 100 can provide translation information corresponding to the selected bubble icon 920. For example, referring to FIG9c, the electronic device 100 can display a bubble icon 940 including text in a second language next to text in a first language, the second language text being a translation of the first language text corresponding to the selected bubble icon 920. However, the appearance of the icon displayed next to the text in the first language is only an embodiment and is not limited to the above.

Figures 10a to 10d are diagrams illustrating the process by which an electronic device 100 provides translated information according to an embodiment. Content that is repeated in Figures 2a to 2g is simplified or omitted.

According to one embodiment, electronic device 100 can execute a first application and display the screen 1000 of the first application. For example, referring to FIG10a, electronic device 100 can display a portion of the search results page when searching for "vitamin D" in the screen 1000 of the first application.

According to one embodiment, the electronic device 100 can confirm whether image information has been acquired. For example, the electronic device 100 can confirm whether image information has been acquired based on whether a new image file has been generated in a set storage library or whether the image capture function has been activated.

According to one embodiment, upon confirming that the image has been acquired, the electronic device 100 can determine whether to translate the image. For example, the electronic device 100 can determine whether to translate the image by providing a user interface that asks whether to translate the image, or by providing an input indicating whether a request to capture an image and translate the captured image has been received.

According to one embodiment, electronic device 100 can obtain image translation information based on image information using a second application driven in the background while the first application is active. For example, electronic device 100 can obtain image translation information by directly translating the text in the first language included in the image 210 into the second language; or by sending an image translation request to an external server and then receiving the image translation information from the external server.

According to one embodiment, electronic device 100 can provide the acquired translation information. For example, referring to Figure 10b, electronic device 100 can drive a second application in the foreground and display the translation information of the image on the screen 1020 of the second application.

According to one embodiment, electronic device 100 can obtain input for settings in the translated information for a first area. For example, referring to Figure 10b, electronic device 100 can obtain user input for selecting area 1040, which displays a specific product in the translated information displayed on screen 1020 of the second application.

According to one embodiment, electronic device 100 can confirm a second area within a first application corresponding to the first area based on a set input. For example, referring to FIG10c, electronic device 100 can confirm the coordinate value of the selected area 1040 in the screen 1020 of the second application, and confirm the area 1060 having the same coordinate value as the area 1040 in the screen 1000 of the first application.

According to one embodiment, electronic device 100 can provide a page corresponding to the second area. For example, referring to FIG10d, electronic device 100 can drive a first application in the foreground and display a product details page related to a specific product displayed in area 1060 in the screen 1000 of the first application.

Figure 11 is a flowchart illustrating a method for providing translated information using an electronic device according to an embodiment. The above description can be applied to repetitive content.

In step S1100, the electronic device can confirm whether it has obtained information about the image corresponding to the first application.

According to one embodiment, when an electronic device confirms whether it has obtained information about the image corresponding to the first application, it can confirm whether a new image file has been generated in the set storage library.

According to one embodiment, when the electronic device confirms whether it has obtained information about the image corresponding to the first application, it can confirm whether the image capture function has been activated.

In step S1120, when the electronic device acquires image information, it can acquire image translation information.

According to one embodiment, when an electronic device obtains translation information of an image, it can use a second application driven in the background to obtain translation information based on the image information while the first application is active.

According to one embodiment, translated information may include a second text in a second language obtained based on a first text in a first language included in the image.

According to one embodiment, when an electronic device obtains translation information for an image, it can provide a user interface (UI) that asks whether to translate the image.

According to one embodiment, when an electronic device obtains translation information of an image, it can provide a user interface that allows users to query the translation language of the image.

According to one embodiment, when an electronic device acquires translation information for an image, it can confirm whether it has acquired a Type 1 input requesting the capture and translation of the captured image. In this case, the Type 1 input may be different from the Type 2 input requesting the capture of the image.

In step S1140, the electronic device can provide the acquired translation information.

According to one embodiment, the electronic device can obtain a translation request for an object region within an image. Thereby, the translation information may include translation information for the object region.

According to one embodiment, the electronic device can identify areas that change over time within an image. The electronic device can then determine whether to translate the time-varying areas and obtain a translation request for the time-varying areas. At this time, the translation information may include translation information for the time-varying areas.

According to one embodiment, the electronic device can identify areas that vary in time within an image. At this point, translation information for the time-varying areas can be displayed in a configured manner.

According to one embodiment, the image may include an image captured by scrolling a scrollable page. In this way, when providing translation information, the electronic device can provide translation information along with the page, corresponding to areas within the page that are displayed on the screen as the page scrolls.

According to one embodiment, the image includes an image captured from a page of a first application. Herein, the electronic device can confirm whether the page has changed due to an update of the first application, and upon confirming the page change, confirm whether to translate the image corresponding to the changed page. Upon confirming the translation of the image corresponding to the changed page, the electronic device can obtain information about the image corresponding to the changed page. Subsequently, the electronic device can obtain translation information about the image corresponding to the changed page and provide the translation information.

According to one embodiment, the obtained translation information can be displayed on the content of the first application.

According to one embodiment, the electronic device can obtain input for settings related to the first area within the translated information, and based on the input for settings, confirm the second area within the first application corresponding to the first area. Thereafter, the electronic device can provide a page corresponding to the second area.

According to one embodiment, when the image information and the translated information are of different sizes, the settings elements in the translated information can be displayed in a set manner.

Figure 12 is a block diagram showing an embodiment of an electronic device 100.

According to one embodiment, electronic device 100 may include a communication device 1220, memory 1240, and a controller 1260. The electronic device 100 shown in FIG12 only illustrates the components relevant to this embodiment. Therefore, those skilled in the art related to this embodiment will understand that, in addition to the components shown in FIG12, other common components may be included. Electronic device 100 may include the aforementioned server content, omitting repeated details. In this embodiment, the communication device may include one or more transceivers. Furthermore, in this embodiment, the controller may include one or more processors.

The Communication Unit 1220, as a device for performing wired/wireless communication, can communicate with external electronic devices. These external electronic devices can be terminals or servers. Furthermore, the communication technologies utilized by the Communication Unit 1220 may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G (5th Generation Mobile Communication Technology), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), Wi-Fi, and NFC (Near Field Communication).

The control unit 1260 can control the overall operation of the electronic device 100 and process data and signals. The control unit 1260 may include at least one hardware unit. Furthermore, the control unit 1260 can perform operations by executing one or more software modules generated from program code stored in memory 1240. The control unit 1260 may include a processor and memory; the processor can execute program code stored in memory to control the overall operation of the electronic device 100 and process data and signals.

The control unit 1260 can confirm whether the information of the image corresponding to the first application has been obtained. When the information of the image has been obtained, the translation information of the image is obtained and the obtained translation information is provided.

The electronic device described in the above embodiments may include a processor, memory for storing and executing program data, such as permanent storage on a disk drive, a communication port for communicating with external devices, such as a touch panel, keys, or buttons, and a user interface device. The computer-readable code or program commands implemented using software modules or algorithms, which can be executed on the aforementioned processor, can be stored on a computer-readable recording medium. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, Digital Versatile Disc). Computer-readable recording media are distributed across computer systems connected to a network, allowing for the distributed storage and execution of computer-readable code. The media can be read by the computer, stored in memory, and executed in the processor.

This embodiment can be composed of functional blocks and various processing steps. These functional blocks can be implemented by different numbers of hardware and/or software components that perform specific functions. For example, the embodiment can employ integrated circuits capable of performing various functions via the control of one or more microprocessors or other control devices, such as memory, processing, logic, lookup tables, etc. The components can be executed by software programming or software elements. Similarly, this embodiment includes algorithms implemented by combinations of data structures, programs, norms, or other programming structures, and therefore can be implemented using programming or scripting languages such as C, C++, Java, and assemblers. In terms of functionality, this can be achieved by algorithms executed in one or more processors. Furthermore, this embodiment can utilize prior art for electronic environment setup, signal processing, and/or data processing. Terms such as "mechanism," "component," "method," and "structure" are used broadly and are not limited to mechanical and physical structures. The aforementioned terms can be associated with processors and include the meaning of a series of software processes.

The above embodiments are merely examples, and other embodiments may be implemented within the scope of the invention application described below.

100: Electronic Device 120: Server 1 140: Server 2 160: Network 200: Screen 210: Video 220: User Interface 230: User Interface 240: Translated Information 250: Screen 260: Icon 270: User Interface 300: Video 320: Object Area 340: Translated Information 400: Screen 410: Area 420: Video 430: Video 440: User Interface 450: Translated Information 460: Translated Information 500: Screen 520: Area 540: Video 560: Translated Information 580: Screen 600: Screen 620: Video 640: Translated Information 660: Screen 680: Icon 700: Screen 720: Area 740: User Interface 760: Image 780: Screen 800: Screen 900: Screen 920: Icon 940: Icon 1000: Screen 1020: Screen 1040: Area 1060: Area 1220: Communication Unit 1240: Memory 1260: Control Unit S1100: Step S1120: Step S1140: Step

Figure 1 illustrates a system according to an embodiment. Figures 2a to 2g are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 3a to 3b are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 4a to 4g are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 5a to 5e are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 6a to 6f are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 7a to 7e are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 8a and 8b are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 9a and 9c are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figures 10a and 10d are diagrams illustrating the process of an electronic device providing translated information according to an embodiment. Figure 11 is a flowchart illustrating the method of providing translated information by an electronic device according to an embodiment. Figure 12 is a block diagram illustrating an electronic device according to an embodiment.

S1100: Steps

S1120: Steps

S1140: Steps

Claims (17)

A method for providing translated information, performed by an electronic device, includes the following steps: confirming whether information about an image corresponding to a first application has been obtained; if the information about the image has been obtained, obtaining translation information about the image; and providing the obtained translation information, wherein the method further includes identifying areas that change over time within the image, wherein the translation information about the areas that change over time is displayed in a highlighted manner. The method for providing translation information as requested in Item 1 includes the following steps for obtaining the translation information: While the first application is active, a second application driven in the background is used to obtain the translation information based on the information of the image. The method for providing translation information as described in claim 1, wherein the translation information includes a second text in a second language obtained based on a first text in a first language included in the aforementioned image. The method for providing translation information as described in Request 1 includes the following steps for confirming whether the information on the aforementioned images has been obtained: Confirming whether a new image file has been generated in the configured storage library. As in the translation information provision method of Request 1, the step of confirming whether the information of the above image has been obtained includes the following steps: confirming whether the image capture function has been activated. As in the method for providing translation information in Request 1, the steps for obtaining translation information of the above image include the following steps: providing a user interface that asks whether the above image should be translated. As in the method for providing translation information in Request 1, the steps for obtaining the translation information of the above image include the following steps: providing a user interface that queries the translation language of the above image. As in the method for providing translation information for request item 1, the steps for obtaining the translation information of the aforementioned image include the following steps: confirming whether the Type 1 input for requesting to capture an image and translate the captured image has been obtained; and that the aforementioned Type 1 input is different from the Type 2 input for requesting to capture an image. The method for providing translation information as described in claim 1 further includes the following steps: obtaining a translation request for the object area within the aforementioned image; and the aforementioned translation information includes the translation information for the aforementioned object area. The method for providing translation information as described in claim 1 further includes the following steps: confirming whether a translation is required for the area that changes according to the aforementioned time; and obtaining a translation request for the area that changes according to the aforementioned time; and the aforementioned translation information includes translation information for the area that changes according to the aforementioned time. As in the method for providing translation information in claim 1, where the aforementioned image includes an image obtained by scrolling and capturing a scrollable page, the steps for providing translation information for the aforementioned image include the following steps: providing translation information corresponding to the area within the aforementioned page that is displayed on the screen as the aforementioned page is scrolled, together with the aforementioned page. As in the method for providing translation information in claim 1, wherein the aforementioned image includes an image obtained by capturing a page from the aforementioned first application, the method for providing translation information includes the following steps: confirming whether the aforementioned page has changed due to an update of the aforementioned first application; upon confirming that the aforementioned page has changed, confirming whether to translate the image corresponding to the changed aforementioned page; upon confirming that the image corresponding to the changed aforementioned page has been translated, obtaining information about the image corresponding to the changed aforementioned page; obtaining translation information about the image corresponding to the changed aforementioned page; and providing translation information about the image corresponding to the changed aforementioned page. The translation information provided in Request 1 is displayed on the content of the first application. The method for providing translation information as described in claim 1 further includes the following steps: obtaining input for settings related to the first area within the aforementioned translation information; confirming the second area within the aforementioned first application corresponding to the aforementioned first area based on the input for the aforementioned settings; and providing a page corresponding to the aforementioned second area. As in the translation information provision method of Request 1, when the size of the information of the aforementioned image differs from that of the aforementioned translation information, the setting elements within the aforementioned translation information are displayed using a setting method. A non-transitory computer-readable recording medium that records programs for executing methods such as those described in Request 1 on a computer. An electronic device includes: a communication unit; a memory; and a control unit; wherein the control unit is configured to: confirm whether information corresponding to an image for a first application has been obtained; when the information of the image has been obtained, obtain translation information of the image; and provide the obtained translation information, wherein the control unit further confirms areas that change according to time within the image, and displays the translation information of the areas that change according to time in a highlighted manner.