JP2015069234A - Display processing apparatus, and control method thereof and control program - Google Patents

Abstract

To improve the readability of translated characters. A display processing method for displaying a character in which a character included in a photographed image is translated. The character color and the background color of the character included in the photographed image are specified (S15), and the character color and / or the background color is adjusted based on the difference between the specified character color and the background color (S16) and translated. A translation image in which the character and background are the adjusted character color and background color is created and displayed (S17 and S18). [Selection] Figure 1

Description

  The present invention relates to a display processing device that displays a translated character on a display unit, and a control method and a control program thereof for a character in which a character included in a photographed image photographed by a photographing unit is translated by a translation unit. About.

  Recently, portable electronic devices such as mobile phones, portable game machines, and smartphones are generally equipped with a camera and have a photographing function. For this reason, development of application software (hereinafter abbreviated as “application”) using the photographing function is underway.

  For example, Patent Document 1 discloses an information display device that can translate a foreign language or pictograph included in a document, a signboard, or the like in real time and display it at an appropriate display position. The information display device discriminates and extracts a character string included in an object such as a document imaged by a camera, a signboard, etc., translates the extracted character string using a dictionary, and uses the translated character string as a pre-translation character string. The character string is replaced or displayed superimposed on the character string before translation. In addition, “Word Lens Translator”, an application for smartphones, has realized the same function.

JP 2011-134144 A (released on July 07, 2011)

  In Patent Document 1, the pre-translation character string is covered with a color around the pre-translation character string, and the post-translation character string is displayed as the position and size of the pre-translation character string. , Angle, and color. That is, the character string before translation and the surrounding color are the same as the character string after translation and the surrounding color. In this case, the translated character string can be displayed without breaking the design intent of the subject such as the document or the signboard. However, in Patent Document 1, since the readability (readability) of the translated character string is not considered, the translated character string may be difficult to read.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a display processing apparatus and the like that can improve the readability of translated characters.

  A display processing apparatus according to an aspect of the present invention is a display processing apparatus that displays a translated character on a display unit with respect to a character obtained by translating a character included in a photographed image photographed by a photographing unit. In order to solve the above problem, a color specifying unit that specifies a character color that is a character color included in the photographed image and a background color that is a background color of the character, and the color specifying unit specifies A color adjusting means for adjusting at least one of the character color and the background color based on the difference between the character color and the background color, and the character color and the background adjusted by the color adjusting means for the translated character and its background. And display instruction means for instructing the display unit to display each color.

  According to one embodiment of the present invention, the readability of translated characters can be improved.

It is a flowchart which shows the flow of the process in the portable terminal which concerns on one Embodiment of this invention. It is a block diagram which shows schematic structure of the said portable terminal. It is a figure which shows the example of a display of a picked-up image. It is a flowchart which shows the flow of the process in the portable terminal which concerns on other embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail. For convenience of explanation, members having the same functions as those shown in the embodiments are denoted by the same reference numerals, and description thereof is omitted as appropriate.

Embodiment 1
First, an embodiment of the present invention will be described with reference to FIGS. FIG. 2 shows a schematic configuration of the mobile terminal according to the present embodiment. Examples of the mobile terminal include a mobile phone, a smartphone, a tablet terminal, a PDA (Personal Digital Assistant), a notebook PC (Personal Computer), and a PHS (Personal Handyphone System) terminal. Note that the present invention is not limited to a portable terminal, and can be applied to any electronic device having a photographing unit and a display unit, such as a portable game machine and a digital camera.

(Configuration of mobile terminal 1)
As shown in FIG. 2, the mobile terminal 1 according to the present embodiment includes a control unit (display processing device) 10, a storage unit 11, a photographing unit 12, an operation unit 13, and a display unit 14.

  The control unit 10 comprehensively controls various configurations in the mobile terminal 1 and includes a processor such as a CPU (Central Processing Unit). The storage unit 11 stores various data and programs, and includes a storage element such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. Details of the control unit 10 and the storage unit 11 will be described later.

  The photographing unit 12 shoots a subject and generates a photographed image such as a still image and a moving image. For example, an optical system such as a lens and externally converts light from the optical system into an electrical signal. An image pickup device. Examples of the image pickup device include a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) image sensor, and the like. The imaging unit 12 transmits the generated captured image data to the control unit 10.

  The operation unit 13 creates operation data and transmits it to the control unit 10 when a user operates an input device such as a button switch provided on the surface of the mobile terminal 1. Examples of input devices include a touch panel in addition to button switches.

  The display unit 14 receives display data from the control unit 10 and performs display on the display screen based on the received display data. Specifically, the display unit 14 includes a display element such as an LCD (Liquid Crystal Display) or an EL (Electroluminescence) display, and a driver circuit that drives the display element based on received display data.

  In the mobile terminal 1 of the present embodiment, the control unit 10 translates characters included in the photographed image taken by the photographing unit 12 and overwrites the pre-translation characters with the translated characters. The display unit 14 is instructed to display. Further, the control unit 10 specifies a character color that is a character color included in the photographed image and a background color that is a background color of the character, and based on the difference between the specified character color and the background color, the character At least one of the color and the background color is adjusted, and the display unit 14 is instructed to display the translated character and its background with the adjusted character color and background color, respectively. In general, since the readability of characters depends on the difference between the character color and the background color, the readability of the translated characters can be improved according to this embodiment.

  This will be described with a specific example. FIG. 3A shows a display example in which the captured image captured by the imaging unit is displayed as it is on the display unit. Also, (b) and (c) in the figure translate a part of a plurality of characters (two lines in the illustrated example) included in the photographed image into the photographed image, and translate the characters before translation. The example of a display displayed on the said display part after overwriting with a later character is shown. Among these, (b) of the figure shows a display example by the portable terminal 1 of the present embodiment as an example, while (c) of the figure shows a display example by a conventional portable terminal as a reference example. .

  In the conventional example, as shown in FIG. 3C, the character area including the character to be translated is painted with the background color of the character area, and the character after translation with the character color of the character area is displayed. Superimposed. In contrast, in this embodiment, as shown in FIG. 3B, the lightness of the background color is adjusted to be low based on the lightness difference between the character color and the background color in the character region. The character region is filled with the background color, and the translated character is superimposed with the character color of the character region. Comparing (b) and (c) in FIG. 3, the present embodiment has a greater lightness difference between the character color and the background color in the character region than the conventional example, and the readability of the translated character is improved. I can understand that.

(Details of control unit 10 and storage unit 11)
Next, details of the control unit 10 and the storage unit 11 will be described. As shown in FIG. 2, the control unit 10 includes a captured image acquisition unit 20, a character extraction unit 21, a character recognition unit 22, a translation unit 23, a color identification unit (color identification unit) 24, and a color adjustment unit (color adjustment unit). 25, a translation image creation unit (display instruction unit) 26, and a superimposition unit (display instruction unit) 27. In addition, the storage unit 11 stores a color adjustment table 30 that includes a combination of a character color (character color) and a background color (background color) of the character and a combination that improves the readability of the character. ing.

  The captured image acquisition unit 20 acquires captured image data captured by the imaging unit 12. The captured image acquisition unit 20 sends the acquired captured image data to the character extraction unit 21 and the superimposition unit 27.

  The character extraction unit 21 uses the captured image data from the captured image acquisition unit 20 to identify a character region that includes one or more characters in the captured image, and extracts the character from the identified character region. To extract. The character extraction unit 21 sends the data of the character region in the photographed image to the color specifying unit 24 and the translation image creating unit 26, and also includes character feature data indicating the extracted character features in the character recognition unit 22 and the color specifying unit. 24.

  The character recognition unit 22 recognizes the extracted character based on the character feature data from the character extraction unit 21. The character recognition unit 22 sends recognized character data indicating the recognized character to the translation unit 23. Note that the functions of the character extracting unit 21 and the character recognizing unit 22 can be realized by using a known technique such as a template matching method, and thus description thereof is omitted.

  The translation unit 23 performs machine translation of the recognized character based on the recognized character data from the character recognition unit 22. The translation unit 23 sends the translated character data indicating the translated characters to the translated image creation unit 26. In addition, about machine translation, since well-known techniques, such as rule-based machine translation and statistical machine translation, can be utilized, the description is abbreviate | omitted.

  The color specifying unit 24 specifies the character color and background color of the extracted character based on the character area data and the character feature data from the character extracting unit 21. The color specifying unit 24 sends the specified character color and background color data to the color adjusting unit 25.

  When the color adjustment unit 25 acquires the character color and background color data from the color specifying unit 24, the color adjustment unit 25, based on the difference between the character color and the background color, improves the character color and / or the background color. Alternatively, the background color is adjusted. The color adjustment unit 25 sends the adjusted character color and background color data to the translated image creation unit 26.

  Specifically, the color adjustment unit 25 refers to the color adjustment table 30 in the storage unit 11 and searches for the combination including the background color that is the same as or close to (similar to) the background color specified by the color specifying unit 24. To do. Note that the color adjustment unit 25 may search for the combination including a character color that is the same as or close to the character color specified by the color specifying unit 24. Then, the color adjusting unit 25 sends the character color and background color data in the searched combination, that is, the combination with good readability, to the translated image creating unit 26 as the adjusted character color and background color data. .

  The translation image creation unit 26 uses the character region data from the character extraction unit 21, the translation character data from the translation unit 23, and the adjusted character color and background color data from the color adjustment unit 25. A translated image, which is an image including the translated character in the character region in the adjusted character color and background color, is created. Specifically, the translated image creation unit 26 creates the translated image by filling the character area with the adjusted background color and then overwriting the translated character with the adjusted character color. . The translation image creation unit 26 sends the created translation image data to the superposition unit 27.

  The superimposing unit 27 superimposes the translated image on the character region of the captured image using the captured image data from the captured image acquisition unit 20 and the translated image data from the translated image creating unit 26. is there. The superimposing unit 27 transmits the superimposed image data to the display unit 14 so that the superimposed image is displayed on the display unit 14.

(Processing flow of the control unit 10)
FIG. 1 shows a processing flow of the control unit 10 in the portable terminal 1 having the above-described configuration. As shown in the figure, first, the photographing unit 12 is activated (S10), and the photographed image acquisition unit 20 obtains a photographed image created by the photographing unit 12 (S11). Next, the character extraction unit 21 specifies the character region in the captured image acquired by the captured image acquisition unit 20, and extracts characters from the specified character region (S12).

  Next, the character recognition unit 22 recognizes the character extracted by the character extraction unit 21 (S13), and the translation unit 23 machine translates the character recognized by the character recognition unit 22 (S14). The color specifying unit 24 specifies the character color and background color of the character extracted by the character extracting unit 21 (S15, color specifying step), and the color adjusting unit 25 sets the color so that the readability of the character is good. The character color and / or the background color is adjusted based on the difference between the character color and the background color specified by the specifying unit 24 (S16, color adjustment step). In addition, step S13 * S14 and step S15 * S16 may be performed by changing order, and may be performed in parallel.

  Next, the translation image creation unit 26 creates a translation image that includes the character translated by the translation unit 23 in the character region specified by the character extraction unit 21 with the character color and background color adjusted by the color adjustment unit 25. (S17, display instruction step). Then, the superimposing unit 27 superimposes the translated image created by the translated image creating unit 26 on the character area of the photographed image from the photographed image acquiring unit 20, and displays the superimposed image via the display unit 14 (S18). , Display instruction step). Then, it returns to step S11 and repeats the said process.

(Modification)
In the present embodiment, the character extraction unit 21 and the character recognition unit 22 are separate functional blocks. However, since character extraction and recognition can be performed by a series of OCR (Optical Character Recognition) processing, 1 It may be combined into one functional block. In this embodiment, the character before translation is overwritten with the character after translation, but the character after translation may be displayed in another display area. Furthermore, in the present embodiment, the character recognition unit 22 and the translation unit 23 are included in the control unit 10, but may be devices different from the control unit 10.

[Embodiment 2]
Next, another embodiment of the present invention will be described with reference to FIG. The mobile terminal 1 of this embodiment is different from the mobile terminal 1 shown in FIGS. 1 to 3 in that the color adjustment table 30 is omitted and the detailed function of the color adjustment unit 25 is different. The configuration is the same. In the present embodiment, the color adjustment unit 25 compares the character color and / or the background color with the character color and / or the background color so that the difference between the character color and the background color is close to a predetermined condition that improves the readability as compared with the color adjustment unit 25 illustrated in FIG. The other functions are the same except that the background color is adjusted.

  FIG. 4 shows the flow of processing in the color adjustment unit 25 of this embodiment, and is a subroutine of step S16 in FIG. In general, the character color and the background color are easier to read when the brightness difference is larger, and are easier to read when the saturation difference is larger. In general, the character color and the background color are difficult to read when the eyes are flickering when they are in a complementary color relationship or opposite color relationship, and are easy to read when they are similar colors.

  Therefore, in the present embodiment, as shown in FIG. 4, the color adjustment unit 25 first increases the brightness difference by a predetermined value when the brightness difference between the character color and the background color is smaller than a predetermined value. Then, the brightness of the character color and / or the background color is adjusted (S20 / S21). Next, when the saturation difference between the character color and the background color is smaller than a predetermined value, the saturation of the character color and / or the background color is adjusted so that the saturation difference is increased by a predetermined value. (S22 / S23). When the hue difference between the character color and the background color is larger than a predetermined value, the hue of the character color and / or the background color is adjusted so that the hue difference is reduced by a predetermined value (S24 / S25). ). Thereafter, the process returns to the original routine shown in FIG. 1 and proceeds to step S17.

(Modification)
In the present embodiment, a color space using three color attributes such as brightness, saturation, and hue is used, but other color spaces may be used. In this embodiment, the adjustment is performed based on the conditions of all three components in the color space. However, the adjustment may be performed based on the conditions of one component or two components in the color space. In the present embodiment, the color adjustment unit 25 performs the adjustment so that the difference between the text color and the background color approaches the predetermined condition, but the difference satisfies the predetermined condition. The above adjustment may be performed.

  For example, consider the case of a YUV color space processing system in which each component of YUV takes a value between 0 and 255. In this processing system, the adjustment is performed by paying attention to the Y (luminance) value of YUV. Specifically, when the difference between the Y values of the character color and the background color is less than 128, the correction is made to be 128. That is, if 128> (Y value of background color) − (Y value of character color) ≧ 0, the character color is set so as to satisfy (Y value of character color) = (Y value of background color) −128. And / or adjust the Y value of the background value. On the other hand, if 128> (Y value of the character color) − (Y value of the background color)> 0, the character color and the character color are set so as to satisfy (Y value of the character color) = (Y value of the background color) +128. / Or adjust the Y value of the background value.

  Moreover, you may adjust according to the characteristic of the image pick-up element of the imaging | photography part 12. FIG. For example, when an image sensor with low blue sensitivity is used and the character color or the background color is close to blue, the saturation of blue is increased for the corresponding character color or the background color. You may adjust.

[Embodiment 3]
The control block (especially the control unit 10) of the portable terminal 1 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or software using a CPU (Central Processing Unit). It may be realized by.

  In the latter case, the mobile terminal 1 includes a CPU that executes instructions of a program that is software that implements each function, a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by a computer (or CPU), or A storage device (these are referred to as “recording media”), a RAM (Random Access Memory) that expands the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Summary]
The display processing apparatus (control part 10) which concerns on aspect 1 of this invention is using the translated character as a display part about the character in which the character contained in the picked-up image image | photographed by the imaging | photography part was translated by the translation part. A display processing device for displaying, a color specifying means (color specifying unit 24) for specifying a character color which is a character color included in the photographed image and a background color which is a background color of the character; Based on the difference between the character color and the background color specified by the specifying means, the color adjusting means (color adjusting unit 25) for adjusting at least one of the character color and the background color, the translated character and the background thereof, The display unit includes a display instruction unit (translation image creation unit 26 / superimposition unit 27) that instructs the display unit to display the character color and the background color adjusted by the color adjustment unit.

  According to the above configuration, the character color (character color) and the background color (background color) included in the photographed image are specified, and the character color and the background are determined based on the difference between the specified character color and background color. At least one of the colors is adjusted, and the translated character and its background are displayed with the adjusted character color and background color, respectively. In general, since the readability of characters depends on the difference between the character color and the background color, the readability of the translated characters can be improved by the present invention.

  In order to improve the readability, the color adjusting means adjusts the difference in brightness to be greater than or equal to a predetermined value and adjustment to increase or approximate the difference in saturation for the character color and the background color. And at least one of adjustments that make the difference in hue within a predetermined value or close to it.

  Alternatively, the color adjusting unit has a background color that is the same as or similar to the background color specified by the color specifying unit from a storage unit that stores a plurality of combinations of the character color and the background color with good readability. The combination may be read, and the display instruction unit may instruct the display unit to display each of the translated character and its background with the combination of the character color and the background color read by the color adjustment unit. .

  The control method of the display processing device according to the aspect 2 of the present invention causes the display unit to display the translated characters of the characters included in the photographed image photographed by the photographing unit and translated by the translation unit. A method for controlling a display processing apparatus, comprising: a color specifying step (S15) for specifying a character color that is a character color included in the photographed image and a background color that is a background color of the character; and the color specifying step A color adjustment step (S16) for adjusting at least one of the character color and the background color based on the difference between the character color and the background color specified in step S16, and the color adjustment step for adjusting the translated character and its background. And a display instruction step (S17 / S18) for instructing the display unit to display the character color and the background color adjusted in step S1, respectively. In this case, the same effects as those of the first aspect are obtained.

  The display processing apparatus according to each aspect of the present invention may be realized by a computer. In this case, the display processing apparatus is realized by a computer by causing the computer to operate as each unit included in the display processing apparatus. A control program for the display processing apparatus and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

  According to the present invention, the character readability can be improved by adjusting the character color and / or the background color based on the difference between the character color and the background color included in the photographed image. It can be applied to any electronic device.

1 mobile terminal 10 control unit (display processing device)
DESCRIPTION OF SYMBOLS 11 Memory | storage part 12 Image | photographing part 13 Operation part 14 Display part 20 Captured image acquisition part 21 Character extraction part 22 Character recognition part 23 Translation part 24 Color identification part (color identification means)
25 Color adjustment part (Color adjustment means)
26 Translated image creation unit (display instruction means)
27 Superimposition unit (display instruction means)
30 color adjustment table

Claims (5)

A display processing device that displays a translated character on a display unit with respect to a character that has been translated by a translation unit in a photographed image photographed by a photographing unit,
Color specifying means for specifying a character color that is a color of a character included in the photographed image and a background color that is a background color of the character;
Color adjusting means for adjusting at least one of the character color and the background color based on the difference between the character color and the background color specified by the color specifying means;
A display processing apparatus comprising: a display instruction unit that instructs the display unit to display the translated character and its background in a character color and a background color adjusted by the color adjustment unit, respectively.   The color adjusting means adjusts the difference in brightness to be greater than or equal to a predetermined value for the character color and the background color, adjustment to increase or approach the difference in saturation is greater than or equal to a predetermined value, and sets the difference in hue to a predetermined value The display processing apparatus according to claim 1, wherein at least one of adjustments within or close to is performed. The color adjusting unit is configured to store the combination having a background color that is the same as or similar to the background color specified by the color specifying unit from a storage unit that stores a plurality of combinations of the character color and the background color with good readability. reading,
The display instructing unit instructs the display unit to display the translated character and its background by a combination of the character color and the background color read by the color adjusting unit, respectively. The display processing apparatus according to 1.   A control program for causing a computer to function as the display processing device according to claim 1, wherein the control program causes the computer to function as each of the means. A control method for a display processing device for displaying a translated character on a display unit with respect to a character in which a character included in a photographed image photographed by a photographing unit is translated by a translation unit,
A color specifying step of specifying a character color that is a character color included in the photographed image and a background color that is a background color of the character;
A color adjustment step of adjusting at least one of the character color and the background color based on the difference between the character color and the background color specified in the color specifying step;
A display instruction step for instructing the display unit to display the translated character and its background in the character color and the background color adjusted in the color adjustment step, respectively. Control method.

Images