JP2016171379A - Document reader, display control method and display control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display a portion of a document by translating the portion of the document while sequentially displaying images of a plurality of sheets of the document.SOLUTION: A document reader includes: a document display control part 13 for displaying respective image data of a plurality of pages obtained by causing a document read part to continuously read a document of a plurality of pages and output the document in each first period; a position detection part 71 for detecting a position instructed by a user in image data of a displayed page; a character area determination part 15 for determining one or more character areas representing characters from the image data of the displayed page; a character recognition part 17 for performing character recognition of a part of the character area including the position detected by the position detection part 71 from the one or more character areas in the image data of the displayed page; a translation part 23 for translating a character string undergoing character recognition into a prescribed language; and a relation display control part 27 for displaying the character string of the translated language in association with the image data of the displayed page.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a document reading device, a display control method, and a display control program, and more particularly to a document reading device having a translation function, a display control method and a display control program executed by the document reading device.

  A document reading apparatus represented by a multifunction peripheral (hereinafter referred to as “MFP”) includes an automatic document feeder, and a preview function for displaying an image obtained by reading the document in order to confirm the result of reading the document. There is known an image forming apparatus having For example, in Japanese Patent Application Laid-Open No. 2011-076669, as a document reading device for informing a user of document inclination or protrusion, a document reading unit that reads a document image, and processes the read document image to generate an output image An image processing unit for outputting, an output unit for outputting the output image to the outside, and a display unit for a user to preview the document image. There is described a document reading apparatus with a preview function that generates preview image data from a partial region corresponding to a character region and provides the preview image data to a display unit.

  On the other hand, a character recognition process for recognizing a character included in an image and converting it into character information, and a translation process for translating the character information into another language are known. For this reason, it is possible to translate characters included in an image obtained by reading a document.

However, since the time for the character recognition process and the translation process is longer than the time from reading the original with the original reading apparatus until displaying the image obtained by reading, the image displayed as a preview corresponding to the original is displayed. Instead, there is a problem that it is difficult to display the result of translating characters included in the image corresponding to the document. In particular, when a plurality of originals are continuously read, it takes a long time to display a translation result by performing character recognition processing and translation processing on a plurality of images respectively corresponding to the plurality of originals. It was.
JP 2011-076669 A

  The present invention has been made to solve the above-mentioned problems. One of the objects of the present invention is to display a plurality of original images in order while translating and displaying a part of the original. It is to provide a possible document reading device.

  Another object of the present invention is to provide a document reading apparatus capable of efficiently translating a character string in a document.

  Still another object of the present invention is to provide a display control method and a display control program capable of translating and displaying a part of a document while sequentially displaying images of a plurality of documents.

  Still another object of the present invention is to provide a display control method and a display control program capable of efficiently translating a character string in a document.

  In order to achieve the above-described object, according to one aspect of the present invention, a document reading device includes a document reading unit that reads a document and outputs image data, and the document reading unit continuously reads a plurality of pages of a document. The image display control means for displaying each of the plurality of pages of image data to be output on the display means for each first period in the order that the document reading means outputs, and while any one of the plurality of pages of image data is being displayed. Position detection means for detecting a position designated by the user in the image data of the displayed page, and characters are represented from the image data of the displayed page in response to the position being detected by the position detection means. Character area determination means for determining one or more character areas, and position detection means from among one or more character areas determined in the image data of the displayed page Displayed is a character recognition means for recognizing a part of a character area including the specified position, a translation means for translating a character string obtained by character recognition into a predetermined language, and a translated character string obtained by translation. Associated display control means for displaying on the display means in association with the image data of the page.

  According to this aspect, each of the plurality of pages of image data obtained by continuously reading a plurality of pages of the document is displayed for each first period, and displayed while any one of the plurality of pages of image data is displayed. In response to the detection of the position indicated by the user in the image data of the page, the character area part including the position detected in the image data is recognized, and the character string thus recognized is determined in a predetermined language. And the translation character string obtained by the translation is displayed in association with the image data. For this reason, the character area portion including the position designated by the user in the displayed image data is recognized and translated, so that the processing amount of character recognition and translation can be reduced, and the processing time is shortened. can do. As a result, it is possible to provide a document reading apparatus capable of displaying a plurality of document images in order while translating and displaying a part of the document.

  Preferably, the related display control means sets the translated character string at a predetermined relative position with respect to a position at which the image data of the displayed page is displayed, and with respect to a size at which the image data of the displayed page is displayed. Display in a relatively fixed size.

  According to this aspect, the translated character string is in a predetermined relative position with respect to the position where the image data of the displayed page is displayed, and relative to the size where the image data of the displayed page is displayed. Since the image is displayed in a predetermined size, it is possible to notify the user of the portion corresponding to the translated character string in the image data.

  According to another aspect of the present invention, a document reading device includes a document reading unit that reads a document and outputs image data, an image display control unit that displays image data output from the document reading unit on a display unit, and a document Character area determining means for determining a character area in which characters are represented from image data output by the reading means, character recognition means for recognizing the character area portion determined in the image data, and character recognition Among the character strings, target character string determining means for determining a character string portion in which the corresponding portion in the image data has a predetermined characteristic, and the character string included in the determined character string portion are translated into a predetermined language Translation means, and related display control means for displaying the translated character string obtained by translation on the display means in association with the image data.

  According to this aspect, a character region is determined from image data obtained by reading a document, a character region portion in the image data is recognized, and a corresponding portion in the image data in the recognized character string Determines a character string portion having predetermined characteristics, translates the character string portion of the character string portion, and displays a translated character string obtained by the translation in association with the image data. For this reason, since the characteristic part in the document is translated and displayed, the number of characters to be translated is reduced compared with the case of translating all the character strings in the document, so that the processing time can be shortened. it can. As a result, it is possible to provide a document reading apparatus that can efficiently translate character strings in a document.

  Preferably, the image processing apparatus further includes a feature region determination unit that determines a feature region that is extracted from the image data but has a second feature that is different from the first feature of the character region portion. Character recognition is performed on the portion of the character area including the determined feature area in the data.

  According to this aspect, since the feature region having a feature different from the feature of the character region is determined, a characteristic character string can be determined from the character strings included in the character region.

  Preferably, the characteristic region is a portion in which the shape or background of the character is different from other portions in the image data.

  Preferably, the target character string determining means determines a character string included in the feature region among character strings obtained by character recognition as a character string portion.

  According to this aspect, the character string included in the feature region among the character strings recognized from the character region portion in the image data is determined and translated into the character string portion. Character strings can be translated and displayed.

  Preferably, the target character string determining means does not determine a predetermined character string as a character string portion even if it is a character string included in the feature region.

  According to this aspect, since the predetermined character string is not determined as the character string portion, the character string to be translated can be limited.

  Preferably, the related display control means displays on the display means an identification image indicating the determined characteristic area in the document image in which the image data is displayed on the display means.

  According to this aspect, since the identification image indicating the characteristic area is displayed in the document image displaying the image data, the characteristic area can be notified to the user.

  Preferably, the related display control means superimposes the translated character string image on the portion corresponding to the feature area of the original image displayed on the display means and displays the image data on the display means.

  According to this aspect, the translated character string image is superimposed on the portion corresponding to the feature area of the document image displaying the image data, so that only the feature area is displayed as the translated character string. For this reason, the translated part can be notified to the user.

  Preferably, the related display control unit generates an image of the translated character string with a size equal to or smaller than the size of the portion corresponding to the feature region of the document image in which the image data is displayed on the display unit.

  According to this aspect, an image of the translated character string can be displayed without changing any part other than the part corresponding to the feature region in the document image.

  Preferably, when the document reading unit continuously reads a plurality of pages of the document, the image display control unit outputs each of the plurality of pages of image data continuously output by the document reading unit in the order in which the document reading unit outputs them. The related display control means displays the translated character string on the display means in association with the image data of the corresponding page, and the image reading device has the translation means required for translation by the translation means. The information processing apparatus further includes timing means for measuring time for each of a plurality of pages, and the feature area determination means changes the accuracy of determining the feature area according to the measured translation time.

  According to this aspect, since the accuracy for determining the feature region is changed according to the translation time, the processing time for determining the feature region can be shortened by decreasing the accuracy if the translation time is increased. For this reason, the fluctuation | variation of the period after displaying the image of several pages until a display is complete | finished can be reduced.

  Preferably, when the document reading unit continuously reads a plurality of pages of the document, the image display control unit outputs each of the plurality of pages of image data continuously output by the document reading unit in the order in which the document reading unit outputs them. The related display control means displays the translated character string on the display means in association with the image data of the corresponding page.

  According to this aspect, page image data is displayed when each of a plurality of pages of image data output by continuously reading a plurality of pages of documents is displayed in a first period in the order of output. In the meantime, a character string obtained by translating a part of the image data of the page can be displayed.

  Preferably, the translation means does not translate when the determined character string portion is the same as the character string translated based on the image data of the page before the page of the image data including the character string portion.

  If this situation is followed, the character string to translate can be decreased and the translation time per page can be shortened.

  Preferably, the image display control means displays the image data of the page corresponding to the translated character string on the display means before the second period elapses after the translated character string is displayed on the display means by the related display control means. When the first period elapses after the image is displayed, the image data of the page corresponding to the translated character string is displayed on the display means until the second period elapses.

  According to this aspect, since the image data of the page corresponding to the translated character string is displayed until the second period elapses after the translated character string is displayed, the translated character string image is displayed at least for the second period. be able to.

  Preferably, the translation unit further includes a timing unit that counts the translation time required for translation for each of a plurality of pages, and the target character string determination unit determines the character determined per page according to the measured translation time. Change the number of column parts.

  According to this aspect, since the number of character string portions is changed according to the translation time of the previous page, the variation in the period from the start to the end of the display of the image data of a plurality of pages can be reduced. it can.

  Preferably, the translating means translates the character string portion determined using the character strings before and after the character string portion in addition to the determined character string portion.

  According to this aspect, since the character string portion is translated using the character string before and after the character string portion in addition to the character string portion, the accuracy of translating the character string portion can be improved.

  Preferably, the target character string determination unit does not determine the character string portion that is the same as the character string subjected to translation by the translation unit even if the character string is included in the feature region.

  According to this aspect, since the character string translated in the previous page is not translated, the translation time can be shortened.

  Preferably, the related display control means does not display the translated character string on the display means when the grammar of the translated character string is not normal.

  If this aspect is followed, since the image of the character string translated by mistake is not displayed, it can prevent reporting incorrect information.

  According to still another aspect of the present invention, a display control method is a display control method executed by a document reading apparatus including a document reading unit that reads a document and outputs image data, and the document reading unit includes a plurality of pages. An image display control step for displaying each of a plurality of pages of image data to be read and output continuously on the display means in the order that the document reading means outputs, and any one of the plurality of pages of image data While the is displayed, a position detection step for detecting the position designated by the user in the image data of the displayed page, and the position of the displayed page according to the detection of the position in the position detection step. A character region determination step for determining one or more character regions in which characters are represented from the image data, and 1 determined in the image data of the displayed page A character recognition step for recognizing a part of the character area including the position detected in the position detection step from the upper character area, a translation step for translating a character string obtained by character recognition into a predetermined language, and a translation A related display control step of displaying the translated character string obtained in this manner on the display means in association with the image data of the displayed page.

  According to this aspect, it is possible to provide a display control method capable of translating and displaying a part of a document while sequentially displaying images of a plurality of documents.

  According to still another aspect of the present invention, a display control method is a display control method executed by a document reading apparatus including a document reading unit that reads a document and outputs image data, and the document reading unit outputs the display control method. An image display control step for displaying the image data on the display means, a character area determination step for determining a character area representing the character from the image data output from the document reading means, and a portion of the character area determined in the image data A character recognition step for recognizing a character, and a character string determination step for determining a character string portion in which the corresponding portion in the image data has a predetermined characteristic among character strings obtained by character recognition in the character recognition step; The translation step of translating the character string included in the determined character string portion into a predetermined language and the translated character string obtained by the translation are associated with the image data It includes an associated display control step of displaying on the display means.

  According to this aspect, it is possible to provide a display control method capable of efficiently translating a character string in a document.

  According to still another aspect of the present invention, a display control program is a display control program executed by a computer that controls a document reading device including a document reading unit that reads a document and outputs image data. An image display control step for displaying each of a plurality of pages of image data output by the means for continuously reading a plurality of pages of the document on the display means for the first period in the order that the document reading means is output. While the image data is being displayed, a position detection step for detecting a position designated by the user in the image data of the displayed page, and a display in response to the position being detected in the position detection step A character area determining step for determining one or more character areas in which characters are represented from the image data of the displayed page; A character recognition step for recognizing a portion of the character region including the position detected in the position detection step from one or more character regions determined in the image data of the page, and a character string obtained by character recognition The computer is caused to execute a translation step for translating into a predetermined language and a related display control step for displaying the translated character string obtained by the translation on the display means in association with the image data of the displayed page.

  According to this aspect, it is possible to provide a display control program capable of translating and displaying a part of a document while sequentially displaying images of a plurality of documents.

  According to still another aspect of the present invention, a display control program is a display control program executed by a computer that controls a document reading device including a document reading unit that reads a document and outputs image data. An image display control step for displaying the image data output by the means on the display means, a character area determination step for determining a character area representing a character from the image data output by the document reading means, and a determination in the image data A character recognition step for recognizing a character area portion, and a target character for determining a character string portion having a predetermined characteristic in a corresponding portion of image data among character strings obtained by character recognition in the character recognition step A column determination step, a translation step for translating the character string included in the determined character string portion into a predetermined language, and The translated strings to execute the associated display control step of displaying on the display means in association with the image data, to the computer.

  According to this aspect, it is possible to provide a display control program that can efficiently translate a character string in a document.

1 is a perspective view showing an external appearance of an MFP in one embodiment of the present invention. FIG. 2 is a block diagram showing an outline of a hardware configuration of an MFP in one embodiment of the present invention. It is a top view which shows an example of an operation panel. 3 is a block diagram illustrating an example of functions of a CPU provided in the MFP according to the first embodiment. FIG. It is a flowchart which shows an example of the flow of a display control process. It is a flowchart which shows an example of the flow of the preview display control process in 1st Embodiment. It is a flowchart which shows an example of the flow of the related information display process in 1st Embodiment. It is a flowchart which shows a part of flow of the parallel display process in 1st Embodiment. It is a flowchart which shows a part of flow of the superimposition display process in 1st Embodiment. FIG. 10 is a first diagram illustrating an example of a preview screen including a document image. FIG. 10 is a first diagram illustrating an example of a preview screen in which a character string image is added to a document image in the parallel display mode. FIG. 10 is a first diagram illustrating an example of a preview screen in which a character string image is added to a document image in a superposition mode. FIG. 10 is a block diagram illustrating an example of functions of a CPU provided in an MFP according to a second embodiment. It is a flowchart which shows an example of the flow of the preview display control process in 2nd Embodiment. It is a 1st flowchart which shows an example of the flow of the related information display process in 2nd Embodiment. It is a 2nd flowchart which shows an example of the flow of the related information display process in 2nd Embodiment. It is a flowchart which shows a part of flow of the parallel display process in 2nd Embodiment. It is a flowchart which shows a part of flow of the superimposition display process in 2nd Embodiment. FIG. 10 is a second diagram illustrating an example of a preview screen including a document image. FIG. 10 is a second diagram illustrating an example of a preview screen in which a character string image is added to a document image in the parallel display mode. FIG. 10 is a second diagram illustrating an example of a preview screen in which a character string image is added to a document image in the superposition mode.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a perspective view showing an appearance of an MFP according to one embodiment of the present invention. FIG. 2 is a block diagram showing an outline of the hardware configuration of the MFP according to one embodiment of the present invention. 1 and 2, MFP 100 includes a main circuit 110, a document reading unit 130 for reading a document, an automatic document feeder 120 for conveying a document to document reading unit 130, and an image forming unit. 140, a paper feed unit 150 for supplying paper to the image forming unit 140, a post-processing unit 155 for processing paper on which an image is formed, and an operation panel 160 as a user interface.

  The automatic document feeder 120 supplies the document placed on the document tray to the document reading unit 130. When a plurality of documents are placed on the document tray, the automatic document feeder 120 handles the plurality of documents and conveys them one by one to the document reading unit 130 in order.

  The document reading unit 130 includes a photoelectric conversion device such as a CCD (Charge Coupled Device), optically reads a document transported from the automatic document transport device 120, and outputs an image as electronic data.

  The image forming unit 140 is a laser printer, and visualizes an image output from the document reading unit 130 or an image stored in the HDD 115 on a recording medium such as paper supplied from the paper supply unit 150. In the case of color printing, the image forming unit 140 forms an image with four color toners of yellow, magenta, cyan, and black. The image forming unit 140 may be an ink jet printer.

  The post-processing unit 155 executes a sorting process for sorting and discharging one or more sheets on which images are formed by the image forming unit 140, a punching process for punch hole processing, and a staple process for driving staple needles.

  The main circuit 110 includes a CPU 111, a communication interface (I / F) unit 112, a ROM 113, a RAM 114, a hard disk drive (HDD) 115 as a mass storage device, a facsimile unit 116, and a CD-ROM 118. External storage device 117. CPU 111 is connected to automatic document feeder 120, document reading unit 130, image forming unit 140, paper feeding unit 150, post-processing unit 155, and operation panel 160, and controls the entire MFP 100.

  The ROM 113 stores a program executed by the CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes a program. Further, the RAM 114 includes a setting value storage area and a setting screen storage area. The set value storage area is an area for storing a set value for executing processing. The setting screen storage area is an area for storing a setting screen displayed on the display unit 161. The RAM 114 temporarily stores read data (image data) continuously sent from the document reading unit 130.

  Communication I / F unit 112 is an interface for connecting MFP 100 to a network. The CPU 111 communicates with a computer connected to the network via the communication I / F unit 112 to transmit / receive data. The communication I / F unit 112 can communicate with a computer connected to the Internet via a network.

  The facsimile unit 116 is connected to the public switched telephone network (PSTN) and transmits facsimile data to the PSTN or receives facsimile data from the PSTN. The facsimile unit 116 stores the received facsimile data in the HDD 115 or outputs it to the image forming unit 140. The image forming unit 140 prints the facsimile data received by the facsimile unit 116 on a sheet. Further, the facsimile unit 116 converts the data stored in the HDD 115 into facsimile data, and transmits the facsimile data to a facsimile machine connected to the PSTN.

  The external storage device 117 is loaded with a CD-ROM (Compact Disk ROM) 118. The CPU 111 can access the CD-ROM 118 via the external storage device 117. The CPU 111 loads the program recorded on the CD-ROM 118 attached to the external storage device 117 to the RAM 114 and executes it. The medium for storing the program executed by the CPU 111 is not limited to the CD-ROM 118, but an optical disc (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, It may be a semiconductor memory such as a mask ROM, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM).

  Further, the program executed by the CPU 111 is not limited to the program recorded on the CD-ROM 118, and the program stored in the HDD 115 may be loaded into the RAM 114 and executed. In this case, another computer connected to the network via the communication I / F unit 112 may rewrite the program stored in the HDD 115 of the MFP 100 or may add a new program to be written. Further, MFP 100 may download a program from another computer connected to the network via communication I / F unit 112 and store the program in HDD 115. The program here includes not only a program directly executable by the CPU 111 but also a source program, a compressed program, an encrypted program, and the like.

  Operation panel 160 is provided on the upper surface of MFP 100. FIG. 3 is a plan view showing an example of the operation panel. Note that the dotted line in FIG. 3 is added to indicate an area in the operation panel 160 where the hard key unit 167 is arranged, and does not actually exist. Referring to FIG. 3, operation panel 160 includes a display unit 161 and an operation unit 163. The display unit 161 is, for example, a liquid crystal display device (LCD), and displays an instruction menu for a user, information about acquired image data, and the like.

  The operation unit 163 includes a touch panel 165 and a hard key unit 167. The touch panel 165 is provided to overlap the display unit 161 on the upper surface or the lower surface of the display unit 161. Hard key portion 167 includes a plurality of hard keys. The hard key is, for example, a contact switch. The touch panel 165 detects a position designated by the user on the display surface of the display unit 161. When the user operates the MFP 100, the user often takes an upright posture, and thus the display surface of the display unit 161, the operation surface of the touch panel 165, and the hard key unit 167 are arranged facing upward. This is because the user can easily visually recognize the display surface of the display unit 161 and the user can easily instruct the operation unit 163 with a finger.

  FIG. 4 is a block diagram illustrating an example of the functions of the CPU provided in the MFP according to the first embodiment. The functions shown in FIG. 4 are functions formed in CPU 111 when CPU 111 provided in MFP 100 executes a display control program stored in ROM 113, HDD 115, and CD-ROM 118. Referring to FIG. 4, CPU 111 includes document reading control unit 11, image display control unit 13, character area determination unit 15, character recognition unit 17, translation unit 23, check unit 25, and related display control. Part 27 and a position detector 71.

  The document reading control unit 11 controls the automatic document feeder 120 and the document reading unit 130 to cause the document reading unit 130 to read the document, and the image reading unit 130 reads the document and outputs the image data to the image display control unit. 13 and the character area determination unit 15. When the document reading control unit 11 causes the document reading unit 130 to read a plurality of documents, the document reading unit 130 sequentially reads the plurality of documents one by one. The document reading unit 130 outputs a plurality of pages of image data in the order in which a plurality of documents are read one by one. The document reading control unit 11 outputs one page of image data to the image display control unit 13 and the character area determination unit 15 every time the document reading unit 130 outputs one page of image data corresponding to one document. .

  The image display control unit 13 receives image data from the document reading control unit 11, controls the display unit 161, and a document to be displayed on the display unit 161 based on the image data input from the document reading control unit 11. An image is generated and the document image is displayed on the display unit 161. Further, the image display control unit 13 outputs the position of the document image on the display surface of the display unit 161 to the related display control unit 27. The image display control unit 13 generates a document image having a size that can be displayed on the display unit 161 based on the image data input from the document reading control unit 11, and displays the document image on the display unit 161. Also, when a plurality of pages of image data are sequentially input from the document reading control unit 11, the image display control unit 13 displays a plurality of pages of document images respectively corresponding to the plurality of pages of image data for the first time. . The first time is not limited, but is preferably longer than the reading time for the document reading unit 130 to read one document. Thus, while the document reading unit 130 is reading a plurality of pages of documents, a plurality of pages of document images corresponding to the plurality of pages of documents can be sequentially displayed on the display unit 161.

  The position detection unit 71 controls the touch panel 165 to detect a position where the user points to the display surface of the display unit 161, and outputs the detected position to the character region determination unit 15 as the designated position.

  The character area determination unit 15 receives a plurality of pages of image data one by one from the document reading control unit 11 and inputs an instruction position from the position detection unit 71. The character area determination unit 15 sets the image data of the page input from the document reading control unit 11 immediately before the instruction position is input from the position detection unit 71 as a processing target, and the image of the page set as the processing target. One or more character areas in which characters are represented in the data are determined.

  Further, the character region determination unit 15 specifies a position corresponding to the position in the document image specified by the designated position input from the position detection unit 71 in the image data of the page set as the processing target. The character area determination unit 15 determines a character area including the specified position as a processing target from one or more determined character areas. In other words, the character area corresponding to the position instructed by the user in the document image is determined as the processing object among one or more character areas included in the image data of the page set as the processing object. The character area determination unit 15 outputs the part of the image data corresponding to the character area determined as the processing target and the position information to the character recognition unit 17. The position information indicates the position that the character area occupies in the image data input from the document reading control unit 11. Processing for determining a character area in image data is well known. For example, although not limited, character data is detected by binarizing image data and detecting a mountain having a high frequency of black pixels in a projection histogram in which the number of black pixels in the binarized image data is counted in each of the vertical and horizontal directions. Determine the area.

  The character recognizing unit 17 performs character recognition on the character region portion in the image data input from the character region determining unit 15, character strings obtained by character recognition, and position information input from the character region determining unit 15 Are output to the translation unit 23. A technique for recognizing characters from a character region portion in image data is well known. For example, but not limited to, a pattern matching technique can be used.

  The translation unit 23 translates the character string into a predetermined language in response to the input of the combination of the character string and the position information from the character recognition unit 17. When the language before translation and the language after translation are determined in advance and the character string input from the character recognition unit 17 is determined in advance as the language before translation, the character string is converted into the language after translation. As a predetermined language. The translation unit 23 outputs a set of the translated character string, which is a character string obtained by translating the character string input from the character recognition unit 17, and the position information to the check unit 25.

  The check unit 25 receives a set of the translation character string and the position information from the translation unit 23, and determines whether or not the translation character string is correctly translated. Since the technique for determining whether or not the translation is correct is well known, the description will not be repeated here. For example, the grammar of the translated character string input from the character recognition unit 17 from the translation unit 23 is checked to determine whether the grammar is correct. Judging. The check unit 25 outputs a set of the translation character string, the position information, and the check result to the related display control unit 27.

  The related display control unit 27 receives a position on the display surface of the document image display unit 161 from the image display control unit 13, and receives a set of a translation character string, position information, and a check result from the check unit 25. The related display control unit 27 displays the translated character string image as a character string image on the display unit 161 for the second time in association with the document image. The second period is preferably shorter than the first period in which the document image is displayed. This is because the character string image is generated after the document image is displayed. The related display control unit 27 outputs the elapsed time after displaying the character string image to the image display control unit 13. When displaying a plurality of character string images, the related display control unit 27 outputs to the image display control unit 13 an elapsed time since the last displayed character string image among the plurality of character string images.

  The related display control unit 27 includes a mistranslation non-display unit 51, a parallel display unit 53, and a superimposed display unit 55. The related display control unit 27 causes one of the parallel display unit 53 and the superimposed display unit 55 to function effectively. Which of the parallel display unit 53 and the superimposed display unit 55 is to be functioned may be determined according to a setting predetermined by the user.

  The mistranslation non-display unit 51 does not display the character string image when the check result input from the check unit 25 indicates that the grammar is not correct. Specifically, the mistranslation non-display unit 51 outputs a display instruction to the parallel display unit 53 and the superimposed display unit 55 only when the check result input from the check unit 25 indicates that the grammar is correct. When the check result input from the unit 25 indicates that the grammar is not correct, the display instruction is not output to the parallel display unit 53 and the superimposed display unit 55. The display instruction includes position information paired with the check result.

  The parallel display unit 53 includes a first character string image generation unit 61, a first size determination unit 63, and a corresponding portion instruction unit 65. The first size determining unit 63 determines the size of the character string image with reference to the size of the document image. For example, when one document image is displayed on the display unit 161, a blank area other than the document image is determined on the display surface of the display unit 161, and the size of the character string image is determined to be equal to or smaller than the blank area.

  The first character string image generation unit 61 generates an image of the translated character string input from the check unit 25 with the size determined by the size determination unit 63 as a character string image. The first character string image generation unit 61 displays the character string image and the document image at the same time. The first character string image generation unit 61 displays the character string image and the document image side by side at positions where they do not overlap. The direction in which the character string image is arranged on the document image may be any of the top, bottom, left and right directions of the document image. Further, the first character string image generation unit 61 is specified by the position information in the document image in the blank area in order to indicate that the character string image corresponds to the position specified by the position information in the document image. A character string image is arranged at a position closest to the position.

  The corresponding portion instruction unit 65 synthesizes the identification image in an area corresponding to the character area specified by the position information in the document image. The identification image is a line surrounding an area corresponding to a character area in the document image. Further, the identification image may be an image in which the area corresponding to the character area in the document image is expressed in a single color. As a result, the identification image is combined with the area corresponding to the character area, so that the user can be notified of the area corresponding to the character string image in the document image. In addition, even when there are a plurality of character string images, the identification image is synthesized in an area corresponding to each of the plurality of character string images. Users can be notified.

  The superimposed display unit 55 includes a second character string image generation unit 67 and a second size determination unit 68. The second size determining unit 68 determines the size of the character string image as the size of the portion corresponding to the feature area in the document image.

  The second character string image generation unit 67 generates an image of the translated character string input from the check unit 25 as a character string image with the size determined by the second size determination unit 68. The second character string image generation unit 67 displays the character string image superimposed on the document image. For this reason, the part corresponding to the specific area of the document image is replaced with the character string image.

  The image display control unit 13 includes a display period extension unit 14. The image display control unit 13 displays the document image for the first period, but the first time may elapse before the elapsed time input from the related display control unit 27 elapses. This is because the timing when the character string image is displayed by the related display control unit 27 is later than the timing when the document image is displayed. Even if the first period has elapsed since the display of the document image, the display period extending unit 14 displays the related display if the elapsed time input from the related display control unit 27 is less than the second period. The period for displaying the document image is extended until the elapsed time input from the control unit 27 exceeds the second period. Thereby, the character string image displayed by the related display control unit 27 can be displayed for at least the second period.

  FIG. 5 is a flowchart illustrating an example of the flow of display control processing. The display control process is a process executed by CPU 111 when CPU 111 provided in MFP 100 executes a control program stored in ROM 113, HDD 115, and CD-ROM 118. Referring to FIG. 5, CPU 111 determines whether a scan instruction has been accepted (step S01). When the user inputs an operation for instructing reading of a document to the operation unit 163, a scan instruction is accepted. For example, a scan instruction is accepted when a key to which an operation for instructing reading of a document, for example, a start key is instructed, among a plurality of hard keys included in the hard key unit 167 included in the operation unit 163 is received. The process waits until a scan instruction is accepted (NO in step S01). If a scan instruction is accepted (YES in step S01), the process proceeds to step S02.

  In step S02, reading of a document is started. The automatic document feeder 120 and the document reading unit 130 are controlled to cause the document reading unit 130 to read the document conveyed by the automatic document feeder 120. When a plurality of documents are placed on the automatic document feeder 120, the automatic document feeder 120 conveys the plurality of documents one by one to the document reading unit 130, so that the document reading unit 130 accepts one document. Each time it is read, one image data is output.

  In step S03, it is determined whether a related information display instruction has been accepted. If a related information display instruction is accepted, the process proceeds to step S04; otherwise, the process proceeds to step S06. When the user inputs an operation to instruct display of related information to the operation unit 163, a related information display instruction is accepted. For example, when a key to which an operation for instructing display of related information is assigned among a plurality of hard keys included in the hard key unit 167 included in the operation unit 163, a related information display instruction is accepted.

  In step S04, a translation language is set. Set the language before translation and the language after translation. For example, when English is translated into Japanese, English is set as the language before translation, and Japanese is set as the language after translation. The translation language may be set according to an operation input by the user to the operation unit 163 by displaying a setting screen on the display unit 161, or may be set as a default value in advance.

  In step S05, the related information display mode is set, and the process proceeds to step S06. In step S06, it is determined whether or not the document reading process started in step S02 is completed. When all of a plurality of documents placed on automatic document feeder 120 are read by document reading unit 130, it is determined that the document reading process has been completed. If the document reading process has not been completed, the process returns to step S03. If the document reading process has been completed, the process ends.

  FIG. 6 is a flowchart illustrating an example of a preview display control process according to the first embodiment. The preview display control process is a process executed by CPU 111 when CPU 111 provided in MFP 100 executes a display control program stored in ROM 113, HDD 115, and CD-ROM 118. The preview display control process is executed in parallel with the display control process shown in FIG.

  Referring to FIG. 6, CPU 111 determines whether image data output from document reading unit 130 has been acquired (step S11). The process waits until the image data output by the document reading unit 130 is acquired (NO in step S11). When the image data is acquired (YES in step S11), the process proceeds to step S12.

  In step S12, the document image is displayed on display unit 161, and the process proceeds to step S13. In step S 11, the image data acquired from the document reading unit 130 is converted into a document image, and the document image is displayed on the display unit 161. The document image is an image generated in a size that can be displayed on the display unit 161 from the image data output from the document reading unit 130.

  In step S13, the timer ET1 is reset. The timer ET1 measures the time after displaying the document image. The value of the timer ET1 indicates an elapsed time after displaying the document image. In the next step S14, it is determined whether or not the indicated position has been detected. The touch panel 165 is controlled, and the position where the user points to the display surface of the display unit 161 is detected as the indicated position. If the designated position is detected, the process proceeds to step S15; otherwise, the process proceeds to step S23. In step S23, it is determined whether or not the value of timer ETI is equal to or greater than threshold value ETH1. If the value of timer ETI is equal to or greater than threshold value ETH1, the process returns to step S11; otherwise, the process returns to step S14.

  In step S15, a related information display process is executed, and the process proceeds to step S16. Although the details of the related information display process will be described later, it is a process of displaying an image of a translated character string obtained by translating a portion at a position designated by the user in a document image. In step S16, timer ET2 is reset and the process proceeds to step S17. The timer ET2 measures the time after displaying the translated character string image. The value of the timer ET2 indicates the elapsed time after displaying the translated character string image.

  In step S17, it is determined whether the value of timer ETI is equal to or greater than threshold value ETH1. The process waits until the value of timer ETI is equal to or greater than threshold value ETH1 (NO in step S17), and when the value of timer ETH is equal to or greater than threshold value ETH1 (YES in step S17), the process proceeds to step S18. In step S18, it is determined whether or not the value of timer ETH2 is equal to or greater than threshold value ETH2. The process waits until the value of timer ETH is equal to or greater than threshold value ETH2 (NO in step S18), and when the value of timer ETH is equal to or greater than threshold value ETH2 (YES in step S18), the process returns to step S11.

  FIG. 7 is a flowchart illustrating an example of the flow of related information display processing according to the first embodiment. The related information display process in the first embodiment is a process executed in step S15 of the preview display control process in the first embodiment shown in FIG. Before the preview display control process is executed, the image data output by the document reading unit 130 reading the document is acquired, and the designated position designated by the user is detected in the document image. The image data acquired in the stage before the preview display control process is executed is image data output by the document reading unit 130 reading the document, and an image whose position in the corresponding document image is designated by the user. It is.

  Referring to FIG. 7, CPU 111 determines a character area in the image data output by document reading unit 130 (step S31). The image reading unit 130 reads the document and outputs the image data as a processing target, and determines one or more character areas in which characters are represented in the image data set as the processing target. In the next step S32, a character area corresponding to the position designated by the user in the document image is determined from one or more character areas. Specifically, a character area including a position corresponding to the position in the document image specified by the designated position instructed by the user in the document image in the image data output by the document reading unit 130 reading the document. decide.

  In the next step S33, the portion of the character area determined in step S32 is recognized in the image data output by the document reading unit 130 reading the document. In step S34, the character string recognized as a character is translated. A character string obtained by translating a character string is called a translated character string.

  In the next step S35, the grammar of the translated character string is checked. Then, it is determined whether the result of checking the grammar is normal. If the result of the grammar check is normal, the process proceeds to step S37; otherwise, the process returns to the preview display control process. In step S37, the process branches depending on the related display mode. If the related display mode is set to the parallel display mode, the process proceeds to step S38. If the related display mode is set to the superimposed display mode, the process proceeds to step S39. In step S38, the parallel display process in the first embodiment is executed, and the process returns to the preview display control process. In step S39, the superimposed display process in the first embodiment is executed, and the process returns to the preview display control process.

  FIG. 8 is a flowchart showing a part of the flow of parallel display processing in the first embodiment. The parallel display process is a process executed in step S38 in FIG. Referring to FIG. 8, CPU 111 determines the size of the character string image (step S71). The character string image is an image generated in the next step S72. The size of the character string image is determined based on the size of the document image. For example, when one document image is displayed on the display unit 161, a blank area other than the document image is determined on the display surface of the display unit 161, and the size of the translated character string image is determined to be equal to or smaller than the blank area. .

  In the next step S72, a character string image is generated. An image of the translated character string obtained by translating in step S34 of the related information display process shown in FIG. 7 with the size determined in step S71 is generated as a character string image. In the next step S73, the generated character string images are displayed side by side at positions that do not overlap the document image. In order to indicate that the character string image corresponds to the position specified at the indicated position instructed by the user in the document image, the character string image is positioned closest to the position specified at the indicated position in the document image. It is preferable to arrange.

  In the next step S74, the identification image is combined with the document image, and the process returns to the related information display process. The identification image is combined with an area in the original image corresponding to the character area selected in step S32 of the related information display process shown in FIG. The identification image is a line surrounding the area corresponding to the character area in the document image. Further, the identification image may be an image in which the area corresponding to the character area in the document image is represented by a single color. Since the identification image is synthesized with the area corresponding to the character area, the user can be notified of the area corresponding to the character string image in the document image.

  FIG. 9 is a flowchart showing a part of the flow of the superimposed display process in the first embodiment. The superimposed display process in the first embodiment is a process executed in step S39 in FIG. Referring to FIG. 9, CPU 111 determines the size of the character string image (step S71A). The character string image is an image generated in the next step S72A. The size of the character string image is determined to be the same size as the region corresponding to the character region selected in step S32 of the related information display process shown in FIG. 7 in the document image displayed on the display unit 161.

  In the next step S72A, a character string image is generated. An image of the translated character string obtained by translating in step S34 of the related information display processing shown in FIG. 7 with the size determined in step S71A is generated as a character string image. In the next step S73A, the character string image is synthesized with the document image, and the process returns to the related information display process. The character string image generated in step S72A replaces the image in the region corresponding to the character region in the document image displayed on the display unit 161. As a result, the character string image is synthesized in the area corresponding to the character area of the document image displayed on the display unit 161, so that the character string image is displayed superimposed on the document image. Users can be notified.

  FIG. 10 is a first diagram illustrating an example of a preview screen including a document image. Referring to FIG. 10, preview screen 300 includes a document image 301. The document image 301 is an image in which Japanese characters are represented, and includes three character string areas 303, 305, and 307. The dotted lines in the figure are added to indicate the character string areas 303, 305, and 307, and do not actually exist. The preview screen 300 is displayed for a first period after the document image 301 is displayed.

  FIG. 11 is a first diagram illustrating an example of a preview screen in which a character string image is added to a document image in the parallel display mode. The preview screen 310 shown in FIG. 11 is displayed on the display unit 161 after the user designates the character area 303 in the document image 301 of the preview screen 300 shown in FIG. Referring to FIG. 11, preview screen 310 includes a document image 301 and a character string image 313. The character string image 313 is an image generated from a translated character string obtained by translating a character string included in the character area 303 of the document image 301. The document image 301 includes an identification image 303A. The identification image 303 </ b> A is a rectangular image surrounding the character area 303. A user viewing the preview screen 310 can visually recognize the translated character string from the character string image 313. Further, it can be recognized from the identification image 303A that the translated character string included in the character string image 313 is a character string obtained by translating the character string in the character area 303 in the document image 301.

  FIG. 12 is a first diagram illustrating an example of a preview screen in which a character string image is added to a document image in the superposition mode. The preview screen 310 shown in FIG. 12 is displayed on the display unit 161 after the user designates the character area 303 in the document image 301 of the preview screen 300 shown in FIG. Referring to FIG. 12, preview screen 320 includes a document image 301A in which character string image 313A is arranged instead of character region 303 in document image 301 of preview screen 300 shown in FIG. The document image 301A includes a character string image 313A and two character areas 305 and 307. Character string image 313A is the same size as character area 303 in document image 301, and is an image generated from a translated character string obtained by translating a character string included in character area 303 of document image 301.

  The user viewing the preview screen 320 can visually recognize the translated character string from the character string image 313A, and the translated character string included in the character string image 313 from the position where the character string image 313A is arranged on the document image 301A It can be recognized that the character string in the character area 303 in the document image 301 is a translated character string.

<First Modification>
In the first embodiment, a plurality of documents are read in order, and a plurality of image data obtained by reading are displayed on the display unit 161 as a document image in the order of reading, for the first period. Then, among one or more character areas included in the image data, the character string included in the character area corresponding to the position designated by the user in the document image displayed on the display unit 161 is translated.

  While a plurality of document images are displayed one by one in order for the first period, the document image displayed at the time pointed by the user may be displayed for a longer time than the first period. When a plurality of document images are displayed on the display unit 161, for example, when two images can be displayed, each of the plurality of document images may be displayed for the first period. Specifically, the original image of the first page is displayed on the display unit 161, and when the half of the first period has elapsed since the display of the original image of the first page, the original image of the second page is displayed. Is displayed on the display unit 161. At this stage, the original image of the first page and the original image of the second page are displayed on the display unit 161. Thereafter, when the first period has elapsed since the display of the first page of the document image, the display of the first page of the document image is terminated and the third page of the document image is displayed. In this way, when half of the first period has elapsed since the display of the nth page of the original image, the (n + 1) th page of the original image is displayed, and the nth page of the original image is displayed. When the first period has elapsed, the display of the nth page of the document image is terminated, and the (n + 2) th page of the document image is displayed. In this case, when the nth page of the document image is designated by the user, a translated character string obtained by translating the character string included in the designated character area is displayed in association with the nth page of the document image. Further, the document image of the nth page may be displayed for a period longer than the first period.

  As described above, MFP 100 according to the first embodiment functions as a document reading device, and each document image of a plurality of pages of image data obtained by continuously reading a plurality of pages of documents is displayed for each first period. And when any of the image data of multiple pages is displayed, the position indicated by the user is detected in the image data of the displayed page. The character region including the position is recognized, the character string recognized is translated into a predetermined language, and the translated character string is displayed in association with the document image. For this reason, the character area image including the position designated by the user in the displayed document image is recognized and translated, so that the processing amount of character recognition and translation can be reduced, and the processing time can be shortened. can do. Therefore, it is possible to translate and display a part of the document while sequentially displaying images of a plurality of documents.

  In addition, the MFP 100 displays the translated character string image at a predetermined relative position with respect to the document image in a size determined relative to the size of the displayed image. For example, in the parallel display mode, the translated character string image has a size equal to or smaller than the blank area other than the original image, and is arranged in the blank area at a position closest to the character area corresponding to the translated character string. In the superimposed display mode, the size of the translated character string image is the same as the size of the character region corresponding to the translated character string, and the translated character string image is superimposed on the character string region. For this reason, it is possible to notify the user of the portion corresponding to the translated character string in the image.

  In addition, when the MFP 100 displays a plurality of pages of image data output by continuously reading a plurality of pages of the document as a document image for each of the first periods in the order in which they are output, While being displayed, a character string obtained by translating a part of the document image can be displayed.

  In addition, when the first period elapses after the document image is displayed before the second period elapses after the translation character string image is displayed, the MFP 100 causes the document to remain until the second period elapses. Since the image is displayed, the image of the translated character string can be displayed for at least the second period.

<Second Embodiment>
The appearance and hardware configuration of MFP 100 in the second embodiment are the same as the appearance and hardware configuration of MFP 100 in the first embodiment shown in FIG. 1 or FIG. Therefore, description is not repeated here.

  FIG. 13 is a block diagram illustrating an example of the functions of the CPU provided in the MFP according to the second embodiment. The function shown in FIG. 13 is different from the function shown in FIG. 4 in that the character area determining unit 15, the character recognizing unit 17, the translating unit 23, and the related display control unit 27 are replaced with the character area determining unit 15A and the character recognizing unit 17A. The points changed to the translation unit 23A and the related display control unit 27A, the feature region determining unit 19, the target character string determining unit 21, and the time measuring unit 29 are added. The other functions are the same as the functions shown in FIG. 4, and thus description thereof will not be repeated here.

  The character area determination unit 15A receives image data from the document reading control unit 11 and determines a character area in which characters are represented in the image data. The character region determination unit 15A outputs the character region portion of the image data and the position information to the character recognition unit 17A and the feature region determination unit 19. The position information indicates the position that the character area occupies in the image data input from the document reading control unit 11. Since the process of determining the character area in the image data is well known, the description will not be repeated here. When determining a plurality of character regions from one page of image data, the character region determining unit 15 determines the character recognition unit 17A and the feature region for each page by combining the character region portion and the position information for each page. To the unit 19.

  The character recognition unit 17A performs character recognition on the character region portion in the image data input from the character region determination unit 15A, and sets a combination of a character string obtained by character recognition and position information paired with the character region. The data is output to the translation unit 23A and the target character string determination unit 21. Since the technique for recognizing characters existing in an image is well known, the description will not be repeated here. When a plurality of sets of character area portions and position information in the image data are input from the character region determination portion 15A, the character recognition section 17A recognizes the character area portions in the image data for each of the plurality of sets. Then, a set of the character string obtained by character recognition and the position information paired with the character area is output to the translation unit 23A and the target character string determination unit 21. When a plurality of sets of character area portions and position information in the image data are input from the character area determination unit 15A, the character recognition unit 17A receives a set of character string and position information from the character region determination unit 15A. Are output to the translation unit 23 </ b> A and the target character string determination unit 21 by the number of pairs.

  The feature area determination unit 19 receives a set of the character area portion and the position information in the image data from the character area determination unit 15. The feature region determination unit 19 determines the feature region based on the character region portion in the image data input from the character region determination unit 15. The feature region is a character or background region in which a character included in the character region portion in the image data or a background that is a portion other than the character is different from other characters or backgrounds. When the feature region is determined, the feature region determination unit 19 outputs a set of feature region information and position information for specifying the determined feature region to the target character string determination unit 21 and the related display control unit 27A. When the feature region is not determined, the feature region determination unit 19 outputs a set of information indicating that the feature region has not been determined and position information to the target character string determination unit 21 and the related display control unit 27A. The characteristic area information indicates the position of the character included in the characteristic area in the character string included in the character area portion in the image data. The feature region determination unit 19 determines a feature region for each character region, and outputs a set of feature region information and position information for specifying the determined feature region to the target character string determination unit 21 and the related display control unit 27. . When the feature region determination unit 19 determines a plurality of feature regions from an image of one character region, the target character string determination unit 21 and the related display control unit determine a set of feature region information and position information of each of the plurality of feature regions. 27.

  The feature area determination unit 19 includes a character reference determination unit 31 and a background reference determination unit 33. The character reference determining unit 31 determines an area including a character having at least one of a font, a color, a size, and a line thickness different from that of other characters in the character area in the image data. Specifically, the character reference determining unit 31 extracts a plurality of characters included in the character region portion in the image data. Furthermore, the character reference determination unit 31 determines the font of each of the extracted plurality of characters. The character reference determination unit 31 uses a font determined for each of a plurality of characters to determine a character having the smallest number of characters with the same font as a character having a font different from that of other characters, and in the character area. A portion including a character whose font is different from other characters is determined as a feature region. When the font is determined by the character recognition unit 17A, the font determined by the character recognition unit 17A may be used.

  Further, the character reference determining unit 31 determines the color of each of the extracted plurality of characters. The character reference determining unit 31 determines a character whose color determined for each of the plurality of characters is different from a predetermined color as a character having a color different from that of the other characters, and the color of the other characters in the character region. Are determined as feature regions. The predetermined color is a predetermined color, for example, black. Further, the character reference determining unit 31 uses the colors determined for each of the plurality of characters to determine the character having the smallest number of characters having the same color as a character having a color different from that of other characters. Also good.

  Further, the character reference determining unit 31 determines the size of each of the extracted plurality of characters. The character reference determination unit 31 determines a character having a size larger than that of other characters using the sizes determined for each of the plurality of characters, and is determined to have a size larger than that of the other characters in the character region. A portion including characters is determined as a feature region.

  Moreover, the character reference | standard determination part 31 determines the thickness of each line of the extracted several character. The character reference determination unit 31 uses the line thickness determined for each of the plurality of characters to determine a character that is thicker than the other characters, and is thicker than the other characters in the character region. A portion including characters is determined as a feature region.

  The background reference determination unit 33 determines, as a characteristic part, a part including a character whose background is different from that of other characters in the character region of the image data. The background reference determination unit 33 determines a character portion including an image having a predetermined shape as the character background. The predetermined shape is, for example, an underline or a rectangular line. Specifically, the background reference determination unit 33 extracts a plurality of characters included in the image of the character region. The background reference determination unit 33 determines a character including a line in the background portion among the plurality of extracted characters, and determines a portion including a character with a line added to the background portion in the character region as a feature region.

  In addition, the background reference determination unit 33 determines a character portion in which the background color of the character is different from the background color of other characters as a feature portion. The background reference determination unit 33 determines the color of the background portion of each of the plurality of extracted characters, and determines that the background color of each of the plurality of characters is different from a predetermined color, A different character is determined, and a part including a character having a background color different from that of other characters in the character region is determined as a feature region. The predetermined color is a predetermined other color, for example, white. In addition, the character reference determining unit 31 uses the background color determined for each of the plurality of characters to select the character with the smallest number of characters having the same background color and the character having a different background color from other characters. You may make it decide to.

  The target character string determination unit 21 receives the combination of the character string and the position information from the character recognition unit 17A, and has not determined the combination of the feature region information and the position information or the feature region from the feature region determination unit 19 A pair of information indicating position information and position information is input. The position information indicates the position of the character area, and indicates the area occupied by the character area in the image data obtained when the document reading control unit 11 causes the document reading unit 130 to read the document. The target character string determination unit 21 specifies feature area information that is paired with the same position information as the pair of position information paired with the character string input from the character recognition unit 17, and determines the character string input from the character recognition unit 17. Among these, the character string specified by the feature area information is determined as the target character string. The target character string determination unit 21 outputs a set of the target character string and the position information to the translation unit 23A. When the target character string determination unit 21 sets the position information that is the same as the position information paired with the character string input from the character recognition unit 17 and information indicating that the feature area has not been determined, the position information The target character string is not determined for. If the target character string is not determined, the target character string determination unit 21 outputs a set of information indicating that the target character string has not been determined and position information to the translation unit 23A.

  The target character string determination unit 21 includes a default character string exclusion unit 41 and an identical character string exclusion unit 43. If the character string specified by the feature area information among the character strings input from the character recognition unit 17 is a predetermined character string, the default character string exclusion unit 41 does not determine the character string as the target character string. . The character string predetermined so as not to be determined as the target character string is, for example, a character used to indicate the order of headings in a sentence, and is a character string used in combination with a number indicating the order of headings. Specifically, the character string “item” and the character “chapter”. A character string predetermined so as not to be determined as the target character string may be set in advance by the user. If the character string specified by the feature area information among the character strings input from the character recognition unit 17 is a predetermined character string, the default character string exclusion unit 41 outputs the character string to the translation unit 23A. do not do.

  The same character string excluding unit 43 targets the same character string as the character string determined as the target character string based on the image data of the processed page among the images of the plurality of pages output by the document reading control unit 11. Do not decide on a character string. The same character string excluding unit 43 stores the history of the character string determined as the target character string, and the character string included in the feature area among the character strings input from the character recognition unit 17 is included in the history. When compared with the target character string and the same as the target character string included in the history, the character string is not output to the translation unit 23A.

  The translation unit 23A receives a set of a character string and position information from the character recognition unit 17A, and has not determined a set of the target character string and position information or a target character string from the target character string determination unit 21. A set of information and position information is input. The character string input from the character recognition unit 17A is a character string included in the character area specified by the position information. When the combination of the target character string and the position information is input from the target character string determination unit 21, the translation unit 23A translates the target character string into a predetermined language. When the language before translation and the language after translation are determined in advance and the character string input from the character recognition unit 17A is determined in advance as the language before translation, the character string is converted into the language after translation. As a pre-determined language.

  The translation unit 23A includes a front-rear reference unit 47. When the set of the target character string and the position information is input from the target character string determination unit 21, the front / rear reference unit 47 translates the character string included in the character area specified by the position information, and the character after the translation Of the columns, a portion corresponding to the target character string paired with the same position information is determined as a translated character string. The target character string is a character string input from the character recognition unit 17, in other words, a part of the character string included in the character area specified by the position information. For this reason, since the translation part 23 translates using the character string before and behind a target character string, even when there are multiple candidates after translating a target character string, it can translate correctly. The translation unit 23A outputs a set of a character string obtained by translating the character string included in the character region, the translated character string, and the position information to the check unit 25A. When the pair of information and position information indicating that the target character string has not been determined is input from the target character string determination unit 21, the front / rear reference unit 47 includes characters included in the character area specified by the position information. Do not translate columns. In this case, the front-rear reference unit 47 outputs nothing to the check unit 25A when a set of information and position information indicating that the target character string has not been determined is input from the target character string determination unit 21.

  The check unit 25A receives a set of a character string obtained by translating a character string included in the character region, a translated character string, and position information from the translation unit 23A, and determines whether the translated character string is correctly translated. To do. Since the technique for determining whether or not the translation is correct is well known, the description will not be repeated here. For example, the grammar of the character string of the translated word is checked. The check unit 25A determines whether the translated character string is correctly translated by checking the grammar of the character string obtained by translating the character string included in the character area. The check unit 25A outputs a set of the translation character string, the position information, and the check result to the related display control unit 27A.

  The related display control unit 27A receives a position on the display surface of the document image display unit 161 from the image display control unit 13, and receives a set of a translation character string, position information, and a check result from the check unit 25A. A set of feature area information and position information is input from the area determination unit 19. The related display control unit 27A displays the image of the translated character string as a character string image in association with the document image on the display unit 161 for the second time. The second period is preferably shorter than the first period in which the document image is displayed. This is because the character string image is generated after the document image is displayed. The related display control unit 27 outputs the elapsed time after displaying the character string image to the image display control unit 13. When displaying a plurality of character string images, the related display control unit 27 outputs to the image display control unit 13 an elapsed time since the last defined character string image among the plurality of character string images is displayed.

  The related display control unit 27 </ b> A includes a mistranslation non-display unit 51, a parallel display unit 53, and a superimposed display unit 55. The related display control unit 27 causes one of the parallel display unit 53 and the superimposed display unit 55 to function effectively. Which of the parallel display unit 53 and the superimposed display unit 55 is to be functioned may be determined according to a setting predetermined by the user. Here, the related display mode is set to either the parallel display mode or the superimposed display mode, and the related display control unit 27A performs parallel display when the related display mode is set to the parallel display mode. If the related display mode is set to the superimposed display mode, the superimposed display unit 55 is caused to function, and the parallel display unit 53 is not allowed to function.

  The mistranslation non-display unit 51A does not display the character string image when the check result input from the check unit 25 indicates that the translation is not correct. Specifically, the mistranslation non-display unit 51 outputs a display instruction to the parallel display unit 53 and the superimposition display unit 55 only when the check result input from the check unit 25 indicates that the translation is correct. When the check result input from the unit 25 indicates that the translation is not correct, the display instruction is not output to the parallel display unit 53 and the superimposed display unit 55. The display instruction includes position information paired with the check result.

  The parallel display unit 53A includes a first character string image generation unit 61A, a first size determination unit 63A, and a corresponding portion instruction unit 65A. The first size determining unit 63A determines the size of the character string image based on the size of the document image. For example, when one document image is displayed on the display unit 161, a blank area other than the document image is determined on the display surface of the display unit 161, and the size of the character string image is determined to be equal to or smaller than the blank area. In some cases, a plurality of target character strings are determined for one page image. In this case, a plurality of character images are arranged for one original image, so that the character strings are based on the number of target character strings. Determine the size of the image.

  The first character string image generation unit 61A generates a character string image from the translated character string input from the check unit 25 with the size determined by the size determination unit 63A. The first character string image generation unit 61A displays the character string image and the document image at the same time. The first character string image generation unit 61A displays the character string image and the document image side by side so that they do not overlap. The direction in which the character string image is arranged on the document image may be any of the top, bottom, left and right directions of the document image. Further, the first character string image generation unit 61A is specified by the position information in the document image in the blank area in order to indicate that the character string image corresponds to the position specified by the position information in the document image. A character string image is arranged at a position closest to the position.

  Corresponding portion instruction unit 65A combines the identification image with an area corresponding to the feature area in the document image. The identification image is a line surrounding an area corresponding to the feature area. Further, the identification image may be an image in which the area corresponding to the feature area is expressed in a single color. As a result, the identification image is combined with the area corresponding to the feature area, so that the user can be notified of the area corresponding to the character string image in the document image. In addition, even when there are a plurality of character string images, the identification image is combined with the regions corresponding to the plurality of character string images. Users can be notified.

  The superimposed display unit 55A includes a second character string image generation unit 67A and a second size determination unit 68A. The second size determining unit 68A determines the size of the character string image as the size of the area corresponding to the specific area in the document image.

  The second character string image generation unit 67A generates a character string image from the translated character string input from the check unit 25A with the size determined by the second size determination unit 68A. The second character string image generation unit 67A displays the character string image superimposed on an area corresponding to the feature area of the document image. Therefore, the area corresponding to the feature area of the document image is replaced with the character string image.

  The timer unit 29 measures the time for the translation unit 23A to translate the character string input from the character recognition unit 17A for each page of image data. When one or more character regions are determined from one image data by the character region determination unit 15A, one or more feature regions are determined by the feature region determination unit 19. The translation unit 23A translates one or more target character strings respectively corresponding to the one or more feature regions determined by the feature region determination unit 19. In this case, the timing unit 29 measures the time for the translation unit 23 to translate one or more character strings in one page of image data. The timing unit 29 outputs the measured translation time to the target character string determination unit 21 and the feature region determination unit 19 every time the translation unit 23A translates all of one or more character strings corresponding to one page of image data. .

  The target character string determining unit 21 further includes a character string number limiting unit 45. The character string limit unit 45 limits the number of target character strings in accordance with the translation time input from the timer unit 29. When the translation time input from the time measuring unit 29 reaches or exceeds a predetermined value, the character string number limiting unit 45 increases the number of characters until the total number of characters of the plurality of character strings input from the character recognition unit 17A exceeds a predetermined value. Among the character strings, one or more character strings to be processed are determined. A plurality of thresholds for the translation time may be provided. In this case, a threshold value for the number of characters may be determined in advance for each of the plurality of threshold values. The character string number limiting unit 45 determines a plurality of character strings input from the character recognition unit 17A as processing targets in the order input from the character recognition unit 17A until the total number of characters reaches a predetermined value. When the total exceeds a predetermined value, the character string input from the character recognition unit 17A thereafter is discarded. Further, one or more character strings to be processed may be determined based on position information input together with the character string from the character recognition unit 17A. A plurality of character strings input from the character recognition unit 17A are determined as processing targets in order from the upper position of the document image until the total number of characters reaches a predetermined value, and the total number of characters exceeds a predetermined value. Then, the character string paired with the position information below it is discarded. Since the character string to be translated by the translation unit 23A is reduced, the translation time can be set within a predetermined time after the translation time becomes long. For this reason, it is possible to reduce the variation in time from when the document image is displayed until when the character string image is displayed.

  The feature region determination unit 19 includes a detection accuracy change unit 35. The detection accuracy changing unit 35 changes the accuracy for determining the feature region according to the translation time input from the time measuring unit 29. The detection accuracy changing unit 35 decreases the accuracy of determining the feature region when the translation time is long, and increases the accuracy of determining the feature region when the translation time is short. When the accuracy of determining the feature region is high, the time until the feature region is determined is longer than when the accuracy is low. For this reason, when the translation time is long, the time for determining the feature region is shortened, and when the translation time is short, the time for determining the feature region is long. Therefore, the character string image is displayed after the document image is displayed. It is possible to reduce fluctuations in time until display. Although the method for changing the detection accuracy is not limited, for example, the character reference determining unit 31 and the background reference determining unit 33 change the resolution of the character region portion of the image data to be processed. The detection accuracy changing unit 35 reduces the resolution of the image in the character area when the translation time is long, and increases the resolution of the image in the character area when the translation time is short.

  CPU 111 provided in MFP 100 according to the second embodiment executes the display control process shown in FIG.

  FIG. 14 is a flowchart illustrating an example of a preview display control process according to the second embodiment. Referring to FIG. 14, the difference from the preview display control process in the second embodiment shown in FIG. 6 is that steps S19 to S21 are added after step S18. Other processing is the same as the processing shown in FIG. 6, and therefore description thereof will not be repeated here.

  If CPU 111 provided in MFP 100 in the second embodiment determines in step S18 that the value of timer ETH2 is equal to or greater than threshold value ETH2, the process proceeds to step S19. In step S19, it is determined whether or not the translation time is equal to or greater than a threshold value TTTH. The translation time is a time measured in the related information display process described later, and is a time required for translating the target character string with respect to an image of one page. If the translation time is greater than or equal to threshold value TTTH, the process proceeds to step S20; otherwise, the process returns to step S11. In step S20, the character count reduction flag is set to ON, and the process proceeds to step S21. In step S21, the accuracy reduction flag is set to ON, and the process returns to step S11.

  FIG. 15 and FIG. 16 are flowcharts illustrating an example of the flow of related information display processing in the second embodiment. The related information display process in the second embodiment is a process executed in step S15 of the preview display control process in the second embodiment shown in FIG. Prior to the execution of the preview display control process, an image output by the document reading unit 130 reading the document is acquired.

  Referring to FIGS. 15 and 16, CPU 111 determines a character area in the image output by document reading unit 130 (step S <b> 41). An image output by the document reading unit 130 reading the document is set as a processing target, and one or more character regions in which characters are represented in the image set as the processing target are determined. In the next step S42, one of the one or more character areas is selected as a processing target. Then, character recognition is performed on the portion of the character area selected as the processing target in the image data (step S43). The character string obtained by character recognition is determined as the character string in the character area selected as the processing target (step S44). In step S45, it is determined whether or not the accuracy reduction flag is set to ON. If the accuracy reduction flag is set to ON, the process proceeds to step S46. If not, step S46 is skipped and the process proceeds to step S47. In step S46, the resolution of the character area portion in the image data is lowered. As a result, the image of the character area portion in the image data is reduced and the number of pixels is reduced, so that the detection accuracy of the process for determining the feature area executed in the next step S47 is reduced, and the feature area is determined. Speed up processing.

  The accuracy reduction flag is set to ON when the time for translating the previous page is equal to or greater than the threshold value TTTH. For this reason, the processing time for determining the feature region is shortened by reducing the accuracy of detecting the feature region in the next page after the page whose translation time exceeds the threshold value TTTH. As a result, the total period for reading and displaying a plurality of documents can be kept as long as possible.

  In the next step S47, the character region portion selected in step S42 in the image data, or the character region portion after the resolution is lowered when step S46 is executed, the feature region is determined. . There are cases where the characteristic area is determined from the character area portion in the image data and where the characteristic area is not determined. In the next step S48, it is determined whether or not a feature region has been determined. If the feature region is determined in step S47, the process proceeds to step S49. If not, the process proceeds to step S52.

  In step S49, the number of characters N is counted. The number of characters included in the character area selected in step S42 is added to the number of characters N. The number N of characters is a variable indicating the total number of characters to be translated in the image data output from the document reading unit 130. In the next step S50, it is determined whether or not the character number reduction flag is set to ON. If the character count reduction flag is set to ON, the process proceeds to step S51; otherwise, the process proceeds to step S52. In step S51, it is determined whether the number of characters N is equal to or less than a threshold value NTH. If the number N of characters is equal to or less than threshold value NTH, the process proceeds to step S52; otherwise, the process proceeds to step S53.

  In step S52, it is determined whether or not there is a character area not selected as a processing target in step S42 among the one or more character areas determined in step S41. If there is an unprocessed character area, the process returns to step S42; otherwise, the process proceeds to step S53.

  When the character count reduction flag is set to ON, if the character count N exceeds the threshold value NTH, an unprocessed character area is not selected as a processing target. The character number reduction flag is set to ON when the time for translating the previous page is equal to or greater than the threshold value TTTH. The number of characters N indicates the number of characters to be translated per page. For this reason, the number of characters to be translated on the next page after the page whose translation time exceeds the threshold value TTTH is limited to the threshold value NTH so that the translation processing time does not increase. As a result, the total period for reading and displaying a plurality of documents can be kept as long as possible.

  In step S53, the timer TT is reset. The timer TT measures the translation time required to translate the character string for one page image. In the next step S54, one of the character strings determined for the character area in step S44 is selected as a processing target. In the next step S55, it is determined whether or not a feature region has been determined in step S47 from the character region corresponding to the selected character string. If the feature region has been determined, the process proceeds to step S56; otherwise, the process returns to step S54.

  In step S56, the target character string is determined. Of the character strings selected in step S53, the character string included in the feature area determined in step S47 is determined as the target character string from the character area corresponding to the character string. Then, it is determined whether or not the target character string has been translated (step S57). In the display control process shown in FIG. 5, the target character string to be translated after the reading of a plurality of documents is started in step S01 is stored as a translated character string, and determined in step S56. If the same translated character string as the target character string is stored, it is determined that the character string has been translated. If translated, the process returns to step S54; otherwise, the process proceeds to step S58.

  In step S58, it is determined whether the target character string is a default character. The default character string is a predetermined character string, for example, a character indicating an item. If the target character string is a default character, the process returns to step S54; otherwise, the process proceeds to step S59. In step S59, the character string selected as the processing target in step S54 is translated. In the next step S60, a translated character string is determined. Of the translated character strings translated in step S59, the character string corresponding to the target character string is determined as the translated character string. In the next step S61, it is checked whether or not the translation is correct, and the process proceeds to step S62. It is determined whether or not the translation is correct by checking the grammar of the translated character string translated in step S59.

  In step S62, it is determined whether or not there is a character area not selected as a processing target in step S54 among the character areas determined in step S41. If there is a character area not selected as a processing target, the process returns to step S54; otherwise, the process proceeds to step S63.

  In step S63, the value of timer TT is set as the translation time. The translation time is used in step S19 of the preview display control process shown in FIG. In the next step S64, a translation character string to be processed is selected from the translation character strings determined in step S60. Then, the process branches depending on the translation result checked in step S61 for the selected translated character string. If the translation result indicates correctness, the process proceeds to step S66; otherwise, the process returns to step S64.

  In step S66, the process branches depending on the related display mode. If the related display mode is set to the parallel display mode, the process proceeds to step S67. If the related display mode is set to the superimposed display mode, the process proceeds to step S68. In step S67, the parallel display process in the second embodiment is executed, and the process proceeds to step S69. In step S68, the superimposed display process in the second embodiment is executed, and the process proceeds to step S69. In step S69, it is determined whether there is a translated character string that is not selected as a processing target. If there is a translated character string not selected as a processing target in step S64 among the translated character strings determined in step S60, the process returns to step S64. Otherwise, the process returns to the preview image display process. .

  FIG. 17 is a flowchart illustrating a part of the flow of parallel display processing according to the second embodiment. The parallel display process in the second embodiment is a process executed in step S68 of the related information display process shown in FIGS. Referring to FIG. 17, CPU 111 determines the size of the character string image in step S81. The character string image is an image generated in the next step S82. In step S12 of the preview display control process in the second embodiment shown in FIG. 14, the document image is displayed on the display unit 161, and the size of the character string image is determined based on the size of the document image. For example, when one document image is displayed on the display unit 161, a blank area other than the document image is determined on the display surface of the display unit 161, and the size of the translated character string image is determined to be equal to or smaller than the blank area. .

  In the next step S82, a character string image is generated. The translated character string image determined in step S58 of the related information display processing shown in FIGS. 15 and 16 is generated as a character string image with the size determined in step S81. In the next step S83, the generated character string images are displayed side by side at positions that do not overlap the document image. In order to indicate that the character string image corresponds to the position specified at the indicated position instructed by the user in the document image, the character string image is positioned closest to the position specified at the indicated position in the document image. It is preferable to arrange.

  In the next step S84, the identification image is combined with the document image, and the process returns to the related information display process. The feature region determined in step S47 of the related information display process shown in FIGS. 15 and 16 and corresponding to the translated character string is specified. Then, the identification image is combined with an area in the document image corresponding to the specified feature area. The identification image is a line surrounding an area corresponding to the feature area. Further, the identification image may be an image in which the area corresponding to the feature area is expressed in a single color. As a result, the identification image is synthesized with the region corresponding to the translated character string in the document image, so that the user can be notified of the region corresponding to the character string image in the document image.

  FIG. 18 is a flowchart illustrating a part of the flow of the superimposed display process according to the second embodiment. The superimposed display process in the second embodiment is a process executed in step S69 of the related information display process shown in FIGS. Referring to FIG. 18, CPU 111 determines the size of the character string image (step S81A). The character string image is a process determined in the next step S82A. The feature region determined in step S47 of the related information display process shown in FIGS. 15 and 16 and corresponding to the translated character string is specified. The size of the character string image is determined to be the same size as the area corresponding to the specified characteristic area in the document image displayed on the display unit 161.

  In the next step S82A, a character string image is generated. The translated character string image determined in step S60 of the related information display process shown in FIGS. 15 and 16 is generated as a character string image with the size determined in step S81A. In the next step S83A, the character string image is synthesized with the document image, and the process returns to the related information display process. The character string image generated in step S82A replaces the image in the region corresponding to the feature region in the document image displayed on the display unit 161. As a result, the character string image is synthesized with the area corresponding to the feature area of the document image displayed on the display unit 161, so that the user can be notified of the area corresponding to the character string image in the document image.

  FIG. 19 is a second diagram illustrating an example of a preview screen including a document image. Referring to FIG. 19, preview screen 330 includes a document image 331. The document image 331 is an image in which Japanese characters are represented, and includes four character string regions 333, 335, 337, and 339. The dotted lines in the figure are added to indicate the character string regions 333, 335, 337, and 339 and do not actually exist. The preview screen 330 is displayed for a first period after the document image 331 is displayed.

  FIG. 20 is a second diagram illustrating an example of a preview screen in which a character string image is added to a document image in the parallel display mode. A preview screen 340 shown in FIG. 20 is displayed on the display unit 161 after the document image 331 of the preview screen 330 shown in FIG. 19 is displayed. Referring to FIG. 20, preview screen 340 includes a document image 331 and four character string images 343, 345, 347, and 349. The character string image 343 is an image generated from a translated character string obtained by translating a bold character string among the character strings included in the character region 333 of the document image 331. The character string image 345 is an image generated from a translated character string obtained by translating a bold character string among the character strings included in the character region 335 of the document image 331. The character string image 347 is an image generated from a translated character string obtained by translating a bold character string among the character strings included in the character region 337 of the document image 331. The character string image 349 is an image generated from a translated character string obtained by translating a bold character string among the character strings included in the character region 339 of the document image 331.

  The document image 331 includes four identification images 333A, 335A, 337A, and 339A. The identification image 333 </ b> A is a rectangular image surrounding a bold character string among the character strings included in the character region 333. A user viewing the preview screen 310 can visually recognize the translated character string from the character string image 343. Further, it is recognized from the identification image 333A that the translated character string included in the character string image 343 is a character string obtained by translating a bold character string among the character strings included in the character region 333 in the document image 301. be able to.

  Similarly, the identification images 335A, 337A, and 339A are rectangular images that surround a bold character string among the character strings included in the character regions 335, 337, and 339, respectively. A user viewing the preview screen 310 can visually recognize the translated character string from the character string images 345, 347, and 349. Further, from the identification image 335A, the translated character string included in the character string image 345 is a character string obtained by translating a bold character string among the character strings included in the character region 335 in the document image 301, and the identification image From 337A, the translated character string included in the character string image 347 is a character string obtained by translating a bold character string among the character strings included in the character region 337 in the document image 301. From the identification image 339A, It can be recognized that the translated character string included in the column image 349 is a character string obtained by translating a bold character string among the character strings included in the character region 339 in the document image 301.

  FIG. 21 is a second diagram illustrating an example of a preview screen in which a character string image is added to a document image in the superposition mode. The preview screen 350 shown in FIG. 21 is displayed on the display unit 161 after the document image 331 of the preview screen 330 shown in FIG. 19 is displayed. Referring to FIG. 21, preview screen 350 replaces bold characters in character areas 333, 335, 337, and 339 in original image 331 of preview screen 330 shown in FIG. 19 with character string images 343 A, 345 A, and 347 A. , 349A, respectively. The character string image 343A has the same size as the area in which the bold character in the character area 333 in the document image 331 is represented, and is generated from the translated character string obtained by translating the thick character string in the character area 333 in the document image 331. It is an image. Similarly, the character string image 345A has the same size as the area in which the thick characters in the character area 335 in the document image 331 are represented, and the translated characters obtained by translating the thick character string in the character area 335 in the document image 331. This is an image generated from a column. The character string image 347A has the same size as the area in which the bold character in the character area 337 in the document image 331 is represented, and is generated from the translated character string obtained by translating the thick character string in the character area 337 in the document image 331. It is an image. The character string image 349A has the same size as the area in which the thick characters in the character area 339 in the document image 331 are represented, and is generated from the translated character string obtained by translating the thick line character string in the character area 339 in the document image 331. It is an image.

  The user viewing the preview screen 320 can visually recognize the translated character string from the character string image 343A, and the translated character string included in the character string image 343A from the position where the character string image 343A is arranged on the document image 331A It can be recognized that the character string is a translated character string in the character area 333 in the document image 331. Similarly, the translated character string can be visually recognized from the character string image 345A, and the translated character string included in the character string image 345A from the position where the character string image 345A is arranged in the document image 331A is included in the document image 331. It can be recognized that the character string is a character string obtained by translating the thick-line character string in the character region 335. The translated character string can be visually recognized from the character string image 347A, and the translated character string included in the character string image 347A from the position where the character string image 347A is arranged in the document image 331A is a character region 337 in the document image 331. It is possible to recognize that the character string is a translated character string. The translated character string can be visually recognized from the character string image 349A, and the translated character string included in the character string image 349A from the position where the character string image 349A is arranged in the document image 331A is the character region 339 in the document image 331. It is possible to recognize that the character string is a translated character string.

  As described above, MFP 100 according to the second embodiment determines a character area in image data obtained by reading a document, recognizes a character area portion in the image data, and recognizes the recognized character string. Among them, a character string portion having a predetermined characteristic in a corresponding portion in image data is determined, the character string portion of the character string portion is translated into a predetermined language, and the translated character string is associated with the image data. To display. For this reason, since the characteristic part in the document is translated and displayed, the number of characters to be translated is reduced compared with the case of translating all the character strings in the document, so that the processing time can be shortened. it can. Therefore, the character string in the document can be efficiently translated.

  In addition, MFP 100 determines a feature region having a feature different from the feature of the image in the character region. For example, MFP 100 determines an image portion having a character shape or background different from other portions in image data obtained by reading a document as a feature region. For this reason, a characteristic character string can be determined from the character strings included in the character region.

  In addition, since MFP 100 determines and translates a character string included in the feature area among the character strings recognized from the character area portion in the image data, the MFP 100 is characterized by the character string included in the document. Only a simple character string can be translated.

  Further, since MFP 100 does not determine a predetermined character string as a character string portion even if it is a character string included in the feature region, MFP 100 can limit character strings to be translated.

  Further, in the parallel display mode, MFP 100 displays an identification image indicating a feature area in the document image, so that the user can be notified of the feature area in the document image corresponding to the translated character string.

  Further, in the superimposed display mode, the MFP 100 replaces the feature area of the document image with the translated character string image, so that the translated character string image is displayed as the feature area in the document image. For this reason, the translated part can be notified to the user.

  In addition, in the superimposed display mode, the MFP 100 generates a translated character string image with a size of the translated character string image equal to or smaller than the size of the feature region. The image of the translated character string can be displayed without doing so. It is possible to maintain a format in which character strings in a document are arranged.

  In addition, the MFP 100 measures the translation time required for translation for each of a plurality of pages, and changes the accuracy of determining the feature region according to the translation time. The processing time for determination can be shortened. For this reason, it is possible to reduce the fluctuation of the period from displaying the document image of a plurality of pages to ending the display.

  Further, since MFP 100 translates the character string included in the character region including the character string portion, the character string portion is translated using character strings before and after the character string portion included in the feature region. For this reason, the precision which translates a character string part can be improved.

  Further, since MFP 100 does not translate when the character string portion is the same as the character string translated in the document of the page read previously, it is possible to reduce the translation time per page by reducing the number of character strings to be translated. it can.

  In addition, when the first period elapses after the document image is displayed before the second period elapses after the translation character string image is displayed, the MFP 100 causes the document to remain until the second period elapses. Since the image is displayed, the image of the translated character string can be displayed for at least the second period.

  In addition, MFP 100 measures the translation time required for translation by the translation unit for each of a plurality of pages, and changes the number of character string portions determined per page according to the translation time. It is possible to reduce the fluctuation of the period from the display to the end of the display.

  Further, since MFP 100 does not determine a character string that is the same as a character string to be translated even if it is a character string included in the feature area, it does not translate the character string translated in the previous page. Thereby, the translation time can be shortened.

  Further, since MFP 100 does not display an image of a character string in which the grammar of the translated character string is not normal, it is possible to prevent notification of incorrect information.

  In the above-described embodiment, the MFP 100 has been described as an example of the document reading device. However, the processing in the first embodiment illustrated in FIGS. 5 to 9 or the processing illustrated in FIGS. Needless to say, the present invention can be understood as a display control method for causing the MFP 100 to execute the above-described processing and a display control program for causing the CPU 111 that controls the MFP 100 to execute the display control method.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

<Appendix>
(1) The document reading apparatus according to claim 5, wherein the image having the second feature is an image including a character having at least one of a font and a size different from those of other characters.
(2) The document reading apparatus according to (5) or (1), wherein the image having the second feature is an image including an image having a predetermined shape as a background.
(5) The document reading according to claim 5, (1) or (2), wherein the image having the second feature is an image in which a color different from a color added to another background is added to a background. apparatus.

100 MFP, 110 Main circuit, 111 CPU, 112 Communication I / F unit, 113 ROM, 114 RAM, 115 HDD, 116 Facsimile unit, 117 External storage device, 120 Automatic document feeder, 130 Document reading unit, 140 Image forming unit , 150 paper feed unit, 155 post-processing unit, 160 operation panel, 161 display unit, 163 operation unit, 165 touch panel, 165 hard key unit, 167 hard key unit, 11 document reading control unit, 13 image display control unit, 14 display Period extension section, 15, 15A character area determination section, 17, 17A character recognition section, 19 feature area determination section, 21 target character string determination section, 23, 23A translation section, 25, 25A check section, 27, 27A related display control Part, 29 timing part, 31 character reference determining part, 33 background reference determining part, 35 detection accuracy changing part, 41 default character string exclusion unit, 43 identical character string exclusion unit, 45 character string number limiting unit, 47 front and rear reference unit, 51, 51A mistranslation non-display unit, 53, 53A parallel display unit, 55, 55A superimposed display unit, 61 , 61A character string image generation unit, 63, 63A size determination unit, 65, 65A corresponding portion instruction unit, 67, 67A character string image generation unit, 68, 68A size determination unit, 71 position detection unit.

Claims (22)

A document reading means for reading a document and outputting image data;
Image display control means for displaying each of a plurality of pages of image data output by the document reading means by continuously reading a plurality of pages of documents on the display means in order of output by the document reading means;
Position detecting means for detecting a position designated by a user in the image data of the displayed page while any one of the image data of the plurality of pages is displayed;
A character region determination unit that determines one or more character regions in which characters are represented from the image data of the displayed page in response to detection of a position by the position detection unit;
A character recognizing means for recognizing a character area portion including a position detected by the position detecting means from the one or more character areas determined in the image data of the displayed page;
Translation means for translating a character string obtained by the character recognition into a predetermined language;
A document reading apparatus comprising: a related display control unit that displays the translated character string obtained by the translation on the display unit in association with the image data of the displayed page.   The related display control means sets the translated character string at a predetermined relative position with respect to a position at which the image data of the displayed page is displayed and a size at which the image data of the displayed page is displayed. The document reading device according to claim 1, wherein the document reading device displays the image in a relatively determined size. A document reading means for reading a document and outputting image data;
Image display control means for displaying on the display means image data output by the document reading means;
A character area determining means for determining a character area representing a character from the image data output by the document reading means;
A character recognition means for recognizing characters of the determined character region in the image data;
Among character strings obtained by the character recognition, target character string determining means for determining a character string portion in which the corresponding portion in the image data has a predetermined characteristic;
A translation means for translating a character string included in the determined character string portion into a predetermined language;
A document reading apparatus comprising: a related display control unit that displays the translated character string obtained by the translation in association with the image data on the display unit. A feature region determining means for determining a feature region having a second feature different from the first feature of the character region extracted in the image data;
The document reading apparatus according to claim 3, wherein the character recognition unit recognizes a character in a portion of the image data that includes the determined characteristic region.   5. The document reading apparatus according to claim 4, wherein the characteristic region is a portion in which the shape or background of a character is different from other portions in the image data.   The image reading apparatus according to claim 4, wherein the target character string determination unit determines a character string included in the feature region of the character string obtained by the character recognition as the character string portion.   The document reading apparatus according to claim 4, wherein the target character string determination unit does not determine a predetermined character string as a character string portion even if the character string is included in the characteristic area.   The said related display control means displays the identification image which shows the determined characteristic area on the said display means in the original image which displayed the said image data on the said display means, The said display means. Document reader.   The related display control means superimposes the image of the translated character string on a portion corresponding to the feature area of the original image displayed on the display means and displays the image on the display means. The document reading device according to any one of the above.   The said related display control means produces | generates the image of the said translation character string by the size below the size of the part corresponding to the said characteristic area of the original image which displayed the said image data on the said display means. Document reader. When the document reading unit continuously reads a plurality of pages of the document, the image display control unit sequentially outputs each of the plurality of pages of image data output by the document reading unit in the order that the document reading unit outputs. Display on the display means for each first period,
The related display control means displays the translated character string on the display means in association with the image data of the corresponding page,
The image reading apparatus further includes a time measuring unit that measures a translation time required for translation by the translating unit for each of the plurality of pages.
11. The document reading apparatus according to claim 4, wherein the characteristic area determination unit changes the accuracy of determining the characteristic area according to the measured translation time. When the document reading unit continuously reads a plurality of pages of the document, the image display control unit sequentially outputs each of the plurality of pages of image data output by the document reading unit in the order that the document reading unit outputs. Display on the display means for each first period,
11. The document reading apparatus according to claim 3, wherein the related display control unit displays the translated character string on the display unit in association with image data of a page corresponding to the translated character string.   The manuscript according to claim 12, wherein the translating means does not translate when the determined character string portion is the same as the character string translated based on the image data of the page before the page of the image data in which the determined character string portion is included. Reader.   The image display control means displays the image data of the page corresponding to the translated character string before the second period elapses after the translated character string is displayed on the display means by the related display control means. When the first period elapses after being displayed on the means, image data of a page corresponding to the translated character string is displayed on the display means until the second period elapses. The document reading device according to any one of 12 and 13. The translation means further comprises a time measuring means for measuring the translation time required for translation for each of the plurality of pages,
The document reading apparatus according to claim 14, wherein the target character string determination unit changes the number of the character string portions determined per page according to the measured translation time.   The manuscript according to any one of claims 3 to 15, wherein the translation means translates the determined character string portion using character strings before and after the character string portion in addition to the determined character string portion. Reader.   17. The target character string determining means does not determine the same character string as the character string subjected to translation by the translation means as a character string portion even if the character string is included in the feature region. The document reading device according to any one of the above.   18. The document reading apparatus according to claim 1, wherein the related display control unit does not display the translated character string on the display unit when the grammar of the translated character string is not normal. A display control method executed by an original reading apparatus provided with an original reading means for reading an original and outputting image data,
An image display control step for displaying each of a plurality of pages of image data output by the document reading unit continuously reading and outputting a plurality of pages on the display unit in the order in which the document reading unit outputs;
A position detecting step of detecting a position instructed by a user in the image data of the displayed page while any of the image data of the plurality of pages is displayed;
A character region determination step for determining one or more character regions in which characters are represented from the image data of the displayed page in response to detection of a position in the position detection step;
A character recognition step for recognizing a portion of the character region including the position detected in the position detection step from the one or more character regions determined in the image data of the displayed page;
A translation step of translating a character string obtained by the character recognition into a predetermined language;
A display control method including a related display control step of displaying the translated character string obtained by the translation in association with the image data of the displayed page on the display means. A display control method executed by an original reading apparatus provided with an original reading means for reading an original and outputting image data,
An image display control step of displaying on the display means the image data output by the document reading means;
A character region determining step for determining a character region in which characters are represented from the image data output by the document reading unit;
A character recognition step for recognizing a portion of the determined character region in the image data;
Among character strings obtained by character recognition in the character recognition step, a target character string determination step for determining a character string portion whose corresponding portion in image data has a predetermined characteristic;
A translation step of translating a character string included in the determined character string portion into a predetermined language;
A display control method including a related display control step of displaying the translated character string obtained by the translation in association with the image data on the display means. A display control program that is executed by a computer that controls a document reading device including a document reading unit that reads a document and outputs image data.
An image display control step of displaying each of a plurality of pages of image data output by the document reading unit continuously reading a plurality of pages of documents on the display unit only in a first period in the order of output by the document reading unit; ,
A position detecting step of detecting a position instructed by a user in the image data of the displayed page while any of the image data of the plurality of pages is displayed;
A character region determination step for determining one or more character regions in which characters are represented from the image data of the displayed page in response to detection of a position in the position detection step;
A character recognition step for recognizing a portion of the character region including the position detected in the position detection step from the one or more character regions determined in the image data of the displayed page;
A translation step of translating a character string obtained by the character recognition into a predetermined language;
A display control program for causing the computer to execute an associated display control step of displaying the translated character string obtained by the translation in association with the image data of the displayed page on the display means. A display control program that is executed by a computer that controls a document reading device including a document reading unit that reads a document and outputs image data.
An image display control step of displaying on the display means the image data output by the document reading means;
A character region determining step for determining a character region in which characters are represented from the image data output by the document reading unit;
A character recognition step for recognizing a portion of the determined character region in the image data;
Among character strings obtained by character recognition in the character recognition step, a target character string determination step for determining a character string portion whose corresponding portion in image data has a predetermined characteristic;
A translation step of translating a character string included in the determined character string portion into a predetermined language;
A display control program for causing the computer to execute an associated display control step of displaying the translated character string obtained by the translation in association with the image data on the display means.

Images