JP2012039236A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can generate summary information in which a handwritten description on a printed portion can be recognized without reference to a printed document with the handwritten description.SOLUTION: An MFP mounted with the image processing apparatus includes: an input section 21 for inputting image data; an identification section 22 for identifying a plurality of handwritten portions from the input image data; a grouping section 23 for grouping the plurality of handwritten portions according to mutual distance; a decision section 24 for deciding a segmentation area including the grouped handwritten portions and a printed portion positioned around the grouped handwritten portions; and a generation section 25 for generating summary information about the handwritten portions, including image data in the decided segmentation area.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  In general, when proofreading a printed document or the like, the proofreading content may be written in the document by handwriting. In the original document thus obtained, a printed portion and a handwritten portion are mixed. Patent Document 1 below describes an image processing method for extracting only a handwritten part from image data obtained by reading this type of document with a scanner. In the image processing described in Patent Document 1, a difference between an image obtained by scanning an original before handwritten entry and an image obtained by scanning an original after handwritten entry is obtained. Thus, an image of only the handwritten part is extracted.

JP-A-2005-346459

  In Patent Document 1, as an example of a handwritten part, a line surrounding some type letters, a lead line connected to the surround line, and a character “more in detail” positioned at the end of the lead line are shown. In this case, only the characters “more details”, the surrounding line, and the lead line are extracted by the above image processing. Therefore, it is difficult to grasp the target to be “more detailed” from the extracted handwritten part unless the typeface surrounded by the surrounding line can be known by referring to a manuscript with handwritten entries. . For this reason, it is difficult for the user to grasp the contents of handwritten entries on the printed part, such as the instruction contents by the handwritten part and the intention to be handwritten, without referring to the printed document.

  Therefore, the present invention provides an image processing apparatus, an image processing method, and an image processing capable of generating summary information capable of grasping the content of handwritten entry for a printed portion without referring to a printed document in which handwritten entry has been made. The purpose is to provide a program.

  An image processing apparatus according to the present invention is an image processing apparatus that performs image processing on image data of a document in which a handwritten entry has been made in a printed document, and includes an input unit that inputs the image data, and the input unit. Specifying means for specifying a handwritten part from input image data; and determining means for determining a cutout region including a handwritten part specified by the specifying means and a printed part located around the handwritten part; And generating means for generating summary information of the handwritten portion including image data of the cut-out area determined by the determining means.

  In the image processing apparatus according to the present invention, a handwritten part is specified from input image data, and a cutout region including the specified handwritten part and a printed part located around the handwritten part is determined. . As a result, a cutout area including the handwritten part and the printed part highly related to the handwritten part is determined. Then, summary information of the handwritten portion including the image data of the determined cutout area is generated. By printing or displaying the summary information, the user can view the handwritten part and the printed part highly related to the handwritten part together. Therefore, the user can grasp the contents of handwritten entry on the printed portion without referring to the printed document in which the handwritten entry has been made. That is, the image processing apparatus according to the present invention makes it possible to generate summary information that can grasp the contents of handwritten entries for a printed portion without referring to a printed document that has been handwritten.

  The image processing apparatus according to the present invention includes grouping means for grouping a plurality of handwritten parts specified by the specifying means based on a distance between each other, and the determining means includes the hands grouped by the grouping means. It is preferable to determine the cut-out area including a handwriting part and a printed part located around the grouped handwriting part.

  According to this preferred configuration, a plurality of handwriting parts are grouped based on the distance to each other. Thereby, even when the related handwritten parts are divided and specified, the highly relevant handwritten parts can be grouped. Then, a cut-out area including the grouped handwriting part and a printing part located around the grouped handwriting part is determined. By printing or displaying the summary information including the cut-out area, the user can group together the handwritten parts that are highly related and the printed parts that are highly related to the handwritten parts. Can see.

  In the image processing apparatus according to the aspect of the invention, it is preferable that the determination unit determines the cutout region so that the number of pixels of the print portion included in the cutout region is equal to or greater than a predetermined value.

  According to this preferable configuration, since the number of pixels of the print portion included in the cutout area is equal to or greater than a predetermined value, a print portion of a certain level or more is included in the cutout area. Therefore, it is possible to determine a cut-out area including an amount of a print portion necessary for grasping the contents entered by handwriting.

  In the image processing apparatus according to the present invention, the determination unit includes a handwriting region setting unit that sets a handwriting region including the grouped handwriting part, and a larger region including the handwriting region. A candidate area setting means for setting as a candidate area of the area; and a determination means for determining whether or not the number of pixels of the print portion included in the candidate area is greater than or equal to a predetermined number, and is included in the candidate area by the determination means When it is determined that the number of pixels of the print portion is greater than or equal to a predetermined value, the determination unit determines the candidate region as the cutout region, and the determination unit determines the number of pixels of the print portion included in the candidate region. When it is determined that the value is not equal to or greater than the predetermined value, the candidate area setting unit preferably newly sets an area larger than the already set candidate area as a candidate area.

  According to this preferable configuration, a handwriting area including grouped handwriting parts is set, and an area including the handwriting area and larger than the handwriting area is set as a candidate area for the cutout area. Then, it is determined whether or not the number of pixels of the print part included in the candidate area is greater than or equal to a predetermined value. On the other hand, when the number of pixels of the set print portion is not a predetermined number or more, a candidate area larger than the already set candidate area is newly set. As a result, the candidate area is gradually increased, and the cutout area is determined so that the number of pixels in the included print portion is equal to or greater than a predetermined value. Therefore, it is possible to determine the cut-out area so as to include an appropriate amount of printed portion and not to make the area larger than necessary.

  In the image processing apparatus according to the present invention, when the determination unit determines that the number of pixels of the print portion included in the candidate region is not equal to or greater than the predetermined number, the number of times the candidate region is set is predetermined. When the number of times is reached, it is preferable to determine the last set candidate area as the cut-out area.

  According to this preferred configuration, when the set number of candidate areas reaches a predetermined number, the last set candidate area is determined as a cut-out area. For example, when there are few printed parts around the grouped handwritten parts, the cutout area can be prevented from becoming larger than necessary. In addition, it is possible to prevent the candidate area setting process and the pixel number determination process from being repeated redundantly.

  In the image processing apparatus according to the aspect of the invention, it is preferable that the generation unit generates the summary information including the image data of the plurality of cutout regions and the plurality of cutout regions arranged.

  According to this preferable configuration, when a plurality of cut regions are obtained, summary information in which the plurality of cut regions are arranged is generated. Therefore, even when handwritten entries are made at a plurality of locations in a printed document, summary information that allows the user to collectively view the contents entered by handwriting can be generated.

  In the image processing apparatus according to the aspect of the invention, it is preferable that the generation unit generates the summary information to which page information indicating a page including the cutout area is added in the document. It is also preferable that the generation unit generates the summary information to which in-page position information indicating a position in a page where the cutout region is located in the document is added.

  According to this preferable configuration, the summary information is printed or displayed, so that the user can view the page information and / or the in-page position information together with the image of the cutout area. Therefore, it is possible to generate summary information that allows the user to efficiently refer to the location corresponding to the cutout area in the printed document.

  In the image processing apparatus according to the present invention, it is preferable that when the cutout area exceeds a predetermined image size, the generation unit reduces the cutout area to the predetermined image size.

  According to this preferable configuration, when the cutout area is larger than the predetermined image size, the cutout area is reduced. Therefore, it is possible to make it easier to view the image of the cutout area.

  An image processing method according to the present invention is an image processing method for performing image processing on image data of a document in which a handwritten entry has been made in a printed document, the input step for inputting image data, and the input in the input step A specifying step of specifying a handwritten part from the image data obtained, and a determining step of determining a cutout region including the handwritten part specified in the specifying step and a printed part located around the handwritten part; And a generation step of generating summary information of the handwritten portion including the image data of the cut-out area determined in the determination step.

  According to the image processing method of the present invention, the summary information can be generated. That is, it is possible to generate summary information that enables grasping of the content of handwritten entry for a printed portion without referring to a printed document that has been handwritten.

  An image processing program according to the present invention includes a computer that performs image processing on image data of a manuscript that has been handwritten on a printed document, input means for inputting image data, and image data input by the input means. Determining means for specifying a handwritten part; determining means for determining a cutout area including a handwritten part specified by the specifying means and a printed part located around the handwritten part; determined by the determining means It is made to function as a production | generation means which produces | generates the summary information of the said handwritten part containing the image data of a cutting-out area | region.

  According to the image processing program of the present invention, a computer can function as the image processing apparatus. Therefore, it is possible to generate summary information that enables grasping of the contents of handwritten entries for the printed portion without referring to the printed document in which the handwritten entries are made.

  According to the present invention, it is possible to generate summary information capable of grasping the content of handwritten entry for a printed portion without referring to a printed document in which handwritten entry has been made.

1 is a block diagram showing a schematic configuration of an MFP equipped with an image processing apparatus according to an embodiment. 2 is a block diagram illustrating a configuration of a control unit included in the MFP. FIG. It is a figure for demonstrating the example (description entry example of deletion) of the determination process of a cutting-out area | region. It is a figure for demonstrating the example (example of a postscript entry) of the determination process of a cutting-out area | region. It is a figure for demonstrating the example (entry example of correction) of the determination process of a cut-out area | region. It is a figure for demonstrating the example (example of emphasis entry) of the determination process of a cutting-out area | region. It is a figure for demonstrating the example (entry example of balloon writing) of the determination process of a cutting-out area | region. It is a figure which shows the example of summary information. It is a flowchart which shows the process sequence of the production | generation process of summary information. It is a flowchart which shows the process sequence of the determination process of a cutting area. And FIG. 11 is a block diagram illustrating a configuration of a computer that executes an image processing program. It is a figure which shows the module structure of the image processing program which concerns on this embodiment. It is a figure which shows the modification of summary information.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. First, a schematic configuration of an MFP (Multifunction Peripheral) 1 equipped with an image processing apparatus according to the embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a schematic configuration of the MFP 1.

  The MFP 1 is a network multifunction device having a print function, a scan function, a FAX (facsimile) function, and an IFAX (Internet FAX) function. The MFP 1 has a function of processing image data generated by reading a document with a scan function. As image data to be subjected to image processing, there is image data of a document in which a printed portion and a handwritten portion are mixed. For example, image data of a manuscript in which proofreading contents and the like are written by hand after a document is printed is an object of image processing. The MFP 2 determines a cutout area including the handwriting part and the surrounding print part from the image data of the document in which the print part and the handwriting part are mixed, and uses this cutout area to summarize the handwriting part. Generate information.

  As illustrated in FIG. 1, the MFP 1 includes an operation unit 10, a display unit 11, and an HTTP server 12 so that a user can perform various operations on the MFP 1. The operation unit 10 includes various function keys and the like. The display unit 11 is a display device using an LCD or the like, and displays the operation state of the MFP 1, various setting contents, generated summary information, and the like. The HTTP server 12 displays various operation pages on the display of the computer 5 connected via the LAN 3 and accepts an operation from the user via the computer 5. The HTTP server 12 also has a function of displaying the generated summary information on the display of the computer 5.

  The MFP 1 includes a printer 13, a scanner 14, a FAX control unit 15, and an IFAX control unit 16. The printer 13 prints PDL data input from the computer 5, image data generated by the scanner 14, summary information, and the like on a sheet. The FAX control unit 15 controls transmission / reception of FAX via a PSTN (Public Switched Telephone Network) 7 performed by an NCU (Network Control Unit) 17 and a modem 18 provided in the MFP 1. The IFAX control unit 16 controls IFAX transmission / reception performed via the LAN I / F 19 provided in the MFP 1.

  The control unit 20 included in the MFP 1 includes a microprocessor that performs calculations, a ROM that stores programs for causing the microprocessor to execute each process, a RAM that temporarily stores various data such as calculation results, and the like. . The control unit 20 controls the hardware provided in the MFP 1 in an integrated manner by executing the stored program.

  FIG. 2 is a block diagram illustrating a configuration of the control unit 20. The control unit 20 exhibits functions as an input unit 21, a specifying unit 22, a grouping unit 23, a determination unit 24, and a generation unit 25 described below. The input unit 21 functions as an input unit described in the claims, the specifying unit 22 functions as a specifying unit, the grouping 23 functions as a grouping unit, the determining unit 24 functions as a determining unit, and a generating unit 25 functions as a generating means.

  The input unit 21 inputs image data generated by reading a document by the scanner 14. The input unit 21 may input image data from the computer 5 via the LAN 3. The specifying unit 22 specifies a handwritten part from the image data input by the input unit 21. The grouping unit 23 groups a plurality of handwritten parts specified by the specifying unit 22 based on the distance between them. The determination unit 24 determines a cut-out area including the grouped handwritten parts and the printed parts located around the grouped handwritten parts.

  With reference to FIG. 3, the specifying unit 22, the grouping unit 23, and the determination unit 24 will be described in more detail, and image processing until the cutout area is determined will be specifically described. FIG. 3 is a diagram for explaining an example of the cut-out area determination process. This example is an example in which a cut-out area is determined from image data of a document 301 in which contents indicating deletion are handwritten. It is.

  FIG. 3A illustrates an example of a document. The manuscript 301 is handwritten on a printed document. The manuscript 301 is an example in which a handwritten entry is made on a document in which a plurality of characters “printing type” is printed in five horizontal lines. In the manuscript 301, a deletion symbol 31 for deleting the type “type” is handwritten. This deletion symbol 31 is composed of two strikethroughs and overlaps the type “type”. In the manuscript 301, a plurality of characters “printing type” is a printed portion, and a deletion symbol 31 is a handwritten portion. In this example, the print portion is only the type portion, but the print portion may be a character, a symbol, a figure, or the like printed on paper using a printer or the like.

  The specifying unit 22 specifies a handwritten pixel included in the image data of the original 301. First, using one pixel or a plurality of continuous pixels included in the image data of the document 301 as a determination unit, it is determined for each determination unit whether or not it is a handwritten part. The specifying unit 22 determines whether the handwritten part is a handwritten part by utilizing the fact that the density of the handwritten part is lower than the density of the printed type. For example, the specifying unit 22 sets the pixels other than the blank portion as the determination target, and the average density of the pixels included in the determination unit, or the pixel including the pixels included in the determination unit and the pixels positioned around the determination unit. If the average density is equal to or lower than the predetermined density, the pixel included in the determination unit is specified as the handwritten part pixel. In the example shown in FIG. 3A, since the deletion symbol 31 overlaps the type “type”, among the pixels constituting the deletion symbol 31, the portion of the pixel that does not overlap the type “type” is handwritten. Identified as part.

  FIG. 3B is a diagram for explaining grouping processing by the grouping unit 23. The grouping unit 23 collects the pixels of the handwritten portion whose distances are within a certain range into one group. For example, the grouping unit 23 represents a position corresponding to a pixel in the handwriting part as “1” and a position corresponding to a pixel other than the handwriting part as “0” in the binary image. The image processing for expanding the specified area is performed. A hatched region 41 in FIG. 3B indicates a region “1” obtained by the expansion process.

  Then, as a result of performing the expansion process, the grouping unit 23 collects the pixels of the handwritten part included in the connected “1” area into one group. By this grouping process, pixels indicating continuous lines by handwriting, intersecting lines, lines or points within a predetermined distance, and the like are grouped into one group. In the example shown in FIG. 3B, the two lines constituting the deletion symbol 31 are grouped as one handwritten pixel group.

  FIG. 3C is a diagram for explaining the cut-out area determination process. The determination unit 24 includes a handwriting region setting unit 241, a candidate region setting unit 242, and a determination unit 243 in order to determine a cut-out region. The handwriting area setting unit 241 sets the handwriting area 51 including the pixels of all the handwriting parts belonging to one handwriting pixel group for each handwriting group. In the present embodiment, the handwriting area 51 is a rectangular area that circumscribes the pixels belonging to the handwriting pixel group.

  The candidate area setting unit 242 performs overlap processing after the handwriting area 51 is set, and sets the candidate area 61. The candidate area 61 includes the handwriting area 51 and is an area larger than the handwriting area 51. In the present embodiment, the candidate area setting unit 242 sets the candidate area 61 by performing overlap processing for expanding the rectangular handwriting area 51 in four directions, up, down, left, and right. For example, an area larger than a pixel included in the handwriting area 51 by a predetermined number of pixels, an area larger by a predetermined ratio with respect to the vertical and horizontal lengths of the handwriting area 51, or the area of the handwriting area 51 An area larger than a predetermined ratio with respect to can be set as the candidate area 61.

  The determination unit 243 determines whether or not the number of pixels of the print portion included in the set candidate area 61 is greater than or equal to a predetermined value. The determination unit 243 may determine whether or not the number of pixels in the print portion is equal to or greater than a predetermined number of pixels, and determines whether or not the number of pixels in the print portion is equal to or greater than a predetermined ratio of the number of pixels included in the candidate area 61. May be. Note that the determination unit 243 can use any method as a method for specifying the pixel of the print portion from the image data. For example, a pixel having a density equal to or higher than a predetermined value can be specified as a pixel in the print portion.

  When the determination unit 243 determines that the number of pixels of the print portion included in the candidate area 61 is greater than or equal to the predetermined number, the determination unit 24 determines the candidate area 61 as the cut-out area 71. When the determination unit 243 determines that the number of pixels of the print portion included in the candidate area 61 is not equal to or greater than the predetermined number, the candidate area setting unit 242 newly sets an area larger than the already set candidate area 61 as the candidate area 61. To do. FIG. 3C shows a case where a candidate area 61 is newly set.

  The candidate area setting unit 242 sets a new candidate area 61 by expanding the original candidate area 61 in four directions, up, down, left and right. For example, an area larger than a pixel included in the original candidate area 61 by a predetermined number of pixels, an area larger than the original candidate area 61 in the vertical and horizontal lengths by a predetermined ratio, or the original candidate area An area larger than the area 61 by a predetermined ratio can be set as a new candidate area 61. Thereafter, the determination unit 243 determines whether or not the number of pixels of the print portion included in the newly set candidate area 61 is greater than or equal to a predetermined value.

  The candidate area setting unit 242 gradually increases the candidate area 61 to be set until it is determined that the number of pixels of the print portion included in the candidate area 61 is equal to or greater than a predetermined value. By performing the above processing, the cutout region 71 can be set so that the number of pixels in the print portion is equal to or greater than a predetermined value.

  In the example shown in FIG. 3C, regarding the candidate area 61 set for the second time, the number of pixels of the print portion is determined to be a predetermined value or more, and the candidate area 61 set for the second time is determined as the cut-out area 71. The In the cutout area 71, a deletion symbol 31 and a type part are included. The type portion is a type “type” that overlaps with two strikethrough lines constituting the deletion symbol 31. Further, the cutout area 71 includes a blank portion around the type “type”. For this reason, according to the image of the cut-out area 71, it can be understood that the instruction to delete the type “type” is indicated by the handwriting part. Further, according to the image of the cut-out area 71, no handwritten characters are written around the type “type”, so an instruction to correct the type “type” to another type is included in the handwriting part. I can understand that there is no.

  In addition, the determination part 24 determines the candidate area | region 61 set last as a cut-out area | region, when the frequency | count of setting of the candidate area | region 61 reaches predetermined number of times. This is to prevent the process from being repeated redundantly when the grouped handwritten parts are blank and there are no printed parts or when there are few printed parts. Since the candidate area 61 is set larger each time the number of times is increased, the candidate area 61 obtained by setting the predetermined number of times includes a sufficiently large blank portion around the grouped handwritten parts. Become. Therefore, by setting the last set candidate area 61 as a cut-out area, it is understood from the cut-out area image that there are no or few print parts existing around the grouped handwritten parts. Can do.

  Subsequently, with reference to FIGS. 4 to 7, another example of the image processing until the cutout area is determined will be described. 4-7 is a figure for demonstrating the example of the determination process of a cut-out area | region. FIG. 4 shows an example in which the cutout area 72 is determined from the image data of the original 302 in which the content indicating insertion is handwritten.

  In the manuscript 302 shown in FIG. 4A, an insertion symbol 321 and an insertion character (“from”) 322 are handwritten in a printed document. The insertion symbol 321 and the insertion character 322 indicate that “from” is inserted between the type “nting” and the type “type”. The insertion symbol 321 and the insertion character 322 are arranged in the vicinity of the type “nting” and the type “type” and between the type lines.

  In the example shown in FIG. 4A, the pixels constituting the insertion symbol 321 and the insertion character 322 are specified as a handwritten part. Further, as shown in FIG. 4B, an expansion process is performed, and an area 42 in which the insertion symbol 321 and the insertion character 322 are expanded is obtained. The pixels constituting the insertion symbol 321 and the insertion character 322 are located within a certain distance from each other and are included in one region 42. Therefore, the pixels constituting the insertion symbol 321 and the insertion character 322 are grouped as one handwritten pixel group.

  Then, as shown in FIG. 4C, a rectangular circumscribed area 52 circumscribing the insertion symbol 321 and the insertion character 322 is set. Thereafter, a candidate area 62 in which the circumscribed area 52 is expanded in all directions is set. In the example of FIG. 4, it is determined that the number of pixels of the print portion included in the candidate area 62 is not a predetermined number or more, and an area larger than the already set candidate area 62 is set as a new candidate area 62. Then, it is determined that the number of pixels of the print portion included in the newly set candidate area 62 is equal to or greater than a predetermined value, and the newly set candidate area 62 is set as the cutout area 72.

  The cutout area 72 includes an insertion symbol 321, an insertion character 322, a blank portion around the insertion symbol 321 and the insertion character 322, and print portions “ting” and “type” around the insertion symbol 321 and the insertion character 322. ing. For this reason, it can be understood from the image of the cut-out area 71 that the instruction content for inserting “from” between the type “ting” and the type “type” is indicated by the handwriting part.

  FIG. 5 shows an example in which the cutout area 73 is determined from the image data of the original 303 in which the content indicating the correction is handwritten. In a manuscript 303 shown in FIG. 5A, a correction symbol 331 and a correction character (“y”) 332 are handwritten in a printed document. The correction symbol 331 and the correction character 332 indicate that the type “i” included in the type “type” is corrected to “y”. The correction symbol 331 is composed of two lines intersecting each other, and is written at a position overlapping the type “i”. The correction character 332 is entered in the margin above the type “i”.

  In the example shown in FIG. 5A, the pixels constituting the correction symbol 331 and the correction character 332 are specified as the handwritten part. Further, as shown in FIG. 5B, an extension process is performed, and an area 43 in which the correction symbol 331 and the correction character 332 are extended is obtained. The pixels constituting the correction symbol 331 and the correction character 332 are located within a certain distance from each other and are included in one region 43. Therefore, the pixels constituting the correction symbol 331 and the correction character 332 are grouped as one handwritten pixel group.

  Then, as shown in FIG. 5C, a rectangular circumscribed area 53 circumscribing the correction symbol 331 and the correction character 332 is set. Thereafter, a candidate area 63 in which the circumscribed area 53 is expanded in all directions is set. In the example of FIG. 5, it is determined that the number of pixels of the print portion included in the candidate area 63 set for the first time and the second time is not a predetermined number or more, and an area larger than the candidate area 63 set for the second time is a new candidate. The area 63 is set. Then, it is determined that the number of pixels of the print portion included in the candidate area 63 set for the third time is greater than or equal to a predetermined value, and the candidate area 63 set for the third time is set as the cut-out area 73.

  In the cutout area 73, the correction symbol 331, the correction character 332, the type “i” that overlaps the correction symbol 331, the blank portion around the correction symbol 331 and the correction character 332, and other areas around the correction symbol 331 and the correction character 332 The print part is included. For this reason, according to the image of the cut-out area 73, it can be understood that the instruction to correct the letter “i” included in the letter “type” to “y” is indicated by the handwriting part.

  FIG. 6 shows an example in which the cutout areas 74 and 75 are determined from the image data of the original 304 in which the emphasis content is handwritten. The manuscript 304 shown in FIG. 6A emphasizes the emphasis symbol 34 for emphasizing the type “type,” on the first line, and the type “type,” on the third line, on the document on which the type is printed. Is highlighted by handwriting. The emphasis symbol 34 is a line that surrounds the type “type,” in an elliptical shape. The emphasis symbol 35 is a wavy line drawn below the type “type,”.

  In the example shown in FIG. 6A, the pixels constituting the emphasis symbol 34 and the emphasis symbol 35 are specified as a handwritten portion. Next, as shown in FIG. 6B, an expansion process is performed to obtain a region 44 in which the emphasis symbol 34 is expanded and a region 45 in which the emphasis symbol 35 is expanded. The pixels constituting the emphasis symbol 34 are located within a certain distance from each other, and are included in one region 44. The pixels constituting the emphasis symbol 35 are located within a certain distance from each other and are included in one region 45. The pixels constituting the emphasis symbol 34 and the pixels constituting the emphasis symbol 34 are not included in the same regions 44 and 45 because their distances are not within a certain range. Therefore, the pixels constituting the emphasis symbol 34 are grouped as one handwritten pixel group, and the pixels constituting the emphasis symbol 35 are grouped as one handwriting pixel group.

  Then, as shown in FIG. 6C, a rectangular circumscribed area 54 circumscribing the emphasis symbol 34 and a rectangular circumscribed area 55 circumscribing the emphasis symbol 35 are set. Thereafter, a candidate area 64 in which the circumscribed area 54 is expanded in all directions is set, and a candidate area 65 in which the circumscribed area 55 is expanded in all directions is set. Regarding the emphasis symbol 34, the number of pixels of the print portion included in the candidate area 64 set for the second time is determined to be greater than or equal to a predetermined value, and the candidate area 64 set for the second time is set as the cutout area 74. For the emphasis symbol 35, the number of pixels of the print portion included in the candidate area 65 set for the third time is determined to be greater than or equal to a predetermined value, and the candidate area 65 set for the third time is set as the cutout area 75.

  In the cutout region 74, the emphasis symbol 34, the type “type,” surrounded by the emphasis symbol 34, the type “ng” located on the left side of the type “type,”, the type “p” located on the right side, A blank portion around the emphasis symbol 34 is included. For this reason, according to the image of the cutout region 74, it can be understood that the handwriting portion indicates that the type “type,” is emphasized. On the other hand, in the cutout area 75, the emphasis symbol 35, the type “type,” positioned above the emphasis symbol 35, the type “ng” positioned to the left of the type “type,”, and the type “ pr ", a blank portion around the emphasis symbol 45 is included. For this reason, according to the image of the cut-out area 75, it can be understood that the handwriting characters indicate that the type “type,” is emphasized.

  FIG. 7 shows an example in which the cutout area 76 is determined from the image data of the original 305 in which the contents indicating balloon writing are handwritten. In addition, when performing correction or additional writing, there is a case where there is not enough blank space near the target type or drawing. In this case, there is a method in which correction content or additional writing content is written in the margin, the target type or figure is surrounded by a line, and a lead line connecting the surrounding line and the portion where the content is written is entered. In this embodiment, this entry is described as balloon writing.

  In the manuscript 305 shown in FIG. 7A, “not by hand” is added to the printed text “pri” on the second line and “nting type,” on the third line. A balloon symbol 361 is handwritten. In addition, a lead line 362 and additional characters (“not by hand”) 363 are handwritten. The balloon symbol 361 is a line surrounding the type “pri” on the second line and the type “nting type” on the third line. The lead line 362 has one end located near the balloon symbol 361 and the other end located near the additional character 363. The additional character 363 is located in the lower margin of the document 305.

  In the example shown in FIG. 7A, the pixels constituting the balloon symbol 361, the leader line 362, and the postscript character 363 are specified as the handwritten part. Then, as shown in FIG. 7B, an expansion process is performed, and an area 46 in which the balloon symbol 361, the leader line 362, and the additional character 363 are expanded is obtained. The pixels constituting the balloon symbol 361, the leader line 362, and the additional writing character 363 are located within a certain distance from each other and are included in one region 46. Therefore, the pixels constituting the balloon symbol 361, the lead line 362, and the additional writing character 363 are grouped as one handwritten pixel group. Thus, even if the balloon symbol 361 and the additional character 363 are separated from each other, if there is a handwriting portion (leading line 362) connecting the balloon symbol 361 and the additional character 363, they are grouped together.

  Then, as shown in FIG. 6C, a rectangular circumscribed area 56 circumscribing the balloon symbol 361, the leader line 362, and the additional character 363 is set. Thereafter, a candidate area 66 is set in which the circumscribed area 56 is expanded in all directions. In the example of FIG. 6, the number of pixels of the print portion included in the candidate area 66 set for the first time is determined to be greater than or equal to a predetermined value, and the candidate area 66 set for the first time is set as the cutout area 76.

  The cutout area 76 includes a balloon symbol 361, a leader line 362, a postscript character 363, a type surrounded by the balloon symbol 361, a balloon symbol 361, a leader line 362, a blank portion around the postscript character 363, and a print portion. It is. For this reason, according to the image of the cut-out area 76, it is understood that the handwritten part indicates that “not by hand” is added to the type “pri” and the type “nting type,”. it can.

  Next, the generation unit 25 that generates summary information will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of summary information. FIG. 8 shows the summary information printed or displayed. FIG. 8A shows the summary information 801 for the first page, and FIG. 8B shows the summary information 802 for the second page.

  Here, a case where the above-described documents 301 to 305 are a group of documents will be described. For example, the originals 301 to 305 are sequentially fed from the original 301 to the scanner 14 by an ADF (automatic paper feeder) mounted on the MFP 1 and read, whereby page information is obtained for each image data of the originals 301 to 305. Added. That is, page information indicating the first page is added to the image data of the original 301, and similarly, page information indicating the second to fifth pages is included in each of the image data of the originals 302 to 305. Added.

  The generation unit 25 generates summary information including the cut areas 71 to 76. In the present embodiment, the summary information includes a plurality of cutout areas 71 to 76, but the cutout area may be one. The generation unit 25 reduces the cutout area 76 that exceeds the predetermined image size to a predetermined image size. The generating unit 25 refers to the page information added to the image data of each of the originals 301 to 305, and generates summary information in which a plurality of cutout areas 71 to 76 are arranged in the order of the pages of the original. In addition, when the plurality of cutout areas 74 and 75 are cut out from one image data, the generation unit 25 is sequentially arranged from the cutout area 74 above the document 304 based on the coordinates of the cutout areas 74 and 75. As shown, summary information is generated.

  Further, the generation unit 25 adds page information to the summary information. The page information is information indicating pages that include the cut areas 71 to 76 in the originals 301 to 305, respectively. Further, the generation unit 25 adds the in-page position information to the summary information. The in-page position information is information indicating the position in the page where the cutout areas 71 to 76 are respectively positioned in the originals 301 to 305. In the example shown in FIG. 8, character information 81 indicating “upper center of one page” is displayed on the cutout area 71 of the first page of the document 301. The character information 81 includes information indicating “1 page” as page information and “upper center” as in-page position information. The page information and the in-page position information are added to the summary information in association with the image data of the corresponding cutout areas 71 to 76.

  As page information included in the character information 81, page information added to image data can be used. As the in-page position information included in the character information 81, for example, the document 301 can be divided into a plurality of areas, and the area where the cutout area 71 is located can be used as the in-page position information. In the present embodiment, on the two-dimensional coordinates, the document 301 is divided into an upper stage, a middle stage, and a lower stage in the Y-axis direction, and further divided into a left side, a center, and a right side in the X-axis direction. It is divided into. Since the cutout area 71 is located in the “upper center” area, the in-page position information is “upper center”. In addition, when it is located in the whole page like the cut-out area 76, the character information 86 indicating “entire page 5” may be generated.

  Subsequently, the operation of the MFP 1 will be described, and the image processing method according to the present embodiment will be described. With reference to FIG. 9, the procedure of the summary information generation process will be described. FIG. 9 is a flowchart illustrating a processing procedure of summary information generation processing. The generation process of the summary information is performed by the control unit 20.

  First, in step S <b> 101, image data of the originals 301 to 305 read by the scanner 14 is input to the control unit 20. That is, in step S101, the input step described in the claims is performed. In the next step S102, the handwritten pixel is specified from the image data of the original 301 that is the first page. That is, in step S102, the specific step described in the claims is performed.

  In step S103, it is determined whether or not a handwritten pixel exists in the image data of the corresponding page. If it is determined in step S103 that the handwritten portion pixel does not exist in the image data of the corresponding page, the process proceeds to step S109. If it is determined in step S103 that the handwritten part of the image data of the corresponding page is present, the process proceeds to step S104. In step S104, an extension process is performed on the pixels in the handwriting part, and the pixels in the handwriting part whose mutual distance is equal to or smaller than a predetermined distance are grouped.

  Next, in step S105, a cutout area 71 is determined for each grouped handwritten pixel group. That is, in step S105, the determination step described in the claims is performed. Here, with reference to FIG. 10, the cut-out area determination process will be described. FIG. 10 is a flowchart illustrating a processing procedure of the cut-out area determination process.

  First, in step S1051, a circumscribed area 51 is set for a handwritten part included in a handwritten pixel group to be processed. Next, in step S1052, a candidate area 61 is set by performing overlap processing. In the subsequent step S1053, it is determined whether or not the number of pixels of the print portion included in the set candidate area 61 is greater than or equal to a predetermined value.

  If it is determined in step S1053 that the number of pixels of the print portion included in the candidate area 61 is not greater than or equal to the predetermined number, the process proceeds to step S1054. In step S1054, it is determined whether or not the number of overlap processes, that is, the set number of candidate areas 61 has reached a predetermined number. If it is determined in step S1054 that the number of overlap processes has not reached the predetermined number, the process returns to step S1052, the overlap process is performed again, and a new candidate area 61 is set.

  If it is determined in step S1054 that the overlap process has reached the predetermined number of times, the process proceeds to step S1055. If it is determined in step S1053 that the number of pixels of the print portion included in the candidate area 61 is greater than or equal to the predetermined number, the process proceeds to step S1055. In step S 1055, the last set candidate area 61 is determined as the cut-out area 71.

  In step S1056, it is determined whether or not the extraction region determination processing has been completed for all handwritten pixel groups of the image data to be processed. In step S1056, when there is a handwritten pixel group that has not yet been subjected to the cutting area determination process, the process returns to step S1051 to perform a process of determining a cutting area for the next handwritten pixel group. When the cut-out area determination process is completed for all the handwritten pixel groups of the image data to be processed, the cut-out area determination process ends. Then, the process proceeds to step S106 illustrated in FIG.

  In step S106, it is determined whether or not there is a cutout area whose image size exceeds a predetermined size among the cutout areas determined in step S105. If it is determined in step S106 that there is no cutout area exceeding the predetermined size, the process proceeds to step S108. If it is determined in step S106 that there is a cutout area exceeding the predetermined size, the process proceeds to step S107. In step S107, the image size of the cutout area exceeding the predetermined size is reduced to the predetermined size.

  In step S108, the image data of the cut-out area cut out from the image data of the corresponding page, page information, and in-page position information are stored. In step S109, it is determined whether or not the processing has been completed for the image data of all pages. If it is determined in step S109 that processing has not been completed for all pages, the processing returns to step S102 in order to perform processing on the image data of the next page. As a result, image processing is performed on all image data of the originals 302 to 305.

  If it is determined in step S109 that the process has been completed for all pages, the process proceeds to step S110. In step S110, summary information including image data, page information, and in-page position information of the cutout areas 71 to 76 stored for the originals 301 to 305 is generated. That is, in step S110, the generation step described in the claims is performed. Thus, the image processing according to this embodiment is completed.

  The generated summary information can be printed on a sheet by the printer 13 included in the MFP 1 or another printer. Further, the summary information can be displayed on the display unit 11 or the computer 5 provided in the MFP 1. Thereby, the user can browse the summary information shown in FIG.

  In the MFP 1 described above, the cutout areas 71 to 76 including the specified handwriting part and the printing part (printed part) located around the handwriting part are determined, and the image of the cutting area 71 to 76 is determined. Summary information of the handwriting part including data is generated. By printing or displaying the summary information, the user can view the handwritten part and the printed part highly related to the handwritten part together. Therefore, the user can grasp the contents of handwritten entry on the printed portion without referring to the printed document in which the handwritten entry has been made. That is, according to the MFP 1, it is possible to generate summary information that can grasp the contents of handwritten entry for the printed portion without referring to the printed document in which the handwritten entry has been made.

  In the MFP 1, a plurality of handwriting parts are grouped on the basis of a distance from each other, and include a grouped handwriting part and a printing part located around the grouped handwriting part. Out areas 71-76 are determined. By printing or displaying the summary information including the cut-out areas 71 to 76, the user can select a group of handwritten parts highly related to each other and a printed part highly related to these handwritten parts. You can see them all together.

  Since the cutout areas 71 to 76 include a predetermined number or more of the pixels of the print portion, a printout of a certain amount or more is included in the cutout area. Therefore, it is possible to determine a cut-out area including an appropriate amount of printed portion for grasping the contents entered by handwriting.

  In the cutting area determination process, handwriting areas 51 to 56 including grouped handwriting parts are set, and areas including these handwriting areas 51 to 56 and larger than the handwriting areas 51 to 56 are cut out. It is set as the candidate areas 61 to 66 for the outgoing area. Then, it is determined whether or not the number of pixels of the print part included in the candidate areas 61 to 66 is greater than or equal to a predetermined value. As determined. On the other hand, when the number of pixels of the set print portion is not a predetermined number or more, a candidate area larger than the already set candidate area is newly set. Thereby, the candidate areas are gradually increased, and the cutout areas 71 to 76 are determined so that the number of pixels of the included print portion is equal to or greater than a predetermined number. Therefore, it is possible to determine the cut-out area so as to include an appropriate amount of printed portion and not to make the area larger than necessary.

  In the cut-out area determination process, when the number of times that the candidate area is set reaches a predetermined number, the last set candidate area is determined as the cut-out area. For example, when there are few printed parts around the grouped handwritten parts, the cutout area can be prevented from becoming larger than necessary. In addition, it is possible to prevent the candidate area setting process and the pixel number determination process from being repeated redundantly.

  In addition, when a plurality of cut areas 71 to 76 are obtained, summary information indicating an image in which the plurality of cut areas 71 to 76 are arranged is generated. Therefore, even when handwritten entries are made at a plurality of locations in a printed document, summary information that allows the user to collectively view the contents entered by handwriting can be generated.

  By viewing this summary information, the user can quickly and visually grasp the contents of the handwritten part. For example, when handwritten parts are scattered in a document consisting of a large number of pages, the user can efficiently check the contents of only the handwritten parts by looking at the summary information. Therefore, the efficiency of the composition work can be improved by using it for the proofreading work of the printed document. Further, since the OCR process (character recognition process) is not performed on the handwritten characters when generating the summary information, it is possible to avoid the risk that the erroneously recognized characters are displayed.

  Further, since the summary information includes page information and in-page position information, the user can view the page information and / or in-page position information together with the image of the cutout area. Therefore, it is possible to generate summary information that allows the user to efficiently refer to the location corresponding to the cutout area in the printed document.

  In the MFP 1, the cutout area 76 is reduced when the cutout area 76 is larger than a predetermined image size. Therefore, since the image sizes of the plurality of cutout areas 71 to 76 are within a certain range, the images of the cutout areas 71 to 76 can be easily viewed.

  Next, a program 100 that causes the computer 90 to function as the image processing apparatus that performs the above-described summary information generation processing will be described with reference to FIGS. 11 and 12. FIG. 11 is a block diagram showing a configuration of a computer 90 for executing the program 100 according to the present embodiment. FIG. 10 is a diagram showing a module configuration of the program 100.

  The computer 90 includes a control unit (CPU) 91 that controls execution of the program 100, a hard disk 92 that stores the program 100, etc., a memory (RAM) 93, and a program recorded on a recording medium such as a CD-ROM. And the like. The computer 90 includes an input device 95, a display 96, and a LAN I / F 97.

  The program 100 may be recorded on a recording medium such as a CD-ROM, or may be provided to the computer 90 from a server or the like via a network. The program 100 includes a main module 101 that controls processing, an input module 102, a specific module 103, a grouping module 104, a determination module 105, and a generation module 106. For example, when each module of the program 100 read by the reading device 95 and stored (installed) in the hard disk 93 is executed, the computer 90 causes the input unit 21 and the specifying unit constituting the image processing device (control unit 20). 22, functions as a grouping unit 23, a determination unit 24, and a generation unit 25.

  According to the present embodiment, by executing the program 100, the computer 90 can function as an image processing apparatus that performs the above-described summary information generation process. Therefore, it is possible to generate summary information that enables grasping of the contents of handwritten entries for the printed portion without referring to the printed document in which the handwritten entries are made.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made. For example, the display example of the summary information is not limited to the above embodiment, and various additional information other than those described above may be added to the summary information. An example of this case will be described with reference to FIG. FIG. 13 is a diagram illustrating a modified example of the summary information.

  The summary information shown in FIG. 13 can be displayed on the display of the computer 5 using the browser function of the computer 5 or the like. In the example shown in FIG. 13, the image of the cutout area 71 and the image of the entire page of the document 301 including the cutout area 71 are displayed at the same time. The summary information according to this modification includes image data of the entire pages of the originals 301 to 305 including the cutout areas 71 to 76. The summary information includes information indicating the numbers of the cutout areas 71 to 76, the total number of cutout areas 71 to 76, and information indicating the total number of cutout areas for each of the originals 301 to 305.

  This summary information is displayed using the browser function of the computer 5, etc., and the cutout areas 71 to 76 can be displayed in order according to the user's operation. Simultaneously with the cutout areas 71 to 76, images of the originals 301 to 305 including the cutout areas 71 to 76 are displayed. Therefore, it becomes easy for the user to grasp the contents while viewing the image of the cut-out area and at the same time view the printed document. Simultaneously with the cutout areas 71 to 76, page information, the total number of cutout areas on the corresponding page, the total number of cutout areas of the entire document, the number of cutout areas being displayed, position information within the page, and the like are displayed. Therefore, it is possible to easily view the images of the cutout areas 71 to 76 and the originals 301 to 306.

  Further, in the above embodiment, the handwriting part pixel is specified for each determination unit based on the pixel density included in the image data of the originals 301 to 305. However, the handwriting part specifying method is not limited to this, Any method can be used. By taking the difference between the image obtained by scanning the printed document before the handwritten entry and the image obtained by scanning the printed document after the handwritten entry are made, the pixel of the handwritten portion is obtained. You may specify.

  Moreover, in the said embodiment, although the handwriting area | regions 51-56 were made into the rectangular area | region circumscribing the handwriting part, as long as it is an area | region including a handwriting part, it is not restricted to this, It does not need to circumscribe and is rectangular. Not necessarily. In the above embodiment, the candidate areas 61 to 66 are rectangular areas, but they may not be rectangular. Further, the generating unit 25 may adjust the image size and position of the cutout areas 71 to 76 in order to improve the appearance of the displayed or printed summary information.

1 MFP
DESCRIPTION OF SYMBOLS 11 Display part 12 HTTP server 13 Printer 14 Scanner 20 Control part 21 Input part 22 Identification part 23 Grouping part 24 Determination part 241 Handwriting area setting part 242 Candidate area setting part 243 Judgment part 25 Generation part

Claims (11)

An image processing apparatus that performs image processing on image data of a manuscript that has been handwritten in a printed document,
Input means for inputting the image data;
Specifying means for specifying a handwriting part from the image data input by the input means;
Determining means for determining a cutout area including a handwritten part specified by the specifying means and a printed part located around the handwritten part;
Generating means for generating summary information of the handwriting part, including image data of the cut-out area determined by the determining means;
An image processing apparatus comprising: Grouping means for grouping a plurality of handwritten parts specified by the specifying means based on a distance between each other;
The determination unit determines the cut-out area including the handwritten parts grouped by the grouping means and the printed parts located around the grouped handwritten parts. An image processing apparatus according to 1.   The image processing apparatus according to claim 2, wherein the determination unit determines the cutout region so that the number of pixels of a print portion included in the cutout region is equal to or greater than a predetermined value. The determining means includes
A handwriting area setting means for setting a handwriting area including the grouped handwriting part;
Candidate area setting means for setting a larger area including the handwriting area as a candidate area for the cut area;
Determining means for determining whether or not the number of pixels of the print portion included in the candidate area is greater than or equal to a predetermined value;
With
When the determination unit determines that the number of pixels of the print portion included in the candidate area is greater than or equal to a predetermined number, the determination unit determines the candidate area as the cut-out area;
When the determination unit determines that the number of pixels of the print portion included in the candidate area is not a predetermined number or more, the candidate area setting unit newly sets an area larger than the already set candidate area as a candidate area The image processing apparatus according to claim 3.   In the case where the determination unit determines that the number of pixels of the print portion included in the candidate area is not equal to or greater than a predetermined number, and the determination unit has reached the predetermined number of times, The image processing apparatus according to claim 4, wherein the set candidate area is determined as the cut-out area.   The said generation | occurrence | production means contains the image data of the said some cut-out area | region, The said summary information by which this some cut-out area was arranged is produced | generated, The any one of Claims 1-5 characterized by the above-mentioned. Image processing apparatus.   The image processing apparatus according to claim 1, wherein the generation unit generates the summary information to which page information indicating a page including the cutout area is added in the document. .   8. The summary information according to claim 1, wherein the generation unit generates the summary information to which in-page position information indicating a position in a page where the cutout region is located in the document is added. An image processing apparatus according to 1.   The image according to any one of claims 1 to 8, wherein when the cutout area exceeds a predetermined image size, the generation unit reduces the cutout area to the predetermined image size. Processing equipment. An image processing method for performing image processing on image data of a manuscript that has been handwritten in a printed document,
An input step for inputting image data;
A specifying step of specifying a handwriting part from the image data input in the input step;
A determination step for determining a cutout region including the handwriting part specified in the specifying step and a printed part located around the handwriting part;
A step of generating summary information of the handwriting part, including image data of the cut-out area determined in the determining step;
An image processing method comprising: A computer that performs image processing on image data of a manuscript that has been handwritten on a printed document,
Input means for inputting image data;
Specifying means for specifying a handwritten part from the image data input by the input means;
Determining means for determining a cutout area including a handwritten part specified by the specifying means and a printed part located around the handwritten part;
Generating means for generating summary information of the handwriting part, including image data of the cut-out area determined by the determining means;
An image processing program that functions as an image processing program.

Images