JP2011120174A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize a processing target cell by one operation for each of a plurality of continuous cells.
When an area of a table composed of a plurality of cells is designated, a cell contained in the table area and contained in the designated area is recognized as a processing target cell.
[Selection] FIG.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for recognizing a processing instruction written on a recording medium.

  2. Description of the Related Art Conventionally, there has been a technique for recognizing characters or the like written on a paper by analyzing a read image of a paper by a scanner using a computer. If this technology is used, for example, amounts and dates described for a large number of forms can be easily extracted, and the extracted data can be input to processing such as tabulation. However, in order to automate the above processing, it is necessary to instruct the computer which kind of data is present at which position on the paper.

  Conventionally, for example, a series of reading of an answer sheet to be processed, a processing instruction sheet in which processing instruction information for instructing processing on an entry field to be processed on the answer sheet form and contents to be written in the entry field is entered. Read by processing. There has been a technique for recognizing a writing area to be processed on a paper surface and processing contents for each writing area by detecting and analyzing the processing instruction information from the read data of the processing instruction sheet (for example, Patent Document 1). ).

JP 2008-145611 A

  However, in the above-described method, since the process is performed on the processing target area entered by the user in the processing instruction sheet, even when the same processing content is applied to another area, the user must It is necessary to fill in the processing target area and processing contents. For this reason, it has been impossible to instruct the processing target area and the processing content collectively for a plurality of continuous areas. In particular, in the table, the same items continue in the vertical direction or the horizontal direction, but because it was not possible to instruct the processing target area and the processing contents for the cells arranged continuously, the processing instruction sheet Convenience for the user to create was significantly impaired.

  The present invention has been made in view of the above problems, and when an area of a table made up of a plurality of cells is designated, an image processing apparatus capable of recognizing each cell included in the designated area as a processing target cell And an image processing method and a program.

  To achieve the above object, the present invention recognizes each of a plurality of cells included in image data as a processing target cell, and creates a processing instruction sheet for performing processing on the recognized processing target cell. In the image processing apparatus, in the image data obtained by reading the image on the document by the reading unit, there is a region recognition unit for recognizing a region surrounded by handwriting, and whether a table region including a plurality of cells exists in the image data. A determination unit for determining the presence of a table region in the image data by the determination unit, and a region surrounded by handwriting recognized by the region recognition unit is included in a cell of the determined table region A recognition means for recognizing each of a plurality of cells including a region surrounded by the handwriting as a processing target cell.

  According to the present invention, when an area of a table including a plurality of cells is designated, each cell included in the designated area can be recognized as a processing target cell.

It is a block diagram which shows the structure of an image processing apparatus. 3 is a diagram illustrating an example of a document to be processed. FIG. It is a flowchart which shows the flow of the process which produces a scan ticket. It is the figure which expanded some processing instructions. This is the image data after the color reduction processing of the instruction color. It is the image data after table analysis. It is a figure showing the position of a table | surface and a cell. It is the figure which expanded a part of instruction | indication area | region of a process instruction document. It is a figure which shows the example of a display screen of an area | region designation | designated mode selection screen. It is the figure which expanded a part of cell. It is the figure which showed the example of QR code. 6 is a flowchart illustrating a flow of processing for checking a document using a scan ticket. It is a flowchart which shows the flow of the production | generation process of a thumbnail image. It is the figure which showed the example of the thumbnail image. It is the figure which showed the example of the scan ticket.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus which is an example of an embodiment of the present invention.

  The image processing apparatus 100 according to the present embodiment is an example of a multi-function peripheral equipped with various functions such as a copy function and a scanner function. However, these functions may be realized by linking a plurality of apparatuses.

  The CPU 11 controls the overall operation of the image processing apparatus 100 by loading a program stored in the ROM 19 into the RAM 18. Further, the CPU 11 communicates with each component in the image processing apparatus 100 via the bus 12. The operation unit 16 includes a plurality of keys such as a touch panel for a user to give instructions, and a display unit that displays various types of information to be notified to the user. The scanner 15 reads an image on a document set by a user on a document table as a color image, and stores electronic data (image data) obtained thereby in the HDD 13, the RAM 18, and the like. The HDD 13 is a hard disk drive including a hard disk, and stores various types of input information. The scanner 15 includes a document feeder, and can sequentially feed and read a plurality of documents set on the document feeder on the document table. The printer 14 prints an image based on the input image data on a recording paper (sheet). The network I / F 17 connects the image processing apparatus 100 to the network 20 and controls reception of data from an external apparatus on the network and transmission of data to the external apparatus on the network.

  In this embodiment, image data used for processing to be described later will be described as an example. However, for example, image data of a document sent from an external device is transferred to the network I / F 17. It is possible to process in the same manner even if it is input via the interface. The same processing can be performed in a personal computer (PC) or the like to which a scanner or printer is connected. In this case, all or part of the program used in this embodiment can be provided to the PC via a network or the like, or stored in a storage medium such as a CD-ROM and provided to the PC.

  Next, an example of a document used in this embodiment will be described.

  FIG. 2A is a view showing an example of a document form used in the present embodiment. This manuscript is a bill in a state where the user has not entered anything (before a processing instruction sheet to be described later is added). This invoice includes the company name of the recipient, the person in charge of the recipient, subject, total amount, product name, quantity, unit, unit price, amount, bank information, subtotal, consumption tax, etc., remarks column, issue An area for pressing the person's mark is provided. The fields to which the user adds information when the invoice is officially issued include subject name, product name and quantity, unit, unit price, amount of money, creator's mark, and the like.

  In the present embodiment, it is assumed that it is checked whether information is added to a certain column designated by the user among the respective columns of the invoice.

  FIG. 2B is a diagram showing an example in which a user adds an arbitrary area for checking among the items included in the document of FIG. 2A using a color pen. The document shown in FIG. 2B becomes a processing instruction sheet.

  The processing instruction is created by a user who checks the created invoice writing processing instruction information to be described later on a sheet of the same format as the invoice to be checked. That is, the processing instruction sheet is the one in which the processing instruction information is written in the invoice shown in FIG. In the present embodiment, in order to designate a region to be processed, the user surrounds the corresponding region as a closed region such as a rectangle by handwriting with a color pen or the like.

  Here, processing instruction information (additional information) to be written on a bill (original) will be described. In FIG. 2B, for example, an area 31 is an area written by hand using a blue pen, and an area 32 and an area 33 are areas written by hand using a red pen. Note that colors other than those shown here can also be used, and the number of colors may be reduced or increased according to the check contents, not limited to two colors. Although a pen is used here, the pen is not limited to a pen as long as it can be colored.

  Then, the user registers the color information of the processing instruction information to be used in advance and the processing contents in the RAM 18 by using the operation unit 16 in association with each other. That is, it is registered in the RAM 18 to check that information is described for blue, and to check that information is numerically described for red. The CPU 11 determines the color components (for example, hue) of each color registered here, and stores the contents in the RAM 18. The color registration may be performed by causing the scanner 15 to read and register the color written on the paper instead of using the operation unit 16. Further, instead of being registered by the user, the image processing apparatus 100 may be registered in advance. When following the pre-registered content, the user adds processing instruction information to the document according to the registered color and processing content.

  In this way, the color components of the processing instruction information to be used and the processing contents corresponding to the color components are registered, and a processing instruction document according to the color components is created. Using this, processing instruction information is extracted, and the processing content is recognized according to the extraction result, whereby the image processing apparatus 100 checks whether there is information in a specific area of the document for the document to be checked. To do.

  FIG. 2C is a diagram showing an example of a document to be checked used in the present embodiment. It is assumed that the document to be checked is based on a document having the same format as that shown in FIGS. In the present embodiment, processing instruction information added as shown in FIG. 2B is extracted, and according to the extraction result, there are descriptions of characters in the region 41, numerical values in the region 42, It is determined to be normal when each cell in the table has a numerical value. FIG. 2C is an example in which all the conditions for determining normality are satisfied, so the check result of the document is normal. The check result of a document that does not satisfy the condition for determining that it is normal at one place is NG. Note that the content and area of the check are not limited to this, and other check contents and other areas can be instructed.

  Next, a process for creating a scan ticket for executing a check of the description content of a document based on a processing instruction sheet as shown in FIG. Here, the scan ticket recognizes the instruction content in FIG. 2B, and has a format (for example, QR) that allows the image processing apparatus 100 to recognize the check method of the document to be checked as shown in FIG. Code). The scan ticket includes the instruction content recognized from the document in FIG. 2B, the position information of the area to which the instruction content is applied, and the like. When checking the document to be checked, the scanner 15 reads the scan ticket and the CPU 11 recognizes the processing content, and then checks the document to be checked.

  The scan ticket creation process in the present embodiment will be described in detail. FIG. 3 is a flowchart of the scan ticket creation process in the present embodiment. 4A is an enlarged view of the processing instruction area 31, FIG. 4B is an enlarged view of the processing area 33, and FIG. 11 is an example of a QR code that encodes the processing instruction information.

  FIG. 3 is a flowchart showing the flow of processing when creating a scan ticket in this embodiment. A program for executing each process of the flowchart of FIG. 3 stored in the ROM 19 is loaded into the RAM 18 and executed by the CPU 11, whereby the flowchart of FIG. 3 is executed.

  When the user gives an instruction to create a scan ticket via the operation unit 16, this flow is started. When this flow is started, the CPU 11 causes the operation unit 16 to display a combination of an instruction color (hereinafter simply referred to as instruction color) of processing instruction information registered in the RAM 18 and a processing content (step 501). For example, “OK if there is a numerical value in the area surrounded by red”, “OK if there is a description in the area surrounded by blue”, etc. are displayed. Further, the CPU 11 causes the operation unit 16 to display a message asking the user whether the instruction color and processing content displayed in step 501 are acceptable (step 502).

  If it is determined that the user has instructed the user to negate the inquiry in step 502 via the operation unit 16, the CPU 11 causes the operation unit 16 to display that the combination of the instruction color and the processing content is to be changed. (Step 505). Here, a display for inquiring which color to change may be performed, and a new color may be presented instead of the instructed color, or the user may specify an arbitrary color by the operation unit 16. Good. Further, instead of using a new color, the combination of color and processing content may be simply changed. At this time, since the CPU 11 cannot instruct different processing contents with the same color, control is performed so that one processing content is obtained for one color. When the instruction color or the processing content, or both the instruction color and the processing content are changed in step 505, the CPU 11 causes the operation unit 16 to display step 501. Here, a display is made so that the user can confirm that the change process has been performed in step 505. If the CPU 11 determines that an instruction to affirmatively is given through the operation unit 16 in response to the inquiry in step 502, the CPU 11 determines the instruction color of the processing instruction information to be used and the processing content corresponding thereto. Register in the RAM 18.

  In step 502, the user visually confirms the contents of the document (colors included in the document), and it is determined that the color component of the instruction color is similar to the color component included in the document. In this case, they are different from each other, and an error in extracting processing instruction information is prevented.

  If it is determined in step 502 that the color component included in the document and the color component of the instruction color are similar, a monochrome copy of the document may be performed as described later. In this case, the CPU 11 causes the operation unit 16 to display a message prompting the user to set a document. When the CPU 11 determines that the document has been set by the user, the CPU 11 executes monochrome copying. This can also prevent an error in extracting processing instruction information when processing instruction information is added with a chromatic color pen. By making a determination according to the confirmation result to the user in this way, it is possible to reduce the number of times the document is read by the scanner.

  When the CPU 11 determines in step 502 that the instruction color and the processing content are OK, the CPU 11 specifies the color component used for the processing instruction information and stores it in the RAM 18. Subsequently, the CPU 11 causes the operation unit 16 to display whether or not only the document to be checked (FIG. 2C) exists in the user's hand (step 503). This is to confirm whether or not a document (FIG. 2 (a) or (b)) serving as a template when a processing instruction is created. That is, when the user only has a check target document, a document for writing processing instruction information can be created from the check target document as will be described later. The template here is not a formal document to be checked, but a template to which the user can add processing instruction information. When the CPU 11 receives a response that only the check target document exists (no template document exists) via the operation unit 16 in step 503, the CPU 11 sets the check target document in the scanner 15 in step 504. The operation unit 16 is prompted to display. Here, for example, a guidance message “Please set one of the documents to be checked on the scanner. Press the OK button after setting” and an OK button to recognize that the document has been set are displayed. Here, it is assumed that the CPU 11 recognizes that the document is placed by pressing the OK button, but the scanner uses a photo interrupter provided at the lower part of the document table, a document sensor of the document feeder, or the like. It may be possible to automatically recognize that a document is set in the document 15.

  If the CPU 11 determines in step 504 that the OK button has been pressed, the scanner 15 causes the scanner 15 to read an image on the document to be checked in step 506. Subsequently, the CPU 11 converts the image data input from the scanner 15 into monochrome image data, and outputs it to the printer 14 to output a monochrome copy to a recording sheet. In step 506, the document is converted to monochrome and printed by the printer 14, but the present invention is not limited to this. It is also possible to print 14 by converting the color of the read image of the document into another color not including the instruction color. For example, the color is converted and output, for example, the red text in the read document is changed to blue text and output. In addition, a color to be subjected to color conversion may be registered in the RAM 18 in advance, and conversion may be performed when the same color as the registered color is in the read original.

  In step 507, the CPU 11 causes the operation unit 16 to display a message prompting to write processing instruction information as shown in FIG. 2B on the recording paper output by the printer 14 in step 506. In step 503, if there is a response indicating that a template document exists, in step 508, a display asking whether or not the processing instruction information has already been described in the template is displayed on the operation unit 16 (FIG. 2B). In response to this display, when a response from the user indicating that the instruction information is not described in the template is received via the operation unit 16, the CPU 11 displays a display prompting the user to set the template in the scanner 15 in step 509. 16 to do. Here, for example, the guidance display “Please set the template in the scanner. Press the OK button after setting” and the OK button are displayed. Here, it is assumed that the CPU 11 recognizes that the document is placed by pressing the OK button, but the scanner uses a photo interrupter provided at the lower part of the document table, a document sensor of the document feeder, or the like. It may be possible to automatically recognize that a document is placed on the document 15.

  When the OK button is pressed in step 509, the process proceeds to step 510, where the CPU 11 causes the scanner 15 to read the image on the template document. In step 511, analysis / recognition processing is performed on the image data obtained in this way to determine whether the same color component color as the instruction color is included. For example, when analyzing / recognizing whether red is included, the color component analysis / recognition processing performs analysis / recognition by extracting red hue. Various known methods can be adopted for the analysis and recognition of the color components. In addition, parameters other than hue may be used, or other parameters may be combined.

  In step 512, the CPU 11 determines whether the color analyzed and recognized in step 511 includes the same color as the instruction color registered in the RAM 18. The determination as to whether the indicated color is the same as the color analyzed / recognized in step 511 can be determined not only as a perfect match but also as a certain range. For example, when the RGB value is expressed in 256 levels, it may be determined that the RGB value is compared with the RGB value of the indication color and within the range of plus or minus 20, the same. It is also possible to apply a method for determining the same color by a method other than that shown here.

  When it is determined in step 512 that the same color as the instruction color registered in the RAM 18 is included in the template image, the CPU 11 causes the operation unit 16 to display a message prompting the scanner 15 to set the template. Here, for example, the guidance display “Please set the template in the scanner. Press the OK button after setting” and the OK button are displayed. Here, the CPU 11 recognizes that a document has been placed by pressing the OK button. However, the CPU 11 uses a photo interrupter provided at the bottom of the document table, a document sensor of the document feeder, or the like. It may be automatically recognized that a document is placed on the scanner 15.

  When the OK button is pressed in step 513, the process proceeds to step 514, and the CPU 11 causes the scanner 15 to read an image on the document to be checked. The CPU 11 converts the image data input from the scanner 15 into monochrome image data, and outputs it to the printer 14 for monochrome copy output to a recording sheet. In step S514, the original is converted into monochrome and printed by the printer 14, but the present invention is not limited to this. As an alternative process, various methods can be adopted as described above (step 506).

  In step 515, the CPU 11 causes the operation unit 16 to display a message prompting to write processing instruction information as shown in FIG. 2B on the recording paper output by the printer 14 in step 514.

  If it is determined in step 512 that the same color as the instruction color registered in the RAM 18 is not included in the template image, in step 516, the CPU 11 adds processing instruction information as shown in FIG. The operation unit 16 is caused to display to prompt writing.

  In step 508, the CPU 11 causes the operation unit 16 to display whether or not the processing instruction information has already been described in the template. On the other hand, when it is determined that a response indicating that the instruction information is already written in the template is received via the operation unit 16, the process proceeds to step 517, and the original image of the instruction described template is scanned by the scanner 15 (reading). Means). Here, the original is read in the same procedure as the monochrome copy output described above. In other words, the operation unit 16 is displayed on the operation unit 16 to prompt the user to set a document on which instruction information has been written. On the other hand, when the user presses the OK button after setting the document, the scanner 15 reads the document. . However, conversion of image data obtained by reading with the scanner 15 into monochrome image data is not performed here. The image data obtained here is stored in the RAM 18.

  Next, in step 518, processing instruction information is analyzed and recognized from the image data input from the scanner 15. That is, in step 518, area recognition is performed for recognizing the area surrounded by handwriting in the image data input from the scanner 15. Here, first, the position of the target region is specified for each color by analyzing where the instruction color determined in step 502 is in the document and recognizing the color of that portion. The position specified here can determine which size of the region to be processed exists at which position on the document. For example, the position is specified by coordinates.

  FIG. 4A shows an area written by the checker with a blue pen, which is the processing instruction area 31 in FIG. The processing instruction area 31 is recognized as a closed area in step 518, and the start coordinate 401, the width 402, and the height 403 at the upper left end are extracted as check area information. When the position of the XY coordinate is represented as (X, Y), the start coordinate 401 is a coordinate (840, 2010), the width 402 is 660, and the height 403 is 100.

  FIG. 4B shows an area written by the checker with a red pen, which is the processing instruction area 33 in FIG. The processing instruction area 33 is recognized as a closed area in step 518, and the start coordinates 411 at the upper left corner, the width 412, and the height 413 are extracted as check area information. When the position of the XY coordinate is expressed as (X, Y), the start coordinate 411 is a coordinate (1585, 845), the width 412 is 260, and the height 413 is 120.

  Similarly, the processing instruction area 32 written by the checker with a red pen has a start point (840, 1910), a width of 560, and a height of 100. Further, from these process instruction areas 31, 32, and 33, an instruction color is determined and a process code is extracted. Since the process instruction area 31 is an area written with a blue pen, the process code is 1. Similarly, since the process instruction area 32 is an area written with a red pen, the process code is 0, and since the process instruction area 33 is an area written with a blue pen, the process code is 1.

  The position specified here is stored in the RAM 18 in association with the processing content determined in step 502.

  Subsequently, in step 519, the CPU 11 performs color reduction processing of the same color component as the instruction color on the position of the instruction processing information analyzed and recognized in step 518 for the image data input from the scanner 15 in step 517. I do. FIG. 5 shows image data after the color reduction processing of the instruction color is performed in step 519. By the color reduction processing in step 519, image data from which the area written by the user is deleted, that is, image data similar to the original form is created. The image data obtained here is stored in the RAM 18. Note that the method of obtaining image data similar to that of a document form is not limited to this. For example, the document form can be read again.

  In step 520, table analysis is performed on the image data obtained in step 519. Here, first, all ruled lines are extracted from the image data obtained in step 519, and an object constituted by a set of ruled lines is recognized as a table. FIG. 6 shows the image data after performing the table analysis in step 520. Here, there are three tables, a table 601, a table 602, and a table 603. Further, the position of the table and the position of each cell in the table are specified. For example, the position is specified by coordinates. FIG. 7 shows the position information of the table 602. In the table 602, a start coordinate 701 at the upper left corner, a width 702, and a height 703 are extracted as a table region. When the position of the XY coordinate is expressed as (X, Y), the start coordinate 701 is a coordinate (230, 1520), the width 702 is 1620, and the height 703 is 970. Also, from the start coordinate 701, the width 702, and the height 703, the coordinates of the upper right edge 704 of the table are (1850, 1520), the coordinates of the lower right edge 705 are (1850, 550), and the coordinates of the lower left edge 706 are (230, 550). Also, for each cell in the table 602, the start coordinates, width, and height of the upper left corner are extracted. For example, the cell 710 in the table 602 has a start coordinate 711 at the upper left end of (1580, 850), a width 712 of 265, and a height 713 of 60, and the cell 720 has an upper left start coordinate 721 of ( 1580, 790), the width 722 is 265, and the height 723 is 60. In the cell 730, the start coordinate 731 at the upper left corner is (1580, 730), the width 732 is 265, and the height 733 is 60.

  In step 521, the CPU 11 determines the result of the table analysis analyzed in step 520. If the table does not exist in the image data, the process proceeds to step 528. If the table exists, the process proceeds to step 522.

  In step 522, it is determined whether or not the processing target area is in the table area. Here, it is determined from the coordinates of the processing instruction area obtained in step 518 and the coordinates of the table obtained in step 520 whether the processing instruction area exists in the area of the table. FIG. 8 shows a table 602 and a processing instruction area 33 in the image data read by the scanner 15. The start coordinate 411 at the upper left end of the processing instruction area 33 is (1585, 845), the width 412 is 260, and the height 413 is 160. Therefore, the upper right end coordinates, lower right end coordinates, and lower left end coordinates of the processing instruction area 33 can be calculated as (1845, 845), (1845, 725), and (1585, 725), respectively. Since these coordinates are all included in the area surrounded by the upper left corner coordinates 701, the upper right edge coordinates 704, the lower right corner coordinates 705, and the lower left corner coordinates 706 of the table 602, the processing instruction area 33 exists in the area of the table 602. Can be determined. If it is determined in step 522 that the process instruction area exists in the table area, the process proceeds to step 523, and if it is determined that the process instruction area does not exist in the table area, the process proceeds to step 528.

  In step 523, for the processing instruction area determined to be present in the table area in step 522, the processing is performed in the rectangular designation mode in which the designated area is the processing instruction area, or each cell included in the processing instruction area is instructed to be processed. A display for confirming the cell designation mode as the area is displayed on the operation unit 16. FIG. 9 shows an example in which an area designation mode selection screen for the processing instruction area existing on the table area is displayed on the operation unit 16. If the mode selected by the user on the area designation mode selection screen is the rectangle designation mode, the CPU 11 proceeds to step 528. If the mode is the cell designation mode, the CPU 11 proceeds to step 524.

  In step 524, it is determined whether or not a cell including a part or all of the user specified area is set as the user specified area. In step 524, it is determined whether the user specified area is included in the table area cell. Here, it is assumed that whether or not the user specified area is used is determined based on whether or not the center point of the cell is included in the user specified area. FIG. 10 is an enlarged view of the cell 710, the cell 720, and the cell 730 including the processing region 33. Point 101, point 102, and point 103 in FIG. 10 are the center points of cell 710, cell 720, and cell 730, respectively. The CPU 11 first determines whether or not the point 101 that is the center of the cell 710 is included in the processing region 33. If the center point of the cell is included in the processing region, the CPU 11 proceeds to step 525. If not included, the process proceeds to step 526.

  In step 525, the CPU 11 recognizes the cell 710 as a processing area designated by the user. In step 526, the cell is recognized as an area not designated by the user. In step 527, the CPU 11 determines whether or not the processing in step 524 has been performed on all the cells including the processing area 33. If the processing of all the cells has been completed, the process proceeds to step 528, where unprocessed cells are processed. If there is, go to step 524.

  Here, since the processing of the cell 720 and the cell 730 has not ended, the CPU 11 proceeds to step 524 and performs the same processing as the cell 710 on the cell 720 and the cell 730. Since the point 102 of the cell 720 is included in the processing region 33 and the point 103 of the cell 730 is not included in the processing region 33, the CPU 11 recognizes the cell 710 and the cell 720 as processing target cells.

  Here, whether or not the specified process is applied to the cell is determined based on whether or not the center point of the cell is included in the processing area. For example, the determination is made based on the distance between the cell border and the processing area border. It is not limited to this method.

  As described above, in this embodiment, the user can select the cell 710 and the cell 720 by selecting the processing region 33 extending over a plurality of cells. In the prior art, in order to select the cell 710 and the cell 720, it is necessary to surround each cell as a processing region. However, in this embodiment, when the cell designation mode is selected, a plurality of cells can be selected by designating one processing area, and the load of selection by the user is reduced.

  Subsequently, in step 528, the CPU 11 displays the result of analysis / recognition in step 518 and the processing target area recognized in steps 519 to 527 on the operation unit 16. For example, the coordinates of the region corresponding to the specified processing instruction information and the processing content for that region are displayed. Alternatively, a thumbnail image of the read document may be displayed, and in correspondence with the image, the processing instruction information may be displayed at which position, and the processing content may be displayed in an identifiable manner. In step 529, the CPU 11 causes the operation unit 16 to display a message for confirming whether the content displayed in step 528 is correct. If a negative response is received from the user via the operation unit 16 in response to this confirmation, the process proceeds to step 534 where the CPU 11 confirms whether the template image read by the scanner 15 in step 517 is to be monochrome output by the printer 14. To the operation unit 16.

  If an affirmative response is received via the operation unit 16, the process proceeds to step 535, where the image of the original read by the scanner 15 is converted into monochrome image data in step 517, and the monochrome copy output is performed by the printer 14. Make it.

  That is, when the processing instruction information cannot be correctly extracted, the processing instruction document with the processing instruction information added is monochrome copied. Using this, processing instruction information is added again. In step 535, the document is converted to monochrome and printed by the printer 14, but the present invention is not limited to this. As an alternative process, various methods can be adopted as described above (step 506).

  In step 536, the CPU 11 causes the operation unit 16 to display to prompt the processing instruction information to be written on the recording paper output by the printer 14 in step 535.

  In step 537, when an instruction indicating that monochrome copy output is not performed is received from the user in step 534 via the operation unit 16, the CPU 11 displays a display for confirming whether to newly create a processing instruction sheet. Let me do it. In response to this confirmation, when the CPU 11 receives an instruction indicating that a new processing instruction is to be created via the operation unit 16, the process proceeds to step 538, where the CPU 11 sets the newly created processing instruction on the scanner. The operation unit 16 is caused to display to prompt the user to do so. On the other hand, in response to the confirmation in step 537, when the CPU 11 receives an instruction indicating that a new processing instruction sheet is not to be created via the operation unit 16, the CPU 11 ends this processing.

  Subsequent to the display of step 536 and step 538, when a reading instruction such as the user setting a document and pressing the OK button via the operation unit 16 is performed, the processing of step 517 described above is executed again.

  When a response indicating that the analysis result is correct in step 529 is received via the operation unit 16, the analysis content is stored in the RAM 18 as the extraction result of the processing instruction information. In step 530, the CPU 11 causes the operation unit 16 to display whether to create a scan ticket. If an affirmative response to this display is received via the operation unit 16, the process proceeds to step 531 and the CPU 11 encodes the analysis content. The analysis content encoding means that the analysis result displayed in step 528 is encoded using, for example, a two-dimensional code (for example, a QR code). The contents to be encoded are the area instructed to be processed and the processing contents for that area.

For example, in the case of the processing instruction sheet shown in FIG. 2C, characters in which the processing instruction information in the processing instruction areas 31 to 33 are arranged in the order of the start point X coordinate, Y coordinate, width, height, and processing code, separated by commas. The columns are as follows, and the character string encoded in the QR code is shown in FIG.
"840, 2010, 660, 100, 1, 840, 1910, 560, 100, 0, 1580, 850, 265, 60, 1, 1580, 790, 265, 60, 1"
Although a two-dimensional code is described here as an example, it may be coded by other methods, and is not limited to this as long as the image processing apparatus 100 can analyze and recognize. In step 532, the CPU 11 outputs the coded image created in step 531 as an image on the recording paper by the printer 14 and prints it.

  It is possible to check the document to be checked using the printed scan ticket. However, if it is determined in step 529 that the analysis result is correct, this indicates that the processing instruction read by the scanner 15 in step 517 has been correctly recognized, so that the processing in steps 530 to 532 is not performed. The instruction sheet may be a scan ticket. In this case, the processing contents and the like are recognized from the processing instruction sheet at the time of checking.

  In addition, when a negative response is received via the operation unit 16 in response to the inquiry in step 530, the CPU 11 causes the operation unit 16 to display an ID for specifying the analysis content registered in step 529. This is because the analysis content is specified and read from the RAM 18 for use when checking the check document. The ID may be specified by the user from the operation unit 16 other than the CPU 11 presenting the ID. The ID determined here and the analysis contents are associated with each other and stored in the RAM 18. Thereafter, the process proceeds to step 533.

  In step 533, the document to be checked is checked according to the processing instruction information recognized as described above and the corresponding processing content.

  With the above processing, it is possible to apply the same processing content to a plurality of adjacent cells in the table at once. Note that it is not necessary to execute all the processes shown in the flowchart described above, and only a part of the processes may be executed.

  Next, a procedure (step 533) for checking a document according to the extracted processing instruction information using the scan ticket created as described above will be described with reference to FIG.

  FIG. 12 is a flowchart showing the overall processing flow of the image processing apparatus according to the present embodiment. Hereinafter, the check processing operation according to the present embodiment will be described with reference to the flowchart. A program for executing each process of the flowchart of FIG. 12 stored in the ROM 19 is loaded into the RAM 18 and executed by the CPU 11, whereby the flowchart of FIG. 12 is executed.

  First, in step 2501, the CPU 11 displays on the operation unit 16 whether or not to create a scan ticket, which will be described later.

  Next, in step 2502, the CPU 11 that has received an instruction to create a scan ticket from the operation unit 16 proceeds to step 2503. When the CPU 11 receives an instruction from the operation unit 16 not to create a scan ticket because the checker already has a scan ticket or the like, step 2504 is performed without performing the scan ticket creation process. Proceed to

  In step 2503, the CPU 11 performs a scan ticket creation process for creating a scan ticket from the processing instruction sheet described above, and proceeds to step 2504. The details of the scan ticket and the scan ticket creation process are as described above.

  Next, in step 2504, the CPU 11 displays on the operation unit 16 that the scan ticket is scanned on the first sheet and the second and subsequent sheets are scanned, and prompts the user to set the document.

  In step 2505, when the CPU 11 detects that a document is set by a sensor (not shown), the CPU 11 sends a scan ticket and a document to be scanned to the scanner 15 and sends an instruction to store image data in the HDD 13. In this embodiment, in order to simplify the description, the case where only the document to be checked is FIG. 2C will be described, but there may be a plurality of documents to be checked.

  Next, in step 2506, the CPU 11 reads the scan ticket image data stored in the HDD 13 and analyzes the scan ticket. A plurality of processing instruction information is encoded in a QR code and printed on the scan ticket. Each processing instruction information includes check area information indicating which area is to be checked, and a processing code indicating which processing method is used to check the check area. The CPU 11 detects the position of the QR code included in the image data of the scan ticket, decodes the QR code, and acquires the plurality of processing instruction information. The check area information of the processing instruction information in this embodiment represents coordinates on the image data, and is composed of the XY coordinates of the start point of the process area, and the width and height from the start point. The processing code represents a processing method for checking the check area, and is a number associated with the processing method for the check area.

  In the present embodiment, two types of processing methods for checking that a numerical value is described in the check area (processing code: 0) and that a character is described (processing code: 1) are associated with the processing code. ing. If a plurality of documents to be checked are read in step 2505, the processing instruction information described in the scan ticket placed on the first sheet is applied to all the documents to be checked after the second sheet. Is done. The processing instruction information is valid until the check process is completed.

  Next, in step 2507, the CPU 11 sequentially reads the check target image data stored in the HDD 13.

  Next, in step 2508, the CPU 11 selects one from the plurality of processing instruction information. Then, check areas are sequentially acquired from the image data read in step 2507 using the start point, width, and height indicated by the check area information of the processing instruction information.

  Next, in step 2509, the CPU 11 confirms the processing code for the check area from the processing instruction information used in step 2508.

  In step 2510, the CPU 11 performs a process of checking whether there is a description in the acquired check area. In this embodiment, the image data in the check area acquired in step 2508 is converted into the HLS color space, and the check area is described according to the ratio of the luminance L that is darker than the predetermined brightness in the check area. It can be determined whether or not. In the present embodiment, the image data of the check area acquired from the image data is an RGB color space. Therefore, the ratio of the pixels whose luminance L is smaller than 50% to the check area is obtained, and when the ratio of the pixels is 10% or more, it is determined that the check area is described. Note that the determination method is not limited to this, and it is only necessary to be able to determine whether there is a description. For example, it is possible to determine whether there is a description based on the length of the dark pixels in the main scanning direction. good. Further, since the conversion from the RGB color space used here to the HLS color space is a known technique, a detailed description thereof is omitted here.

  In step 2511, the CPU 11 determines whether or not the ratio obtained in step 2510 is smaller than a certain value, and proceeds to step 2512 if the condition is met. If the condition is not met, the process proceeds to step 2516.

  In step 2512, if the processing code confirmed in step 2509 is 0, the process proceeds to step 2515, and if the processing code is 1, the process proceeds to step 2513.

  In step 2513, the CPU 11 performs processing to check whether there is a numerical value in the acquired check area. In this embodiment, OCR processing is performed in the check area acquired in step 2508, and it is confirmed whether or not the information described in the area is only a numerical value. Further, since the OCR processing used here is a known technique, detailed description thereof is omitted here.

  If it is determined in step 2514 that only the numerical value is described as a result of the determination of the description content in step 2513, the CPU 11 proceeds to step 2515. If it is determined that the described information is not only numeric values, the process proceeds to step 2516.

  Next, in step 2515, the CPU 11 stores in the RAM 18 which is a work memory that the check result of the check area acquired in step 2508 is OK.

  In step 2516, the CPU 11 stores in the RAM 18 that the check result of the check area acquired in step 2508 is NG.

  Next, in step 2517, the CPU 11 determines whether or not all of a plurality of processing instruction information has been checked for the image data read in step 2507. On the other hand, if it is not finished, the process proceeds to Step 2508.

  Next, in step 2518, the CPU 11 determines whether or not the processing has been completed for all the image data of the document to be checked read in step 2505, and proceeds to step 2518 if it has been completed. On the other hand, if it is not finished, the process proceeds to Step 2507.

  Next, in step 2519, when the CPU 11 finishes processing all the check areas for the image data of all the documents to be checked, the CPU 11 transmits the check results and the display result to the operation unit 16. Then, the check process ends. Here, the check result displayed on the operation unit 16 is the document to be checked read in step 2505, and if at least one check area is NG, it is notified that the check processing in this embodiment is NG. To do. The display contents and method of the check result are not limited to this. For example, more detailed information such as which check area of which document is NG may be displayed. It goes without saying that a report or the like may be output by generating image data from the check result and transmitting it to the printer engine 14.

  Next, the thumbnail image generation process will be described in detail.

  FIG. 13 is a flowchart of the thumbnail image generation processing in the present embodiment in step 528 of FIG. 3, and FIG. 14 is an example of the thumbnail image in the present embodiment.

  First, in step 1301, the CPU 11 sequentially deletes processing instruction rectangles drawn with a color pen by a checker from the image data of the processing instruction sheet. Specifically, using the processing instruction information acquired in step 518, an area on the image data corresponding to the drawn processing instruction rectangle is specified, and the pixel of the instruction color included in the area is processed. By substituting with the background color, the processing instruction rectangle is deleted from the image data.

  Next, in step 902, the CPU 11 uses the processing instruction information acquired in steps 518 to 527 to create a rectangle composed of straight lines in the area of the image data from which the drawn processing instruction rectangle has been deleted. Draw in the indicated color. The drawing position of the rectangle can be obtained from the start point, width, and height information included in the check area information of the processing instruction information or the start point, width, and height information of each cell in the table. In this embodiment, since the instruction color is set in a certain range, the color for drawing a rectangle is 50% for the luminance L of the instruction color, 100% for saturation S, and for hue H. Use the color located in the center of the range. For example, in the case of the processing instruction sheet of FIG. 2B, the rectangle drawn by the checker in the processing instruction area 31 is deleted, and a rectangle 1401 is drawn with the instruction color of the red color pen according to the processing instruction information. Similarly, the rectangle drawn by the checker in the processing instruction area 32 is also deleted, and the rectangle 1402 is drawn with the instruction color of the blue color pen in accordance with the processing instruction information. In addition, since the rectangle drawn by the checker in the processing instruction area 33 is deleted and the cell designation mode is selected by the user in the processing instruction area 33, the rectangle is displayed in the instruction color of the blue color pen according to the processing instruction information. 1403 and a rectangle 1404 are drawn.

  Next, in step 1303, the CPU 11 determines whether or not the above processing has been completed for all processing instruction information acquired in steps 518 to 527, and proceeds to step 1304 if it has been completed. If it has not been completed, the process proceeds to step 1301.

  Next, in step 1304, the CPU 11 generates a thumbnail image 1400 by reducing the image data obtained through the processing up to step 1303. First, the CPU 11 generates a thumbnail image for display on the operation unit that has been reduced to a size for display on the operation unit 16 in step 528. Further, the image data from which the processing instruction rectangle obtained in the processing up to step 1303 is deleted is reduced to a size for generating a scan ticket in step 531, and a thumbnail image for generating a scan ticket is obtained. Is generated. In the present embodiment, the bicubic method is used for the reduction process, but the present invention is not limited to this, and it goes without saying that a technique such as the two-arrest neighbor method or the bicubic method may be used.

  Next, the scan ticket generated by the scan ticket generation process in step 532 will be described in detail with reference to FIG.

  FIG. 15 is an example of a scan ticket generated from the processing instruction sheet of FIG.

  The scan ticket includes the QR code 111 of FIG. 11 in which processing instruction information is encoded, a thumbnail image 1500, and processing instruction information 1501-1504. The processing instruction information 1501 includes check area information acquired from the processing instruction area 31 (coordinates, width and height of the start point of the check area), a processing method corresponding to the processing code, and whether the instruction processing area is a rectangle designation mode or a cell. The selection mode indicating the designated mode is written in characters. The processing instruction information 1502 includes the processing method corresponding to the check area information and processing code acquired from the processing instruction area 32, and the processing instruction information 1503 and 1504 includes the check area information and processing acquired from the processing instruction area 33. The processing method corresponding to the code is written in characters. The thumbnail image 1500 is a thumbnail image generated from the processing instruction sheet in FIG. 2B obtained by the above-described thumbnail image generation processing. As described above, the thumbnail image 1500 is obtained by deleting the processing instruction rectangle drawn on the processing instruction by the checker and reducing the image data in which the rectangle is drawn in the instruction color according to the processing instruction information for the scan ticket. It is.

  As described above, according to the present embodiment, it is possible to apply the same processing contents to the continuous cells in the table in a lump, improving the convenience of creating the processing instruction by the user. It becomes possible.

  In this embodiment, the process instruction information is converted into a QR code in the scan ticket creation process. However, the method for encoding the process instruction information is not limited to this. For example, the process instruction information is encoded using a color barcode or the like. You may do it. In this case, it is needless to say that the processing instruction information is obtained not by decoding the QR code in step 2506 in FIG. 12 but by performing decoding corresponding to the encoding method.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (8)

In an image processing apparatus that recognizes a plurality of cells included in image data as a processing target cell and creates a processing instruction for processing the recognized processing target cell.
Area recognition means for recognizing an area surrounded by handwriting in image data obtained by reading an image on a document by a reading means;
A discriminating means for discriminating whether or not a table region composed of a plurality of cells exists in the image data;
When it is determined by the determining means that a table area exists in the image data, and a cell surrounded by the handwriting recognized by the area recognition means is included in a cell of the determined table area, the table area is surrounded by the handwriting. An image processing apparatus comprising: recognition means for recognizing each of a plurality of cells including a specified area as a processing target cell.   The image processing apparatus according to claim 1, wherein the recognition unit recognizes the cell as a processing target cell when a center point of the cell is included in the region surrounded by the handwriting.   The image processing apparatus according to claim 1, further comprising a deletion unit that deletes an area surrounded by the handwriting recognized by the area recognition unit from the image data.   In the recognizing means, it is determined by the determining means that a table area composed of a plurality of cells exists in the image data, and the cells of the determined table area are surrounded by handwriting recognized by the area recognizing means. The image processing apparatus according to claim 1, wherein when the area is not included, the cell is recognized as not being a processing target cell.   The image processing apparatus according to claim 1, further comprising an output unit that outputs data including a cell recognized as a processing target cell by the recognition unit to a printing unit.   The image processing apparatus according to claim 5, wherein the output unit outputs data including processing instruction information to the printing unit in addition to the recognized cell. In an image processing method for recognizing a plurality of cells included in image data as a processing target cell and creating a processing instruction for processing the recognized processing target cell,
An area recognition step for recognizing an area surrounded by handwriting in image data obtained by reading an image on a document by a reading means;
A determination step of determining whether or not a table region composed of a plurality of cells exists in the image data;
If it is determined by the determination step that a table area exists in the image data, and a region surrounded by handwriting recognized by the area recognition step is included in a cell of the determined table area, the table area is surrounded by the handwriting. A recognition step of recognizing each of a plurality of cells including the region as a processing target cell.   A program that causes a computer to execute the image processing method according to claim 7.

Images