JP2019114024A - Device, method and program for setting information related to scan image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately detect a region for performing character recognition processing based on a position designated by a user on a screen. An apparatus displays a preview screen of a scan image represented by scan image data. A character string area that is presumed to be a character string in the scanned image is extracted, and at least a part of the extracted character string area is selected in the selected area based on the first coordinate and the second coordinate specified in the preview screen. The character string area where is overlapped is detected as the target area. Information related to the scan image data is set using the character obtained as a result of the character recognition processing of the target area. [Selection diagram] Fig. 10

Description

  The present invention relates to a technique for setting information related to a scanned image obtained by scanning.

  Conventionally, there is a technology in which character recognition processing is performed on image data obtained by scanning a paper document (hereinafter referred to as scan image data), and the recognized characters are used as file names of electronic files of the paper document. is there.

  In Patent Document 1, a rectangular area is designated by a swipe operation or a drag operation with a finger on an operation panel for displaying scanned image data, and a file obtained by character recognition processing of the area is used as a file name. It is disclosed to create. Furthermore, Patent Document 1 also discloses a technique of defining another area at a position shifted by a predetermined amount from the position of the area designated by the finger, and performing the character recognition process also on the determined other area.

JP, 2015-215878, A

  When the user is made to select the area of the character string on the screen, the area selected by the user may not match the area of the character string in the scanned image. This tendency is strong in a mode in which the user selects a character string area on a small screen such as an operation panel. In Patent Document 1, character recognition processing is performed on an area defined at a position shifted by a predetermined amount from the position of a rectangular area designated by the user with a finger. However, the position shifted by the predetermined amount does not necessarily coincide with the area of the character string desired by the user. For this reason, the character recognition process is performed on the area shifted a predetermined amount a plurality of times, and the load on the character recognition process tends to be large.

  An object of the present invention is to appropriately detect a region where character recognition processing is to be performed based on a position designated by a user on a screen.

  An apparatus according to an aspect of the present invention is an apparatus for setting information related to scanned image data obtained by scanning a document, and a display for displaying a preview screen of a scanned image represented by the scanned image data. The control means, the extraction means for extracting a character string area assumed to be a character string in the scan image, and the first coordinates and the second of the extracted character string areas designated in the preview screen Using detection means for detecting as an object area a character string area at least partially overlapping a selection area based on coordinates, recognition means for performing character recognition processing of the object area, and characters obtained as a result of the character recognition process And setting means for setting the information.

  According to the present invention, it is possible to appropriately detect a target area on which character recognition processing is to be performed, based on the position designated by the user on the screen.

FIG. It is a hardware block diagram of MFP. It is a hardware block diagram of a file server. It is a software block diagram of MFP. It is a flowchart which shows a series of processes to an upload. FIG. 6 is a view showing a scan setting screen of the MFP. FIG. 6 is a view showing a preview screen of the MFP. FIG. 6 is a view showing a preview screen of the MFP. FIG. 6 is a view showing an upload setting screen of the MFP. It is a flow chart which shows file name generation processing. FIG. 7 is a diagram showing an operation example on the preview screen of the MFP. FIG. 7 is a diagram showing an operation example on the preview screen of the MFP. FIG. 7 is a diagram showing an operation example on the preview screen of the MFP. FIG. 6 is a view showing a preview screen of the MFP. FIG. 6 is a view showing a preview screen of the MFP. FIG. 6 is a view showing a preview screen of the MFP. It is a flow chart which shows file name generation processing.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the invention according to the claims, and all combinations of the features described in the embodiments are not necessarily essential to the solution means of the invention.

<< First Embodiment >>
<Overall configuration>
FIG. 1 is a diagram showing an overall configuration of an image processing system according to the present embodiment. The image processing system includes an MFP 110 and a file server 120. The MFP 110 and the file server 120 are communicably connected to each other via a LAN (Local Area Network).

  An MFP (Multi Function Printer) 110 is a multifunction peripheral having a plurality of functions such as a scanner and a printer, and is an example of an image processing apparatus. The file server 120 is an example of an external server that stores and manages digitized document files. The image processing system according to the present embodiment includes, but is not limited to, the MFP 110 and the file server 120. For example, the MFP 110 may have the role of the file server 120. Also, instead of the LAN, connection may be made via the Internet or the like. Further, the MFP 110 is connected to a Public Switched Telephone Networks (PSTN), and can perform facsimile communication of image data with a facsimile apparatus (not shown).

<Hardware Configuration of MFP>
FIG. 2 is a hardware configuration diagram of the MFP 110. As shown in FIG. The MFP 110 includes a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 includes the following units 211 to 219, and controls the overall operation of the MFP 110. The CPU 211 reads out a control program stored in the ROM 212, and executes and controls various functions of the MFP 110 such as reading, printing, and communication. The RAM 213 is used as a main memory of the CPU 211 and a temporary storage area such as a work area. In the present embodiment, one CPU 211 executes each process shown in the flowchart described later using one memory (the RAM 213 or the HDD 214), but is not limited to this. For example, a plurality of CPUs and a plurality of RAMs or HDDs may cooperate to execute each process. The HDD 214 is a large-capacity storage unit that stores image data and various programs. The operation unit I / F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 includes a touch panel, a keyboard, and the like, and receives operations, inputs, instructions, and the like by the user. The printer I / F 216 is an interface that connects the printer unit 221 and the control unit 210. The image data for printing is transferred from the control unit 210 to the printer unit 221 via the printer I / F 216 and printed on a recording medium. The scanner I / F 217 is an interface that connects the scanner unit 222 and the control unit 210. The scanner unit 222 reads a document set on an unillustrated document table or ADF (Auto Document Feeder) to generate image data (scanned image data), and the scanned image data is transmitted to the control unit 210 via the scanner I / F 217. Enter The MFP 110 can print out (copy) scanned image data generated by the scanner unit 222 from the printer unit 221, and can transmit a file or an email. The modem I / F 218 is an interface that connects the modem 223 and the control unit 210. A modem 223 carries out facsimile communication of image data with a facsimile apparatus on the PSTN. A network I / F 219 is an interface that connects the control unit 210 (MFP 110) to a LAN. The MFP 110 transmits image data and information to an external device (such as the file server 120) on the LAN using the network I / F 219, and receives various information.

<Hardware Configuration of File Server>
FIG. 3 is a hardware configuration diagram of the file server 120. As shown in FIG. The file server 120 includes a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I / F 315. The CPU 311 controls the entire operation of the file server 120 by reading out the control program stored in the ROM 312 and executing various processes. The RAM 313 is used as a main memory of the CPU 311 and a temporary storage area such as a work area. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I / F 315 is an interface that connects the file server 120 to a LAN. The file server 120 transmits and receives various types of information to and from another apparatus (such as the MFP 110) on the LAN using the network I / F 315.

<Software Configuration of MFP>
FIG. 4 is a software configuration diagram of the MFP 110. The MFP 110 can be roughly divided into two: a native function module 410 and an additional function module 420. The components included in the native function module 410 are provided as standard in the MFP 110, whereas the additional function module 420 is an application additionally installed in the MFP 110. The additional function module 420 is a Java (registered trademark) -based application, and can easily realize the addition of functions to the MFP 110. Note that another additional functional module (additional application) (not shown) may be installed in the MFP 110.

  The native function module 410 includes a scan execution unit 411 and an image data storage unit 412. The additional function module 420 includes a scan instruction unit 421, a metadata generation unit 422, an image analysis unit 423, an upload instruction unit 424, a file generation unit 425, and a display control unit 426.

  The display control unit 426 causes the liquid crystal display unit having the touch panel function of the operation unit 220 of the MFP 110 to receive a scan setting by the user and a scan start operation, an input, and an instruction UI screen (for example, FIG. (Described later) is displayed. The scan instruction unit 421 requests the scan execution unit 411 to perform scan processing together with the scan setting according to the user instruction input via the UI screen.

  The scan execution unit 411 receives a scan request including the scan setting from the scan instruction unit 421. The scan execution unit 411 generates scan image data by reading an image on a document by the scanner unit 222 via the scanner I / F 217 in accordance with a scan request. The generated scan image data is sent to the image data storage unit 412. The scan execution unit 411 sends a scan image identifier uniquely indicating the stored scan image data to the scan instruction unit 421. The image data storage unit 412 stores the scan image data received from the scan execution unit 411 in the HDD 214.

  The scan instruction unit 421 acquires, from the image data storage unit 412, scan image data corresponding to the scan image identifier received from the scan execution unit 411. The scan instruction unit 421 requests the metadata generation unit 422 to generate a file name of the acquired scan image data.

  The metadata generation unit 422 sends a display instruction of the UI screen to the display control unit 426. Based on the display instruction, the display control unit 426 causes the liquid crystal display unit having the touch panel function of the operation unit 220 of the MFP 110 to receive an operation, an input, and an instruction for generating a file name from the user FIG. 7 (a), the details will be described later. The metadata generation unit 422 sends an instruction to the display control unit 426, displays a preview image of the received scan image data on the UI screen, and instructs the image analysis unit 423 to analyze the scan image data.

  The image analysis unit 423 performs layout analysis processing and character recognition processing on the scan image data based on an instruction from the metadata generation unit 422. The image analysis unit 423 returns the processing result to the metadata generation unit 422.

  The metadata generation unit 422 generates metadata based on the user instruction and the analysis result. Metadata is information related to scan image data. An example of such information is a file name given to scan image data. Hereinafter, in the present embodiment, metadata will be described by way of an example of a file name. The metadata generation unit 422 generates a file name based on the user instruction and the analysis result. The metadata generation unit 422 sends the scan image identifier and the file name obtained as a result of the generation to the upload instruction unit 424, and instructs upload of the scan image data to the file server 120.

  The upload instruction unit 424 sends a display instruction of the UI screen to the display control unit 426. The display control unit 426 displays the UI screen on the liquid crystal display unit having the touch panel function of the operation unit 220 of the MFP 110 based on the display instruction. This UI screen is a screen for receiving folder path setting and upload operations, inputs, and instructions from the user (for example, FIG. 9, the details will be described later).

  In response to the upload instruction from the user, the upload instruction unit 424 instructs the file generation unit 425 to generate a file of scan image data indicated by the scan image identifier according to the instruction.

  The file generation unit 425 acquires scan image data corresponding to the instructed scan image identifier from the image data storage unit 412, and generates a file to be transmitted to the file server 120.

  The upload instruction unit 424 connects to the file server 120 using the set folder path setting, the file generated by the file generation unit 425, and the file name generated by the metadata generation unit 422, and transmits the file.

  The upload instruction unit 424 has a server message block (SMB) client function. As a result, file and folder operations are performed on the file server 120 having the SMB server function using the SMB. In addition to SMB, WebDAV (Distributed Authoring and Versioning Protocol for the WWW) may be used. Also, FTP (File Transfer Protocol), SMTP (Simple Mail Transfer Protocol), or the like may be used. In addition, Simple Object Access Protocol (SOAP) or Representational State Transfer (REST) other than the purpose of file transmission may be used.

<Flowchart of the whole process>
FIG. 5 is a flowchart showing the overall flow of control from generation of scan image data to upload. The series of processes are realized by the CPU 211 executing a control program stored in the HDD 214 in the control unit 210. Details will be described below.

  In step 501, the scan instruction unit 421 instructs the display control unit 426 to display a scan setting screen. The display control unit 426 displays on the operation unit 220 a scan setting screen for performing various settings in the scan processing.

  FIG. 6 is a diagram showing an example of the scan setting screen 600. As shown in FIG. Five setting buttons 601 to 605 exist in the scan setting screen 600 of FIG. [Color setting] button 601 is a button for setting color or monochrome when scanning an original. [Resolution setting] button 602 is a button for setting the resolution when scanning an original. [Duplex reading setting] button 603 is a setting button used when it is desired to scan both sides of a document. [Mixed original setting] button 604 is a setting button used when it is desired to scan originals of different sizes collectively. An “image format setting” button 605 is a setting button used to designate a storage format of scan image data. At the time of setting using these setting buttons 601 to 605, settable candidates (options) in the range supported by the MFP 110 are displayed, and the user selects a desired one from the displayed candidates. Note that the above setting button is an example, and all the setting items may not exist, and other setting items may exist. The user makes detailed settings for the scan process via the scan setting screen 600. [Cancel] button 620 is used to cancel the scan setting. [Scan start] button 621 is a button for instructing start of scan processing for the document set on the document table or the like.

  In step 502, the scan instruction unit 421 determines whether the “scan start” button 621 is pressed or the “cancel” button 620 is pressed. If it is determined that the “scan start” button 621 is pressed, the scan instruction unit 421 causes the scan execution unit 411 to execute the scan processing with the setting of the setting item selected by each of the scan setting buttons 601 to 605. If it is determined that the [cancel] button 620 has been pressed, the process ends.

  In step 503, the scan execution unit 411 issues a scan instruction to the scanner unit 222 to scan an original. The scan image data generated by scanning is stored in the image data storage unit 412, and the corresponding scan image identifier is notified to the scan instruction unit 421.

  In step 504, the scan instruction unit 421 acquires scan image data corresponding to the scan image identifier from the image data storage unit 412.

  In step 505, the metadata generation unit 422 sends a layout analysis instruction of the scan image data acquired from the image data storage unit 412 to the image analysis unit 423. The image analysis unit 423 analyzes the layout of the scan image data. For example, a layout of a scan image such as a character string area or a graphic area is analyzed by extracting a histogram of a scan image or extracting a block of pixels. The character string area is an area (image area) assumed to be a character string. The character string area also includes an area of one character. Note that the layout analysis process may include a process of correcting the inclination of the scan image to facilitate layout analysis, or detecting and rotating a direction. The image analysis unit 423 passes, to the metadata generation unit 422, information on the character string area analyzed by layout analysis (hereinafter, referred to as character string area information).

  Table 1 shows an example of character string area information analyzed by layout analysis.

  In Table 1 above, [No.] is a number uniquely indicating each identified character string area. In this example, serial numbers 1 to 11 are assigned in the order of recognition. [X coordinate of area] indicates the X coordinate of the upper left corner of each specified character string area. [Y coordinate of area] indicates the Y coordinate of the upper left corner of each specified character string area. Hereinafter, the term "coordinates" with respect to the character string area means the position coordinates of the upper left corner of the character string area unless otherwise specified. [Area width] indicates the distance from the left side to the right side of each of the specified character string areas. [Area height] indicates the distance from the upper side to the lower side of each specified character string area. In the present embodiment, “X coordinate of area”, “Y coordinate of area”, “width of area”, and “height of area” are all represented by pixels, but may be represented by points, inches or the like. Information of each character string area extracted from the scan image is passed to the metadata generation unit 422 as image analysis data. The image analysis data is, for example, in the format of CSV or XML, but may be another format. Alternatively, the data may be temporarily stored in the HDD 214 and then delivered at a predetermined timing.

  In step 506, the metadata generation unit 422 instructs the display control unit 426 to display the preview image. The display control unit 426 displays a preview screen on the touch panel of the operation unit 220 using the scan image data received from the scan instruction unit 421. The user can set the file name of scan image data via the preview screen.

  FIG. 7A shows an example of the preview screen 700. As shown in FIG. In the preview screen, in the preview area 710 at the center of the screen, a scan image represented by the read scan image data is displayed. Then, in the preview area 710, along with the scanned image, a plurality of buttons 711 to 714 for changing the display state are also displayed. Buttons 711 and 712 appear when the entire scanned image can not be displayed, and are buttons for vertically scrolling the display area. The touch panel provided in the MFP 110 is usually not so large. Therefore, for example, when the scanned image is a read of an A4 vertical / horizontal written document, the entire image in the width direction (short side direction) of the scanned image is displayed in a reduced size so that it fits exactly in the preview area 710. Initialized. That is, in the initial setting, the lower part of the A4 vertical scan image is not displayed in the preview area 710. In such a case, when the “↓” button 712 is pressed, the display area scrolls downward, and the lower part can be displayed. Furthermore, in the case where the scan image is, for example, A4 side or A3 side, a button may be further provided to scroll the display area in the horizontal direction. The buttons 713 and 714 are buttons for enlarging or reducing the display area, and zoom in is performed when the “+” button 713 is pressed, and zoom out is performed when the “−” button 714 is pressed. The operation by these button operations may be realized by an operation by the user's finger such as swipe on the preview screen, pinch out, and pinch in.

  A file name input field 701 displays the file name for the scan image. In the initial state, a character string indicating the date and time of scanning is set. When the user performs an operation (swipe operation or flick operation) to trace the character string area on the preview area 710 with a finger, a character string corresponding to the traced area is input to the file name input field 701. Detailed processing will be described later. A [Back] button 720 is a button used to cancel the preview display. [Next] button 721 is a button for shifting to a screen for setting an upload destination of the read scan image data. A button 702 is a button for setting the format of the file name and the like. Note that the types of various buttons described above and the manner of displaying and selecting each character string area are merely an example, and the present invention is not limited thereto. For example, there may be a button for correcting or changing the character string displayed in the file name input field 701 or determining the file name.

  FIG. 7B is a view showing coordinates and numbers of corresponding areas of the character string area shown in Table 1 in the preview area 710 shown in FIG. 7A. As for the character string area, the coordinate positions are indicated by dotted rectangles, and the numbers are displayed in association with the character string area. In the present embodiment, as shown mainly in FIG. 7A, the character string area is not illustrated on the preview screen on the touch panel by way of example. However, as shown in FIG. 7B, You may employ | adopt the form which displays a character string area | region.

  In step 507, the metadata generation unit 422 generates a file name for the scan image based on the input instruction from the user. Details of the file name generation process will be described later.

  FIG. 8 shows the state of the preview screen 700 after the file name is generated in step S507. In this example, “quotation sheet _ Tokyo Co., Ltd. April 14, 2017” is selected by sequentially selecting the character string corresponding to “quotation sheet”, “Tokyo Ltd.”, and “April 14, 2017”. The character string is displayed (set) in the file name input field 701. In the preview area 710, rectangles 801, 802, 803 indicating the character string used for the file name by the user's tracing operation are displayed. When a desired file name is generated and the user presses the “next” button 721, the process proceeds to step 508.

  In step 508, the metadata generation unit 422 determines whether the “next” button 721 is pressed or the “back” button 720 is pressed. If it is determined that the "Next" button 721 is pressed, the process proceeds to step 509. If it is determined that the "Back" button 720 is pressed, the process returns to step 501.

  In step 509, the metadata generation unit 422 acquires the file name set in the file name input field 701. The metadata generation unit 422 passes the acquired file name and scan image identifier to the upload instruction unit 424.

  In step 510, the upload instruction unit 424 instructs the display control unit 426 to display the upload setting screen. The display control unit 426 displays an upload setting screen on the touch panel of the operation unit 220. The user performs detailed settings for uploading to the file server 120 via the upload setting screen.

  FIG. 9 is a diagram showing an example of the upload setting screen 900. As shown in FIG. The user inputs a folder path for external transfer to the file server 120 in the [folder path] input field 901. In the example of FIG. 9, "\\ Server1 \ Share \ ScanData" is input as the folder path. The folder path can be input by displaying an address book reference screen (not shown) from the [address book] button 902 and the user selecting an address from the address book data stored in the HDD 214 of the MFP 110. . In the address book, user names and passwords for accessing other than the folder path of the file server 120 are also stored. The username and password are used when uploading the file to the file server 120. The [file name] label 903 displays the name of the file stored in the file server so that the user can easily recognize it. A “Back” button 920 is a button used to cancel the detailed setting regarding upload. An “Upload” button 921 is a button for instructing uploading to the folder path set in the “Folder path” input field 901.

  In step 511, the upload instruction unit 424 determines whether the “upload” button 921 is pressed or the “return” button 920 is pressed. If it is determined that the "upload" button 921 is pressed, the process proceeds to step 512, and if it is determined that the "back" button 920 is pressed, the process returns to step 506.

  In step 512, the upload instruction unit 424 generates a file to be uploaded from the scan image data corresponding to the scan image identifier. In step S513, the upload instruction unit 424 uploads the file generated in step 512 to the folder of the file server set in step S510 with the file name acquired in step S509.

  The above is the contents of operation control from generation of scan image data to upload according to the present embodiment. In the present embodiment, it is assumed that the processing in steps 505 to 507 is performed on image data of one page generated by scanning. For example, a button for performing image analysis of the next page is provided in the preview screen 700, and the preview display of the next page obtained by the analysis is performed, and the characters constituting the file name from the character string area of the subsequent pages It may be possible to set the column.

<File Name Generation Process>
Next, detailed processing of the file name generation processing (step S507) in the present embodiment will be described. In the present embodiment, as described above, the character string region in the entire scanned image is extracted by the layout analysis by the image analysis unit 423 in step S505. The character string area is an area (image area) that is presumed to be a character string, as described above. Thereafter, by performing character recognition processing (OCR: Optical Character Recognition) processing on the character string area, characters (text data) included in the character string area (image area) are extracted. The character recognition process is, for example, a process of recognizing a character (text data) by performing a matching process on a pixel group included in a character string area and a dictionary registered in advance. This character recognition process may take time for processing. Therefore, in the present embodiment, the character recognition process is performed on the character string area desired by the user without sequentially performing the character recognition process on the character string area extracted by the layout analysis. We are trying to speed up.

  In the present embodiment, the user traces the desired character string on the preview area 710 to determine the character string region desired by the user. Specifically, the character recognition process is performed on a character string area at least a portion of which overlaps the area traced by the user on the preview area 710. According to such processing, even if the area traced by the user on the preview area 710 does not include the character string area being extracted, the character string area desired by the user is selected. Character recognition processing is to be performed. Hereinafter, description will be made according to a specific flowchart.

  FIG. 10 is a flowchart showing file name generation processing by the metadata generation unit 422. In step S 1001, the metadata generation unit 422 determines whether the “next” button 721 or the “back” button 720 is pressed on the preview screen 700. If it is determined that the "Next" button 721 or the "Return" button 720 is pressed, the file name generation processing is terminated and the processing returns to the processing of the flowchart of FIG. If the “next” button 721 or the “back” button 720 is not pressed, the process proceeds to step 1002.

  In step 1002, the metadata generation unit 422 determines whether the user has touched the preview area 710. If it is determined that the user has touched, the process proceeds to step 1003. If it is not determined that the touch has been made, the process returns to step S1001.

  In step 1003, the metadata generation unit 422 acquires the coordinates touched by the user in the preview area 710 and holds the coordinates as the first operation coordinates.

  In step 1004, the metadata generation unit 422 determines whether the coordinates touched by the user have changed, that is, the user has moved the touched finger. If the coordinates touched by the user have changed, the process proceeds to step 1005. If the coordinates being touched have not changed, the process proceeds to step S1006.

  In step S1005, the metadata generation unit 422 causes the display control unit 426 to set the area from the first operation coordinate held in step S1003 to the coordinate currently touched as the selection area on the preview area 710. Overlap and display. FIG. 11 shows an example of the touch panel of the operation unit 220 displaying the selection area in the preview area 710. In the example of FIG. 11, when the finger is moved from the first operation coordinate 1101 starting the touch to the position 1102 (when swiping is performed), two points of the first operation coordinate 1101 and the current touch coordinate 1102 are vertexes The rectangular area is displayed. In this example, the finger is not released from the preview area 710 during the movement from the first operation coordinates 1101 to the position 1102. When moving to another position not shown, a rectangular area up to the other position is displayed. That is, as the finger moves, the displayed rectangular area (that is, the selected area) changes. In the example shown in FIG. 11, the selection area is displayed as a rectangular area, but it is also possible to display a line connecting two points of the first operation coordinates 1101 and the current touch coordinates 1102 by a straight line. Good.

  In step S1006, the metadata generation unit 422 determines whether the user has released the touching finger. If it is determined that the finger has been released, the process proceeds to step S1007. If it is determined that the finger is not released, the process returns to step S1004 and repeats the processing of steps S1004 to S1006 until the finger is released.

  In step S1007, the metadata generation unit 422 holds the coordinates at which the finger is separated as the second operation coordinates. That is, the coordinates at the time when the touched state is canceled are held as the second operation coordinates. When the process of step S1007 ends, the first operation coordinates and the second operation coordinates are determined. That is, the selected area is determined.

  In step S1008, the metadata generation unit 422 detects a character string area overlapping the selected area determined by the processing up to step S1007. As described above, the selection area may be a rectangular area having two points of the first operation coordinates and the second operation coordinates as vertices, or a straight line connecting the first operation coordinates and the second operation coordinates. May be the selection area. Alternatively, all the coordinates of each change in touch coordinates may be stored, and all the stored points may be set as the selection area. In step S1008, when the selection area and the character string area partially overlap, the character string area is detected as a character string area overlapping the selection area.

  FIG. 12A is a diagram for explaining the details of step S1008. FIG. 12A shows a state in which the user traces the preview area 710 on the touch panel from the first operation coordinate 1201 to the second operation coordinate 1202. The selection area 1210 is a rectangular area having two points of the first operation coordinates 1201 and the second operation coordinates 1202 as vertices. The detection of the character string area overlapping the selection area is performed by determining whether there is an area overlapping each character string area and the selection area, and detecting all character string areas determined to overlap. In the example of FIG. 12A, of the character string areas 1 to 10, the character string area 1, the character string area 2, and the character string area 3 partially overlap the selected area, and these character string areas Is detected as a character string area overlapping the selected area.

  In step S1009, the metadata generation unit 422 determines whether there is a character string area overlapping with the selected area as a result of the detection processing in step S1008. If there is an overlapping character string area, the process proceeds to step S1010. If there is no overlapping character string area, the process advances to step S1011.

  If there is an overlapping character string area, in step S1010, the metadata generation unit 422 detects a character recognition target area to be an area for character recognition using information on the selection area and the character string area overlapping the selection area. Do. In the present embodiment, the character recognition target area is a circumscribed rectangle including a selection area and a character string area overlapping the selection area. For example, the character recognition target area when the operation of FIG. 12 (a) is performed is the area 1220 shown in FIG. 12 (b).

  Here, the reason why the selection area and the character string area overlapping the selection area are set as the character recognition target area will be described. The area touched by the user may not necessarily include the text area. In this case, if only the area traced by the user with the finger is set as the character recognition target area, the character recognition process is performed on the image area where a part of the character is missing. For this reason, appropriate character recognition results can not be obtained. In this embodiment, layout analysis is performed in advance, and a character string area is extracted. Therefore, in the case where the area traced by the user with a finger overlaps with the character string area, the character recognition target area is expanded not only within the range of the selection area but also to the overlapping character string area. Become. Therefore, it becomes easy to obtain the character recognition result desired by the user.

  FIG. 13A shows an example in which the second operation coordinates 1302 overlap with part of the character string area 3 although the character “book” is not included in the lateral direction. Even when such a selection area 1310 is selected, the character recognition target area is an area 1320 shown in FIG. That is, even if the selected area is not determined so as to include a predetermined character on the preview area, the predetermined character is overlapped with the character string area being detected even if it is partially overlapped The character recognition target area also included is detected.

  Another example will be described with reference to FIG. As a result of the layout analysis process, the document may not be recognized as a character string area even though characters are described in the document. This is due to, for example, a decrease in analysis accuracy due to blurring of characters. FIG. 14A shows the case where the string area of “product” is not extracted due to any cause. In this embodiment, a circumscribed rectangle including a selection area and a character string area overlapping the selection area is detected as a character recognition target area. Therefore, a circumscribed rectangle including the selection area 1410 in FIG. 14 and the character string area 1 and the character string area 3 overlapping the selection area 1410 is detected as the character recognition target area. That is, the area 1420 in FIG. 14B is detected as the character recognition target area.

  The character string area overlapping with the selection area may be an area spanning two lines. In that case, it is possible to detect a region including the selection region and a character line region of two lines overlapping the selection region as a character recognition target region. Alternatively, a line having a large overlap with the selected area in the second direction (Y direction) intersecting the first direction (X direction) in which the character strings are arranged is detected as a character string recognition target area among a plurality of lines as a layout result. It is also good.

  Returning to the flowchart of FIG. 10, the description will be continued. If there is no character string area overlapping the selection area, the metadata generation unit 422 detects a character recognition target area from the selection area in step S1011. Although the selected area may be used as the character recognition target area as it is, when the height of the selected area (the width in the second direction) is not sufficient, only a part of the character image is included in the character recognition target area. It disappears. Because of this, characters can not be detected. Therefore, if the character string area exists in the left and right areas (area in the first direction) of the selected area, correct the Y coordinate and height of the selected area at the Y coordinate and height position of the character string area. The corrected selected area can be detected as a character recognition target area.

  In addition, it is conceivable that there is no character string area overlapping the selected area because the tracing position is shifted. For example, in the case of the form of tracing with a finger, the designated coordinates assumed by the user may not coincide with the detected coordinates detected on the screen. Therefore, when it is determined in step S1009 that there is no character string area overlapping the selected area, the selected area is reset by shifting the selected area up and down by a predetermined amount (for several points). Then, detection of a character string area overlapping the reset selection area may be performed again. If an overlapping character string area is detected as a result of the second detection, even if a process of detecting a selected area shifted by several points and an area including a character string area overlapping the selected area as a character recognition target area is added. Good. As a result of the process of step S1010 or step S1011 as described above, the character recognition target area is identified.

  In step S1012, the metadata generation unit 422 performs character recognition on the character recognition target area of the scan image, and acquires character coordinates and character information. For example, in the example of FIG. 12B, the character recognition process is performed on the area 1220 to acquire a character string (text data) “estimate”. In addition, the character coordinates of each character “m”, “product”, and “write” are also obtained.

  In step S1013, the metadata generation unit 422 detects a character to be used as a file name (hereinafter referred to as a file name usage character) from the character string acquired as a result of the character recognition processing in step S1012. In the present embodiment, the character string acquired as a result of the character recognition process is not used as a file name as it is, and processing is performed to detect a character to be used as a file name from the character string acquired as a result of the character recognition process. This is because, in the present embodiment, as the character recognition target area, an area including a selection area and a character string area partially overlapping the selection area is detected. That is, the character recognition target area is detected as a wider area so that the character string desired by the user can be easily detected. However, in some cases, more character string areas desired by the user may be detected as a result of layout analysis. Then, character recognition processing is performed on the character recognition target area including the area. As a result, when the obtained character string is used as the file name as it is, the result desired by the user may not be obtained. That is, a situation occurs in which the user wants to use only a part of the characters among the character strings acquired as a result of the character recognition processing of the character string recognition target area. Therefore, in the present embodiment, the character to be used as the file name is detected based on the character coordinates of each character of the character string (text data) acquired by the character recognition processing and the selection area. Hereinafter, this will be specifically described using the drawings.

  FIG. 15A shows a state in which the selection area 1510 is detected by the user following the operation from the first operation coordinate 1501 to the second operation coordinate 1502. A character string area overlapping the selection area 1510 is detected as the character string area 4 only, and a circumscribed rectangle including the selection area 1510 and the character string area 4 overlapping the selection area 1510 is detected as a character recognition target area. For this reason, in the case of FIG. 15A, the character string (text data) obtained by performing the character recognition processing in step S1012 is “Tokyo Tokyo Co., Ltd.”. Further, by performing the character recognition process, character coordinates for each character are detected as shown in FIG. FIG. 15B shows a circumscribed rectangle of each character based on character coordinates. In the process of step S1013, the character of the area where the character coordinates of each character and the selection area partially overlap with each other is detected as a character string (text data) used for the file name. In FIG. 15 (b), the characters "east", "kyo", "stock", "form", "meeting" and "company" overlap the selected area, and "G" and "middle" are overlapped with the selected area. It does not. For this reason, "Tokyo Ltd." is detected as the file name usage character. According to such processing, the character corresponding to the selected area selected by the user is detected as the file name usage character. Therefore, characters desired by the user are detected.

  In step S1014, the metadata generation unit 422 determines whether the change of the file name is performed on the preview screen 700. If the file name has not been edited even once, since the file name is in the initial state, the process proceeds to step S1015 and the file name is set to be empty. Thereafter, the process proceeds to step S1016. On the other hand, if the file name has already been edited, the user has already added the character on which the touch operation has been performed, and the process advances to step S1016. In step S1016, the file name usage character detected in step S1013 this time is added to the file name. Then, the metadata generation unit 422 causes the display control unit 426 to set and display the character string in the file name input field 701. At this time, it is also possible to add a delimiter (for example, “-” hyphen or “_” underbar) and then add a file name usage character.

  In the flowchart of FIG. 10, an example in which the point where the touch is started is the first operation coordinate and the point where the finger is released is the second operation coordinate has been described, but other methods may be used for the first operation coordinate and Second operation coordinates may be designated. For example, by performing the touch twice, it is possible to use a designation method in which the first operation coordinate is the first operation coordinate and the second operation is the second operation coordinate. When multi-touch is supported, each coordinate at which a plurality of simultaneous touches by the user is detected may be set as the first operation coordinate and the second operation coordinate.

  As described above, in the present embodiment, an area including the selection area selected by the user and the character string area at least partially overlapping the selection area is determined as the character recognition target area. Then, character recognition processing is performed on the character recognition target area. According to such processing, even if the selection region selected by the user does not necessarily designate the character string region properly, it is possible to appropriately detect the target region on which the character recognition processing desired by the user is to be performed. As a result, a character string that the user desires to use as information related to the scan image data is properly obtained.

<< Embodiment 2 >>
In the first embodiment, an example has been described in which a character having character coordinates at least partially overlapping a selection area is acquired as a character used for a file name. Since the liquid crystal display unit of the operation unit 220 of the MFP 110 has a narrow operation area, the user may not be appropriately operated. In the present embodiment, a mode in which simple editing is performed after acquisition as characters used for a file name will be described.

  FIG. 16 is a diagram showing on the screen a correction button for making corrections after the file name is edited by the user specifying the selected area three times on the preview screen. In FIG. 16, the user tried to input up to "April 2017" after "Estimate" and "Tokyo Co., Ltd.", but the operation area is narrow and the operation fails and "April 1 2017" is input. It shows that it has been done. In order to easily correct such an operation mistake, correction buttons 1601 to 1604 are displayed in the vicinity of the selected character string area. The vicinity of the character string area may be an area overlapping with the character string area or an area not overlapping with the character string area. In FIG. 16, the character part (character recognition target area part) of the characters constituting the currently set file name is highlighted. When the correction button 1601 is pressed, the first character of the selected area is added one character. When the correction button 1602 is pressed, the first character is deleted. Similarly, when the correction button 1603 is pressed, the last character of the selected area is deleted, and when the correction button 1604 is pressed, the last character is added. The user can delete one character from the end by pressing the correction button 1603 in order to correct "April 1, 2017" to "April 2017". When there is no character to be added at the beginning, the character is not added even if the correction button 1601 is pressed. The same applies to the correction button 1604. Alternatively, if there is no character to add, the corresponding correction button may not be displayed. The correction buttons 1601 to 1604 are displayed for each selection area, and corresponding characters in the file name input field 701 are added and deleted each time the button is pressed.

  Although the form using the button-like icon which is easy to understand addition and deletion of a character was explained here, it is not restricted to this. Any icon that the user can easily understand can be used.

  As described above, according to the present embodiment, the file name can be edited easily even when the user does not appropriately designate the selection area.

<< Third Embodiment >>
In the first embodiment, an example has been described in which layout analysis is performed first and then character recognition processing is performed when the preview area is touched by the user. This is because if the layout analysis and the character recognition processing are continuously performed on the entire scanned image, the processing time will be long, the user's waiting time will increase, and as a result, operability may deteriorate. According to the process described in the first embodiment, after the scan image data is generated, the waiting time for the user to start setting of the file name is reduced. However, waiting time occurs when the user operates. This is because the character recognition process is performed on the target area only when it is touched. In the present embodiment, an embodiment in which the waiting time when the user operates is suppressed will be described. Specifically, an embodiment in which the layout analysis process and the character recognition process are continuously performed will be described. Since the character recognition process is performed prior to the user's operation of starting the setting of the file name, the character coordinates of the character unit are detected before the user's operation. The third embodiment will explain an embodiment using character coordinates detected in this manner. In the following, mainly the points different from the first embodiment will be mainly described.

  In the present embodiment, in the analysis processing of scan image data in step S505 in FIG. 5, character recognition of the entire scan image is also performed following layout analysis, and character coordinates for each character are also detected. The other respects are the same as in the first embodiment, and thus the description thereof is omitted.

  FIG. 17 is a flowchart showing file name generation processing in the third embodiment. The processes of steps S1701 to S1703 are the same as the processes of steps S1001 to S1003 of FIG.

  In step S1704, the metadata generation unit 422 holds the file name at the time of the touch. As described later, in the present embodiment, the file name of the file name input field 701 changes in real time according to the situation where the user traces the preview area. For example, while the first selection area is being selected (while expanding or reducing the selection area), the file name of the file name input field 701 changes in real time. After that, the user once removes the finger from the preview area. Then, at that time, the file name corresponding to the first selected area is determined. Thereafter, while another second selection area is being selected, the file name corresponding to the second selection area in the file name input field 701 changes in real time. As described above, the current file name determined at that time is decided once, and the character corresponding to the selected area being edited is edited in real time. Therefore, in step S1704, processing for holding the current file name is performed.

  In step S1705, the metadata generation unit 422 detects a change in coordinates touched by the user. If the coordinates touched by the user do not change, monitoring continues until the touch coordinates change. If the coordinates touched by the user have changed, the process advances to step S1706 to hold the changed coordinates as the second operation coordinates.

  In step S1707, the metadata generation unit 422 superimposes and displays a rectangular area having two points of the first operation coordinate and the second operation coordinate as the selected area on the preview image. Then, in step S1708, the metadata generation unit 422 detects a character area overlapping the selected area. In the present embodiment, since each character coordinate has already been acquired, a character area of character coordinates at least partially overlapping the selected area is detected. In step S1709, the metadata generation unit 422 detects a character string obtained by combining the characters of the detected area as a file name usage character.

  The processes of steps S1710 and S1711 are the same as steps S1014 and S1015, and are thus omitted.

  In step S1712, the metadata generation unit 422 adds the character detected as the file name usage character to the file name held in step S1704. Then, in step 1513, it is detected whether or not the user has released the finger from the touch panel. If the user has released the finger, the process returns to the process of step S1701, and if not, the process returns to the process of step S1705.

  As described above, according to the process of the present embodiment, while the user is operating the preview area, the selection area is continuously detected. When the characters overlapping the selection area increase, the characters of the file name input field 701 also increase, and when the characters overlapping the selection area decrease, the characters of the file name input field 701 also decrease. Therefore, the characters of the file name input field 701 are set in real time according to the user operation.

  Note that, in the present embodiment, an example has been described in which the character recognition processing is performed collectively before the operation of starting the setting of the file name following the layout analysis processing. However, when the character recognition process is sufficiently fast, the character recognition may be performed on the selected area each time the touch coordinates change.

  In addition, although basically the configuration of the first embodiment is employed, character recognition processing may be performed in the background, for example, from the upper character string area in the background from the time the preview screen is displayed to the time the user operates it. . Then, the processing can be switched to the processing of the third embodiment for the region where the character recognition is completed.

  When the touch panel of the operation unit 220 of the MFP corresponds to multi-touch, file name generation is performed with the first touched coordinates as the first operated coordinates and the second touched coordinates as the second operated coordinates. Processing can also be performed. In that case, both sides of the selection area can be expanded or narrowed by detecting changes in both touch coordinates and changing the selection area.

<< Other Embodiments >>
In the embodiment described above, although the file upload mode has been mainly described as an example, the present invention is not limited to this. The scan image data file may be stored in the HDD in the MFP 110.

  Further, in the embodiment described above, the scene of setting the file name using the character recognition result of the character string area in the scan image has been described as an example, but the present invention is not limited to this form. The metadata may be used for, for example, transfer destination setting of data such as a scan image upload destination, or destination setting for fax transmission or e-mail transmission. In this case, for example, in the upload setting screen 900 of FIG. 9 described above, the character recognition result of the character string area in the scan image can be displayed as selectable folder path candidates so that the path name can be set according to user selection. You may Also, on the destination setting screen (not shown), the character recognition result of the character string area in the scanned image is displayed as selectable as a FAX number or a mail address candidate, and a FAX number or a mail address is set according to the user selection. It is also good. Furthermore, the metadata may be attribute information of the scan image data. For example, information of a character string selected by the user may be added to the attribute information of the file.

  In the embodiment described above, an example in which a document in a format in which characters are described from left to right is scanned has been described, but the arrangement order of character strings is not limited to this. Further, the positional relationship between the first operation coordinates and the second operation coordinates, and the arrangement order of the character strings are not limited to the aspect described in the above-described embodiment. For example, when characters are described from left to right, the first operation coordinates may be relatively at the right position, and the second operation coordinates may be relatively at the left position. That is, the selection areas may be designated in the reverse order of the arrangement of the character strings. Further, the arrangement order of the character strings may be arranged in the vertical direction (Y direction).

  Further, in the embodiment described above, the form in which the character coordinates of each character are obtained as a result of the character recognition processing has been described, but the present invention is not limited to this. Depending on the type of layout analysis processing, not only the character string area but also the character coordinates of each character contained in the character string area may be obtained as an analysis result. In this case, it is possible to obtain a desired character recognition target area by determining whether there is a character string area overlapping the selection area described in step S1009 described above. Therefore, in step S1013, the result of performing the character recognition process on the character recognition target area may be detected as the file name use character string as it is.

  Further, in the above embodiment, although the user has described the form in which the selection area is specified by tracing the area of the character string on the preview screen with a finger (swipe operation or flick operation), the present invention is not limited thereto. . Instead of the finger, the area of the text on the preview screen may be traced with a stylus pen or the like. Further, the area of the character string on the preview screen may be designated by another input means such as a mouse cursor.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

Claims (21)

An apparatus for setting information related to scanned image data obtained by scanning a document, comprising:
Display control means for displaying a preview screen of a scanned image represented by the scanned image data;
Extracting means for extracting a character string area assumed to be a character string in the scan image;
Detection means for detecting, as a target area, a character string area at least partially overlapping a selected area based on the first coordinate and the second coordinate designated in the preview screen among the extracted character string areas;
Recognition means for performing character recognition processing of the target area;
An setting unit configured to set the information using characters obtained as a result of the character recognition processing.   The apparatus according to claim 1, wherein the detection unit detects an area including the selection area and a character string area at least partially overlapping the selection area as the target area.   3. The apparatus according to claim 2, wherein the detection means detects an area of a circumscribed rectangle including the selection area and a character string area at least partially overlapping the selection area as the target area.   The said setting means sets the said information using the character which at least one part overlaps with the said selection area among the characters obtained as a result of the said character recognition process, It is characterized by the above-mentioned. The device described in the section.   When the detection unit has a plurality of lines of character string areas at least partially overlapping the selection area, the detection unit has an overlap with the selection area in a second direction intersecting the first direction in which the character strings are arranged in the character string area. The apparatus according to any one of claims 1 to 4, wherein a large row is detected as the target area.   The detection means is characterized in that, when there is no character string area at least partially overlapping the selected area, a character string area overlapping at least partially with the area shifted by the predetermined amount as the target area is detected as the target area. The device according to any one of claims 1 to 5, wherein   The display control means displays the set information at a predetermined position on the preview screen, and adds or deletes a character forming the information in the vicinity of the character string area including the character that is the source of the information. The apparatus according to any one of claims 1 to 6, characterized in that an icon is displayed to indicate.   The apparatus according to claim 7, wherein the display control means changes the information displayed at the predetermined position when the icon is selected.   9. The apparatus according to claim 7, wherein the display control means highlights a character string area including a character that is the source of the information on the preview screen. An apparatus for setting information related to scanned image data obtained by scanning a document, comprising:
Display control means for displaying a preview screen of a scanned image represented by the scanned image data;
The information is set using a character obtained as a result of performing character recognition processing on a character string area at least partially overlapping a selected area based on the first coordinate and the second coordinate designated in the preview screen. And setting means,
The display control means displays the set information at a predetermined position on the preview screen, and adds or deletes a character forming the information in the vicinity of the character string area including the character that is the source of the information. Displaying an icon for instructing the user.   The apparatus according to claim 10, wherein the display control means changes the information displayed at the predetermined position when the icon is selected.   12. The apparatus according to claim 10, wherein the display control means highlights a character string area including characters as a source of the information on the preview screen. An apparatus for setting information related to scanned image data obtained by scanning a document, comprising:
Display control means for displaying a preview screen of a scanned image represented by the scanned image data;
Extracting means for extracting a character string area assumed to be a character string in the scan image;
Recognition means for performing character recognition processing of the extracted character string area;
Detection means for detecting a change in first and second coordinates designated in the preview screen;
A determination unit configured to determine a selected area based on the first coordinate and the second coordinate each time a change by the detection unit is detected;
Among the characters obtained by the character recognition process, characters overlapping at least in part with the determined selection area are displayed at a predetermined position on the preview screen, and at least in the selection area when the coordinates are not detected. And setting means for setting the information by using partially overlapping characters.   The first coordinate is a coordinate at which a touch by a user is detected in the preview screen, and the second coordinate is a coordinate at a time when the touched state is canceled. The device according to any one of 1 to 13.   The first coordinate is a coordinate at which a first touch by a user is detected in the preview screen, and the second coordinate is a coordinate by the user after the first touch is canceled in the preview screen. 14. An apparatus according to any one of the preceding claims, characterized in that it is the coordinates at which the second touch was detected.   The said 1st coordinate and the said 2nd coordinate are each coordinates which detected the several simultaneous touch by the user in the said preview screen, The said 1st coordinate and the 2nd coordinate are characterized by the above-mentioned. apparatus.   The apparatus according to any one of claims 1 to 16, wherein the information is a file name given to the scan image data. A method for setting information related to scanned image data obtained by scanning a document, comprising:
Displaying a preview screen of a scanned image represented by the scanned image data;
Extracting a character string area assumed to be a character string in the scanned image;
Detecting, as a target area, a character string area at least partially overlapping a selected area based on a first coordinate and a second coordinate designated in the preview screen among the extracted character string areas;
Performing character recognition processing of the target area;
Setting the information using characters obtained as a result of the character recognition process. A method for setting information related to scanned image data obtained by scanning a document, comprising:
Displaying a preview screen of a scanned image represented by the scanned image data;
The information is set using a character obtained as a result of performing character recognition processing on a character string area at least partially overlapping a selected area based on the first coordinate and the second coordinate designated in the preview screen. Displaying the set information at a predetermined position of the preview screen;
Displaying an icon for instructing addition or deletion of characters constituting the information in the vicinity of the character string area including the character as a source of the information. A method for setting information related to scanned image data obtained by scanning a document, comprising:
Displaying a preview screen of a scanned image represented by the scanned image data;
Extracting a character string area assumed to be a character string in the scanned image;
Performing character recognition processing of the extracted character string area;
Detecting changes in first and second coordinates designated in the preview screen;
Determining a selected region based on the first coordinate and the second coordinate each time the change is detected;
Among the characters obtained by the character recognition process, characters overlapping at least in part with the determined selection area are displayed at a predetermined position on the preview screen, and at least in the selection area when the coordinates are not detected. Setting the information using partially overlapping characters.   The program for functioning a computer as each means of the apparatus as described in any one of Claims 1-17.