JP2011028503A - Image processor, image processing method, and program - Google Patents

Abstract

When assigning metadata to objects in an input image document, if all the objects are managed individually, the amount of metadata to be added increases, resulting in an increase in file size. This increases the search efficiency.
A region dividing unit for dividing an input document into regions, an attribute information adding unit for adding attribute information, a character recognition unit for obtaining character information, and a hierarchization level for grouping objects are calculated. A hierarchy level calculation unit that integrates the objects in a group having the same caption, and an object identifier generation unit that generates an individual identifier for each group, and an associated caption is detected from the input document. And a metadata extraction unit that extracts metadata related to the object and associates the identifier for the object with the metadata and stores it in the storage area.
[Selection] Figure 5

Description

  The present invention relates to an information processing apparatus, an information processing system, an information output control method, and a computer readable program storing a program for implementing the method for providing metadata for efficiently searching for an object from a document image. The present invention relates to a storage medium and the program.

  In order to effectively use the input document image, when a character string adjacent to an object (for example, a photo, a drawing, a line drawing, a table, etc.) excluding characters in the document is a caption (a character string describing the object), A caption is associated with the object as metadata. (Hereinafter, unless otherwise specified, an object means an object excluding characters (for example, a photograph, a drawing, a line drawing, a table, etc.).) Thus, an electronic document image is used on the application side. In this case, it is possible to search for an object using the metadata as a search keyword.

When the caption adjacent to this object is a figure number (for example, “FIG. 1”, “FIG. 1”, etc.), in a general document image, the same expression as this figure number is described in the text. In this case, a link is automatically generated between the figure number and the same expression of the text, and converted into hypertext. For example, when the caption adjacent to the object is “FIG. 1” and the description “FIG. 1 is AAA” in the text, the caption “FIG. 1” and “FIG. 1” in the text are the same expression. Because there is, a link is generated. (Patent Document 1)
On the other hand, the input document image is analyzed, and the layout structure related to the geometric information of the document element (for example, text, drawing, photo, table, etc.) and the logical semantic information (for example, chapter, section, document paragraph, etc.) A structured document having a logical structure or the like is being created. As a result, it is possible to group text and chart areas included in a document image, tag the above caption information, and specify the document structure (Patent Document 2).

Japanese Patent Laid-Open No. 10-228473 JP 2003-288334 A

  However, in the preceding cases, there are many objects with high relevance that are classified by sub-captions (for example, “FIG. 1A”, “FIG. 1B”, etc.) in the input image document. Even in such a case, the relevance of each object is not detected. For this reason, captions and sub-captions corresponding to the objects are grouped and held as all independent objects. As a result, the keywords detected from the explanatory text in the image document are assigned to all objects, and there is a problem that the file size increases. Further, since the number of candidates corresponding to the keyword search increases, there is a problem that it becomes difficult to find a desired result and the search efficiency is lowered.

  In order to solve the above problems, an area dividing unit that divides an input document into regions, an attribute information adding unit that assigns attribute information to the divided regions, and a character recognition unit that obtains character information for a character region; A hierarchy level calculation unit for calculating a hierarchy level when grouping the objects, and the objects are integrated in a group having the same caption in the hierarchy level, and an individual identifier is generated for each group. An object identifier generation unit that detects the caption associated with the object from the input document, and extracts metadata associated with the object, and associates the object identifier with the metadata in a storage area. The image processing apparatus includes a metadata extraction unit for storing.

  According to the present invention, the file size can be reduced and the search efficiency can be improved by grouping in consideration of the relevance of the objects in accordance with the file size of the input document image or the setting by the user.

Example of image processing system configuration Configuration diagram of MFP Example of data processing unit configuration Hierarchical level and grouping illustration Processing flow in the first embodiment Explanatory drawing of area division / attribute information assignment Example of hierarchical level calculation processing in the first embodiment Explanatory diagram of object hierarchy analysis Illustration of metadata extraction process Illustration of image search Comparison of image search results and metadata according to difference in hierarchical level N (a) N = 1, (b) N = 2, (c) N = 3 Example of operation screen Hierarchical level calculation processing in the second embodiment Example of format (a) N = 1, (b) N = 3

Example 1
The best mode for carrying out the present invention will be described below with reference to the drawings.

  In the first embodiment, in order to perform an efficient search, a method of grouping related objects and associating metadata for searching the objects with the grouped objects will be described.

  FIG. 1 is a block diagram showing the configuration of the image processing system according to the first embodiment of the present invention.

  In FIG. 1, an MFP (Multi Function Peripheral) 100 that is a multi-function peripheral that realizes a plurality of functions (for example, a copy function, a print function, a transmission function, etc.) is connected to a LAN 102 constructed in an office A. A client PC 101 and a proxy server 103 that receive transmission data from the MFP 100 and use functions realized by the MFP 100 are connected to the same LAN 102. The LAN 102 is connected to the network 104 via the proxy server 103. In the client PC 101, for example, by transmitting print data to the MFP 100, a printed matter based on the print data can be printed by the MFP 100.

  The configuration in FIG. 1 is an example, and a plurality of offices having the same components as the office A may be connected on the network 104. The network 104 is the Internet, LAN, WAN, telephone line, dedicated digital line, ATM, frame relay line, communication satellite line, cable TV line, data broadcasting wireless line, or the like. Or what is necessary is just to be able to transmit / receive data in what is called a communication network implement | achieved by these combination. The various terminals of the client PC 101 and the proxy server 103 are standard components (for example, CPU, RAM, ROM, hard disk, external storage device, network I / F, display, keyboard, mouse, etc.) mounted on a general-purpose computer. ).

  Next, a detailed configuration of the MFP 100 will be described with reference to FIG. FIG. 2 is a diagram showing a detailed configuration of the MFP according to the first embodiment of the present invention.

  As shown in FIG. 2, the MFP 100 includes a network I / F 204, a scanner unit 200, a printer unit 201, an operation unit 207, a display unit 208, and a controller unit 209. The controller unit 209 includes a data processing unit 202, a storage unit 203, a PDL processing unit 205, and a control unit 206. A process flow inside the MFP will be described.

  A scanner unit 200 including an auto document feeder (ADF) irradiates an input original image with a light source, forms an original reflection image on a solid-state image sensor with a lens, and outputs a raster-like image read signal from the solid-state image sensor to a predetermined density. Obtained as image data (for example, 600 DPI). The control unit 206 sends the image data obtained by the scanner unit 200 to the data processing unit 202.

  On the other hand, the PDL data output from the client PC 101 is received by the PDL processing unit 205 via the network I / F 204. The PDL processing unit 205 renders the PDL data, and the control unit 206 sends the rendered PDL data to the data processing unit 202.

  Then, the data processing unit 202 performs image processing so that the received input signal is suitable for output by the printer, and sends it to the printer unit 201 via the storage unit 204.

  Also, the transmission function via the network I / F 204 converts the image signal obtained from the scanner unit 200 into an image file in a compressed image file format such as TIFF or JPEG or a vector data file format such as PDF. Output from F204. The output image file is transmitted to the client 101 via the LAN 102, and further transferred to an external terminal (for example, another MFP or client PC) via the network 104.

  A user's instruction to the MFP 100 is performed from an operation unit 207 and a display unit 208 including a key operation unit and a touch panel provided in the MFP 100, and a series of these operations is controlled by the control unit 206. Further, the display of the operation input status and the image data being processed are displayed on the display unit 208.

  The storage unit 203 is realized by a large-capacity hard disk, for example, and constitutes a database that stores and manages image data read by the scanner unit 200 and PDL data processed by the PDL processing unit 205. In particular, in the present invention, image data and area information obtained by dividing the image data into areas can be managed in association with each other.

  Next, the configuration of the data processing unit 202 in FIG. 2 and the outline of the processing of each configuration unit will be described with reference to FIG.

  The data processing unit 202 includes an area dividing unit 300, an attribute information adding unit 301, a character recognizing unit 302, a hierarchized level calculating unit 303, an object identifier generating unit 304, and a metadata extracting unit 305. Then, the data received from the scanner unit 200 or the PDL processing unit 205 is subjected to area division, object grouping, metadata extraction processing, etc., and data with metadata added is created and stored in the storage unit 203. send.

  The area dividing unit 300 receives an image document received from the scanner unit 200 or the PDL processing unit 205 as an input, and divides the area.

  The attribute information adding unit 301 provides attribute information for each area divided by the area dividing unit 300. First, it is classified into a character area and an object area depending on whether characters are included in the area. In the character area, attributes such as “chapter”, “section”, “body”, “caption”, “subcaption”, “header / footer”, and “character part” are used. ”And“ Noise ”attributes.

  The character recognition unit 302 performs character recognition on character regions to which attributes such as “text”, “caption”, “subcaption”, “chapter”, “section”, “header / footer”, and “character part” are assigned. I do. Then, the result is associated with the target area.

  The hierarchization level calculation unit 303 determines the hierarchization level (N) according to the data size received from the scanner unit 200 or the PDL processing unit 205. The hierarchization level is used as a reference for identifying a plurality of objects as one group when generating an object identifier described later. If the layering level is reduced, objects are classified at a more detailed level, so that grouping is not performed much. On the other hand, by increasing the hierarchization level, objects are classified in a more abstract group.

  The object identifier generation unit 304 groups related objects, and generates and assigns object identifiers for identifying each grouped object. For example, when the object is subdivided by sub-caption as shown in FIG. 4, when the hierarchization level is minimum (when N = 1), the sub-caption level (FIG 1 (a ) And FIG. 1 (b)). As a result, no grouping is performed, and the two objects 400 and 401 are treated as independent objects as shown in FIG. Here, when the hierarchization level is increased by one (when N = 2), the abstraction level is increased and the comparison is made at the caption level (FIG. 1). As a result, as shown in FIG. 4B, two objects 400 and 401 are grouped into one, and can be handled as if they were one object 402.

  The metadata extraction unit 305 searches the body area for the same or synonymous expression as the caption given to the object. If found, it is extracted from the text as metadata when searching for the object and stored in association with the identifier for the object.

  Next, with reference to the flowchart shown in FIG. 5, the flow of the process of the 1st Example in this embodiment and the detailed process of each structure part are demonstrated.

  First, in step S500, the region dividing unit 300 performs processing for dividing the image document received from the scanner unit 200 and the PDL processing unit 205 into regions. An example of the region extraction method will be described. First, an input image is divided into M × N pixels, and if even one pixel exists, a thinned image is created assuming that there is an image there. Next, a small rectangle is created by collecting the portions where the thinned images are connected. Since rectangles having a large aspect ratio and short sides close to each other are highly likely to be character lines, the rectangles are combined in this case. In addition, a set of rectangles having substantially the same short side length and arranged at equal intervals is likely to be a body part, and thus is combined. As a result, for example, a region 419 is extracted from the region 410 as shown in FIG.

  In step S501, the attribute information adding unit 301 assigns attributes such as a figure, a table, a text, and a caption to each divided area. First, it is determined whether or not a character is included, and it is classified into a character region (region including a character) and an object region (other than the character region). Subsequently, the character area and the object area are classified in detail.

  For the object area, an attribute such as “noise” is assigned to a portion having a small area, “table” is assigned to a portion having a low pixel density, and “figure” is assigned to a portion other than the figure.

  On the other hand, regarding the character area, it is determined that an area created by combining a plurality of small rectangles is a body part, and an attribute “body” is added. In addition, a character area existing in the vicinity of an object to which the attribute “table” or “photo” is assigned is given an attribute as “caption” for explaining the object. Here, if a smaller character area exists in the vicinity of the caption, the caption is re-classified into a relationship of “sub-caption” and “caption” as a caption that further explains the caption. For the remaining area, attributes such as “chapter”, “section”, “header / footer”, and “character part” are given depending on the positional relationship with the body part, the character size, and the line spacing of the character string.

  FIG. 6 shows the result of classifying FIG. 6 based on the above classification. Area 410 is a node, areas 411 to 413 are diagrams, areas 414 and 415 are captions, areas 416 and 417 are sub-captions, and areas 418 are The text and area 419 are given the attribute of page.

  In step S502, the character recognizing unit 302 performs character recognition processing on a character area to which an attribute such as a text or a caption is added, and stores the character area in association with the character area. This makes it possible to search for text in the image document.

  In step S503, the hierarchization level calculation unit 303 determines the hierarchization level N required in an object identifier generation process S507 described later. Decreasing the hierarchization level inevitably increases the file size because the relationship between the object and metadata is maintained in detailed units. Therefore, the hierarchization level N is determined step by step according to the data size received from the scanner unit 200 and the PDL processing unit 205. As shown in FIG. 7, for example, N = 1 if the number of pages is 5 or less, N = 2 if the number of pages is 10 or less, and N = 3 if it is more than 10 pages. Alternatively, the file capacity, the number of objects included in the image document, and the like may be used as the determination conditions. Of course, these may be combined and determined. The maximum level of hierarchization is 3 here, but it can be set freely. However, if the maximum level is increased too much, even objects that are not originally related may be grouped because the level of abstraction has been increased too much, which may reduce search efficiency. The degree seems to be appropriate.

  Subsequently, the object identifier generation unit 304 groups related objects and generates a unique object identifier for each of them. First, in step S504, an object area with a caption attribute is detected. If not detected, the process proceeds to step S508. If detected, the hierarchical structure is analyzed for all objects (step S505), and the objects are grouped according to the hierarchical level obtained in step S503 (step S506). An identifier is calculated and assigned (step S507).

  FIG. 8 shows the result of analyzing the hierarchical structure of the object with respect to FIG.

  If an object area with a caption attribute is detected in step S504, the object areas 411 to 413 correspond. Therefore, the layout structure and the logical structure are analyzed for these three areas, and character strings that are captioned hierarchically are detected (step S505). The state of structuring will be described using the object region 412 as an example. First, in the object area 412, there is an object area 416 "Fig2 (a)" as a sub-caption that gives a detailed description. This is the first layer. In the second layer, it can be seen that the object area 415 having the caption “FIG2” summarizes the objects having the subcaption. In the third hierarchy, the object area 412 is regarded as a figure included in the section “1.1 XX” (object area 410). In the fourth hierarchy, the object area 412 may belong to the unit “Page1” (object area 419). Recognize. FIG. 8 shows the result of analyzing the object 411 and the object 413 in the same process.

  In step S506, the analysis result is compared with the hierarchization level (N) calculated by the hierarchization level calculation unit 303 described above, and the objects having the same or the same caption are determined to be related to each other. Do. Then, a unique object identifier is generated and assigned to each group (step S507). For example, when the hierarchization level N = 1, since the comparison is performed using the captions assigned in the first hierarchy, it is determined that each object is independent. As a result, object identifier IDs = 1, 2, and 3 are generated and assigned to each. When the hierarchization level N = 3, since the comparison is performed in the third hierarchy, it can be seen that the caption “1.1 XX” is given to all objects. As a result, the three objects are integrated as one group, and a common object identifier ID = 1 is generated and assigned to each object.

  Thereafter, the metadata extraction unit 305 searches the body area for the same or synonymous expression as the caption given to the grouped object (step S508). If not found, the process proceeds to step S511. If it is found, the process proceeds to step S509, where it is extracted from the text as metadata when searching for the object. Then, it is stored in association with the object identifier (step S510). For example, when the hierarchization level N = 3, all the captions assigned to the three object areas are searched from the text, the corresponding keywords are extracted, and the duplicate keywords deleted are assigned as metadata ( (See FIG. 9).

In step S511, an image document with the extracted metadata added to the input image document is generated. The format is a format (for example, PDF, XPS, OOXML) that can associate metadata with an object. ("PDF" and "XPS" are registered trademarks)
FIG. 14 shows an example of creating a format with metadata.

  Object areas 411 to 413 having attributes shown in the figure correspond to format elements 800 to 803, respectively. Information such as the position (Position) and size (Size) of the area divided by the area dividing unit 300 and the object identifier (ID) generated by the object identifier generating unit 304 are held. In addition, a character area, for example, a body area 418 having a body attribute, is recognized as a format element 803 by the character recognition unit 302 and information such as the position (Position) and size (Size) of the region divided by the region dividing unit 300. The character string information (Data) is described. Metadata associated with the object is described independently of the object (804 to 807). In the metadata, captions and keywords extracted from the text are associated and described. At this time, the metadata number and the object identifier (ID) written in the object area correspond to each other.

  FIG. 14A shows a case where the hierarchization level N = 1. When N = 1, the objects 411 to 413 have different object identifiers (ID = 1, 2, 3), and each refers to individual metadata 804 to 806. On the other hand, FIG. 14B shows a case where the hierarchization level N = 3, in which three objects 411 to 413 have a common object identifier (ID = 1) and refer to one metadata 807. become.

  When searching for an object by keyword, information 804 to 807 given as metadata is searched. Therefore, when the hierarchization level N = 1, a keyword is detected from each metadata, and an image corresponding to the keyword is output as an individual result. On the other hand, when the hierarchization level N = 3, there is only one metadata to be searched, so that a corresponding image is obtained as a search result at the same time. As a result, a plurality of objects can be grouped and handled as if one object is held.

  Finally, an example of searching for an object using the above-described metadata as a search keyword will be described. FIG. 10 shows an example of a viewer for searching for an object. For example, Adobe's Acrobat Reader.

  When data to which metadata is added is opened in the Viewer 600, it is displayed in the document display window 601. When the search term “XX” is entered in the search keyword input field 602, a list of image search results is displayed in the search result display field 603. Here, three objects are hit as search results. Then, when the search result 1 is selected, the object area 411 is displayed as a corresponding portion. The reason why the object area can be searched with the search term “XX” is that the metadata “XX” is associated with the object identifier indicating the object area 411 as described above.

  FIG. 11 is a diagram for explaining the search results based on the hierarchical level (N) and the metadata assigned to each object identifier. FIGS. 11A to 11C show search results and metadata when the keyword “XX” is searched when the hierarchization levels N = 1, N = 2, and N = 3, respectively.

  When the hierarchization level N = 1 (FIG. 11A), three objects are assigned different object identifiers and have the keyword XX. Therefore, three search results are obtained. On the other hand, when the hierarchization level is N = 3 (FIG. 11C), three objects are integrated into one group, and when an image is searched, three are integrated in the document display window 601. Is displayed.

  By setting the hierarchization level small, it is possible to search for images at the detailed level. However, as the number of pages increases, the hit results also increase, and it takes time to find a desired result. On the other hand, although the detailed search cannot be performed by increasing the hierarchization level, the search can be performed in a large unit, so that the number of hits can be greatly reduced even when the number of pages increases. As a result, it is possible to easily find a region including a desired image.

  Also, by increasing the hierarchization level N, duplicate keywords can be deleted, so the file size can be reduced.

(Example 2)
In the first embodiment, the method of automatically calculating the hierarchical level to be grouped according to the input data, and performing the object grouping and metadata extraction / assignment has been described. However, if object grouping or the like is performed automatically, it may be contrary to the user's desired action. Therefore, the second embodiment provides a mechanism that allows the user to freely set the file size and grouping hierarchy level.

  FIG. 12 shows an example of the operation screen 700 in the operation unit 207. The operation screen 700 is provided with a slider bar 701 for setting a file size and a slider bar 702 for setting a hierarchical level. As an initial state, the hierarchization level is set to the minimum (N = 1) so as not to group the objects. The user operates the two slider bars 701 and 702 to perform desired data output settings, and presses an OK button 704, whereby the control unit 206 in the MFP 100 transfers the setting contents to the hierarchical level calculation unit 302. Note that the operation can be canceled by pressing the Cancel button 703. The hierarchization level calculation unit 303 calculates the hierarchization level N according to the received setting content (see FIG. 13).

  Except for the hierarchized level calculation process, the process is the same as that of the first embodiment, and a description thereof will be omitted.

(Example 3)
The present invention can take the form of, for example, a system, apparatus, method, program, or storage medium. Specifically, the present invention may be applied to a system including a plurality of devices, or may be applied to an apparatus including a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the drawing) that realizes the functions of the above-described embodiments is directly or remotely supplied to a system or apparatus. In addition, this includes a case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has a program function, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium for supplying the program include a floppy (registered trademark) disk, a hard disk, and an optical disk. Further, as a recording medium, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), etc. is there.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. Then, the computer program itself of the present invention or a compressed file including an automatic installation function can be supplied by downloading it to a recording medium such as a hard disk from the homepage of the connection destination. It can also be realized by dividing the program code of the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who satisfy predetermined conditions. Let It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Further, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

Claims (6)

An area dividing unit 300 for dividing the input document into areas;
An attribute information giving unit 301 for giving attribute information to the divided area;
A character recognition unit 302 for obtaining character information for a character region;
A hierarchization level calculation unit 303 that calculates a hierarchization level when grouping the objects;
An object identifier generating unit 304 that integrates the objects into groups having the same caption at the hierarchical level and generates an individual identifier for each group;
A metadata extraction unit 305 extracts metadata associated with the object by detecting a caption associated with the object from the input document, and associates the identifier for the object with the metadata and stores the metadata in a storage area.
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the hierarchized level calculation unit 303 can automatically calculate a level for grouping objects according to a data size of the input document.   The image processing apparatus according to claim 1, wherein the hierarchized level calculation unit 303 can change the level at which the objects are grouped in a stepwise manner in accordance with a user setting.   Based on the caption information attached to the object or the logical structure of the chapter or section, the anchor identifier generation unit 304 selects an object including the same or similar expression at the hierarchization level determined by the hierarchization level calculation unit 303. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus recognizes the same group and assigns a unique identifier to each group.   The attribute information analysis unit 301 assigns any of the attributes of chapter, section, text, and caption to the character area, and any of the table, photograph, drawing, and line drawing for the object area other than the character. The image processing apparatus according to claim 1, wherein the attribute is assigned.   The metadata extraction unit 305 searches the text for a sentence or keyword that includes the same or similar expression as the caption attached to the object, and assigns the sentence as metadata to the object. Image processing device.

Images