CN116524525A - Table processing method, device, system and storage medium - Google Patents

Abstract

The present application provides a form processing method, device, system and storage medium, which relate to the field of artificial intelligence. The method comprises: obtaining a source table image; extracting a table area in the source table image by using a histogram statistical method to obtain a pre-extracted table area; detecting whether there is a target attribute in the pre-extracted table area, and extracting from the pre-extracted table area according to the detection result A target table area is screened out from the table area; a text part in the target table area is positioned to obtain a table image for text positioning. The method of the present application realizes accurate identification of forms.

Description

Table processing method, device, system and storage medium

technical field

The present application relates to the field of artificial intelligence, and in particular to a form processing method, device, system and storage medium.

Background technique

The amount of tabular data is huge, and the digital preservation of tabular data is particularly important. The processing of tabular images is an important prerequisite for the digital preservation of tabular data.

In the current existing technology, the processing of form images is mainly aimed at regular forms with complete and continuous form frames, horizontal and vertical frame lines and intersection points, and the segmentation method is used to segment the form, using traditional addition forms or text positioning in the form method to position the text in the table.

However, the inventors have found at least the following technical problems: For images of irregular forms such as incomplete frame lines, the prior art has poor robustness and low accuracy during processing.

Contents of the invention

The present application provides a form processing method, device, system and storage medium to solve the problem that irregular forms cannot be accurately processed.

In a first aspect, the present application provides a form processing method, including:

Get the source table image;

Using the histogram statistical method to extract the form area in the source form image, to obtain the pre-extracted form area;

Detect whether there is a target attribute in the pre-extracted table area, and filter out the target table area from the pre-extracted table area according to the detection result;

Locate the text part in the target table area to get the table image where the text is positioned

In a possible design, the histogram statistical method is used to extract the table area in the source table image, and the pre-extracted table area is obtained, including: performing grayscale binarization processing on the source table image to obtain grayscale binarization processing The image after the grayscale binarization processing is carried out to obtain the image after the statistical processing of the vertical histogram; the image after the statistical processing of the vertical histogram is subjected to the minimum external rectangle detection to determine the size of the table area Horizontal boundary; carry out horizontal histogram statistical processing on the image after grayscale binarization processing, and obtain the image after horizontal histogram statistical processing; perform minimum circumscribed rectangle detection on the image after horizontal histogram statistical processing, and determine the vertical direction of the table area Boundary; determine the rectangle of the table area according to the horizontal boundary and the vertical boundary; filter the rectangle of the table area according to the predefined first filtering rule to obtain the rectangle of the filtered table area; obtain the rectangle of the filtered table area based on the rectangle of the filtered table area from the source In the table image, extract the pre-extracted table area.

In a possible design, after performing horizontal histogram statistical processing on the image after grayscale binarization processing to obtain the image after horizontal histogram statistical processing, it also includes: checking the horizontal histogram through the predefined first expansion kernel The image after the statistical processing of the graph is subjected to horizontal expansion processing to obtain the image after the horizontal expansion processing; correspondingly, the minimum circumscribed rectangle detection is performed on the image after the horizontal histogram statistical processing to determine the vertical boundary of the table area, including: horizontal expansion processing The final image is subjected to minimum enclosing rectangle detection to determine the vertical boundary of the table area.

In a possible design, the target attribute is a horizontal straight line; correspondingly, detecting whether the target attribute exists in the pre-extracted table area, and filtering out the target table area from the pre-extracted table area according to the detection result, including: Perform grayscale binarization on the pre-extracted table area to obtain the gray-scale binarized pre-extracted table area; use the morphological method to extract the horizontal line segment in the gray-scale binarized pre-extracted table area; use straight line extraction The algorithm reconstructs the horizontal line segment into a horizontal straight line; detects whether there is a horizontal line in each pre-extracted table area, and obtains a detection result; determines all pre-extracted table areas whose detection results are yes, as the target table area.

In a possible design, the text part in the target table area is positioned to obtain the table image of the text positioning, including: performing grayscale processing on the target table area to obtain the grayscale processed target table area; Edge detection is performed on the target table area after grayscale processing to obtain the target table area after edge detection; binarization is performed on the target table area after edge detection to obtain a binarized target table area; the binarized target table area is obtained The table area is corroded, and the vertical line in the binarized target table area is removed to obtain the target table area after the vertical line is removed; the target table area after the vertical line is removed according to the predefined second expansion kernel and corrosion check Perform processing to obtain the target table area after corrosion processing; perform expansion processing on the target table area after corrosion processing according to the predefined third expansion kernel to obtain the target table area after expansion processing; find the target table area after expansion processing The rectangular outline of the text is positioned; the rectangular outline is screened according to a predefined second screening rule; the area containing the filtered rectangular outline is extracted from the target table area to obtain a table image of the text positioning.

In a possible design, after locating the text part in the target table area to obtain the text-located table image, the method further includes: identifying the text in the table image, and generating an electronic form according to the text.

In a second aspect, the present application provides a form processing device, including:

The acquisition module is used to acquire the source table image;

The pre-extraction module is used to extract the form area in the source form image by histogram statistics to obtain the pre-extracted form area;

A screening module, configured to detect whether there is a target attribute in the pre-extracted form area, and filter out the target form area from the pre-extracted form area according to the detection result;

The positioning module is used to locate the text part in the target form area, and obtain the form image of the text positioning.

In a possible design, the pre-extraction module is specifically used to: perform grayscale binarization processing on the source table image to obtain the image after grayscale binarization processing; Statistical processing of the histogram to obtain the image after the statistical processing of the vertical histogram; the minimum external rectangle detection is performed on the image after the statistical processing of the vertical histogram to determine the horizontal boundary of the table area; horizontal histogram is performed on the image after the grayscale binarization processing Graph statistical processing, obtain the image after the horizontal histogram statistical processing; Carry out minimum circumscribed rectangle detection to the image after the horizontal histogram statistical processing, determine the vertical boundary of the table area; According to the horizontal boundary and the vertical boundary, determine the rectangle of the table area; According to The predefined first filtering rule filters the rectangle of the table area to obtain the rectangle of the filtered table area; based on the rectangle of the filtered table area, the pre-extracted table area is obtained by extracting from the source table image.

In a possible design, the positioning module is specifically used to: perform grayscale processing on the target table area to obtain the grayscale processed target table area; perform edge detection on the grayscale processed target table area to obtain The target table area after edge detection; binarize the target table area after edge detection to obtain a binarized target table area; corrode the binarized target table area to remove the binarized target table area The vertical line in the vertical line obtains the target table area after removing the vertical line; According to the predefined second expansion kernel and corrosion check, the target table area after removing the vertical line is processed, and the target table area after corrosion processing is obtained; according to the pre-defined The defined third expansion kernel performs expansion processing on the target table area after corrosion processing again to obtain the target table area after expansion processing; find the rectangular outline in the target table area after expansion processing, and position the text; according to the predefined first The second filtering rule is to filter the rectangular outline; extract the area containing the filtered rectangular outline from the target table area, and obtain the form image of the text positioning.

In a third aspect, the application provides a form processing system, including:

A camera for capturing source form images;

The server includes: at least one processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that at least one processor executes the table processing method of the above first aspect and various possible designs of the first aspect.

In the fourth aspect, the present application provides a computer storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above-mentioned first aspect and the table processing method of various possible designs of the first aspect are realized .

In the fifth aspect, the embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the above first aspect and the table processing method of various possible designs of the first aspect can be realized.

The form processing method, device, system and storage medium provided by the present application extract the form from the source form image by using the histogram statistics method, and locate the text part of the extracted target form area, so as to obtain the form image of the text positioning, The identification method does not depend on the frame line of the form, and can also accurately identify the form with incomplete frame lines.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

FIG. 1 is a schematic diagram of an application scenario of a table processing method provided in an embodiment of the present application;

FIG. 2 is a schematic flow chart of a form processing method provided by an embodiment of the present application;

Fig. 3 is the image after statistical processing of the longitudinal histogram provided by the embodiment of the present application;

Fig. 4 is the image after the detection of the minimum outer rectangle provided by the embodiment of the present application;

Fig. 5 is the image after statistical processing of the horizontal histogram provided by the embodiment of the present application;

Fig. 6 is the pre-extracted table area provided by the embodiment of the present application;

Fig. 7 is the output result image of text positioning provided by the embodiment of the present application;

FIG. 8 is an image after lateral expansion processing provided by the embodiment of the present application;

FIG. 9 is an image of the minimum circumscribed rectangle detection provided by the embodiment of the present application;

FIG. 10 is a schematic structural diagram of a form processing device provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application.

By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all It is information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation portals for users to choose to authorize or refuse.

It should be noted that the form processing method and device of the present application can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence, and the application field of the form processing method and device of the present application is not limited.

Aiming at the problem that the recognition accuracy of irregular forms is not high in the form processing process, the embodiment of the present application proposes the following technical solutions: use the histogram statistical method to extract the form area in the source form image, and perform text analysis on the text part in the form area. Positioning, to achieve accurate identification of the form. The following uses detailed embodiments to describe in detail.

FIG. 1 is a schematic diagram of an application scenario of a table recognition method provided by an embodiment of the present application. As shown in FIG. 1 , it includes: a camera 101 and a server 102 . The camera 101 shoots the paper form to obtain an image of the form. The server 102 is used for the form image captured by the camera 101 , and recognizes and enters the form text on the form image.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below in conjunction with the accompanying drawings.

FIG. 2 is a schematic flow chart of a form processing method provided by an embodiment of the present application. The execution subject of this embodiment may be the server in the embodiment shown in FIG. 1 , and this embodiment does not make special limitations here. As shown in Figure 2, the method includes:

S201: Acquire a source table image.

Specifically, the source table image uploaded by the camera is acquired.

S202: Using a histogram statistics method to extract form areas in the source form image to obtain pre-extracted form areas.

Specifically, including S2021~S2027:

S2021: Perform grayscale binarization processing on the source table image to obtain an image after grayscale binarization processing.

Among them, the grayscale binarization is to adjust the variable values of the three primary colors of each pixel in the RGB model to the same value, which is the grayscale value, and then adjust the grayscale value of each pixel in the image to 0 (black) or 255 (white).

Specifically, the variable values of the three primary colors of each pixel in the source table image are adjusted to the same gray value through grayscale binarization, and then the grayscale value of each pixel is adjusted to 0 (black) or 255 ( White), to obtain the grayscale binarized image.

S2022: Perform longitudinal histogram statistical processing on the grayscale binarized image to obtain an image after longitudinal histogram statistical processing.

Among them, the histogram statistics change the gray level of each pixel in the image by counting the brightness distribution of the image area, which is mainly used to enhance the contrast of the image with a small dynamic range.

Specifically, the gray level of each pixel in the image is changed by counting the brightness distribution of each pixel in the gray level binarized image to obtain the image in the longitudinal histogram statistical process.

Exemplarily, (a), (b) and (c) in FIG. 3 are images of three table images subjected to statistical processing of longitudinal histograms, and vertical table lines are obtained.

S2023: Perform minimum outer rectangle detection on the image after statistical processing of the vertical histogram, and determine the horizontal boundary of the table area.

Specifically, contour detection is performed on the image processed by the vertical histogram, a circumscribing rectangle is found, an angle of the rectangle is adjusted, and the smallest circumscribing rectangle is found.

Exemplarily, (a), (b), and (c) in Fig. 4 are the images of the three table images after the minimum outer rectangle detection, and the leftmost and rightmost frame lines of the table area are determined, that is, the table area horizontal border.

S2024: Perform horizontal histogram statistical processing on the grayscale binarized image to obtain an image after horizontal histogram statistical processing.

Specifically, the gray level of each pixel in the image is changed by counting the brightness distribution of each pixel in the gray level binarized image to obtain the image in the horizontal histogram statistical process.

Exemplarily, (a), (b), and (c) in FIG. 5 are images of three table images subjected to statistical processing of horizontal histograms, and horizontal table lines are obtained.

S2025: Perform minimum circumscribed rectangle detection on the image after the statistical processing of the horizontal histogram, and determine the vertical boundary of the table area.

S2026: Determine the rectangle of the table area according to the horizontal boundary and the vertical boundary.

Specifically, the area surrounded by the horizontal boundary and the vertical boundary is determined as a rectangle of the table area.

S2027: Filter the rectangle of the table area according to the predefined first screening rule to obtain the screened rectangle of the table area.

Wherein, the first filtering rule is a rectangle of a table area satisfying preset size standards such as area, height and width.

Exemplarily, the first screening rule is a rectangle that satisfies a table area with an area greater than 20 cm 2 , a height greater than 10 cm, and a width greater than 20 cm.

S2028: Extract the pre-extracted table area from the source table image based on the rectangle of the screened table area.

Specifically, according to the position of the rectangle of the filtered table area, the table is extracted from the same position of the source table image to obtain a pre-extracted table area.

Exemplarily, (a), (b) and (c) in FIG. 6 are the pre-extracted form areas of the three form images respectively.

S203: Detect whether there is a target attribute in the pre-extracted table area, and filter out the target table area from the pre-extracted table area according to the detection result.

In this embodiment, the target attribute may be a horizontal straight line, or more than two vertical straight lines.

In an implementation manner, the target attribute is a horizontal straight line, and S203 specifically includes S2031-S2035:

S2031: Perform gray scale binarization on the pre-extracted table area to obtain a gray scale binarized pre-extracted table area.

S2032: Using a morphological method, extracting horizontal line segments in the pre-extracted table area after grayscale binarization.

Among them, morphology is a nonlinear filtering method, which creates corresponding structural elements through the shape of the target image to extract the target image.

Specifically, by pre-creating structural elements corresponding to the horizontal line segments, the horizontal line segments in the pre-extracted table area of the gray scale binarization are extracted.

S2033: Using a straight line extraction algorithm to reconstruct the horizontal line segment into a horizontal straight line.

Wherein, the straight line extraction algorithm is an algorithm for extracting the straight line according to the gradient of the pixel points in the image.

In this embodiment, the straight line can be extracted using the least significant difference method.

Specifically, the Gaussian kernel is used to sample the input image, the gradient value and gradient direction of each sampling point are calculated, all sampling points are pseudo-sorted according to the gradient value, the states of the sampling points are sorted according to the gradient value, and finally the linear longitudinal coordinate Sorting processing, extreme value clustering mean value processing, to obtain linear coordinates.

S2034: Detect whether there is a horizontal straight line in each pre-extracted table area, and obtain a detection result.

S2035: Determine all pre-extracted table areas whose detection result is yes as target table areas.

S204: Position the text part in the target table area to obtain a text-positioned table image.

Among them, literal includes text and data.

Specifically, including S2041 to S2049, FIG. 7 is an image of the text positioning output result provided by the embodiment of the present application.

S2041: Perform grayscale processing on the target table area to obtain the grayscale processed target table area.

Specifically, the grayscale value of each pixel in the target table area is adjusted to 0 (black) or 255 (white) through grayscale processing to obtain the grayscale processed target table area.

Exemplarily, (a) in FIG. 7 shows the target table area.

S2042: Perform edge detection on the gray-scaled target table area to obtain the edge-detected target table area.

In this embodiment, a Sobel operator may be used to perform edge detection on the grayscaled target table area.

Specifically, the Sobel operator seeks the gradient in the x direction, and the function call uses the following parameters xorder=1, yorder=0, aperture_size=3 to calculate the image difference in the first-order x- or y-direction, that is, Sobel(gray, sobel, CV_8U, 1,0,3).

S2043: Binarize the target table area after the edge detection by the maximum inter-class variance method to obtain a binarized target table area.

Specifically, the front and rear backgrounds are segmented by the threshold to maximize the variance between classes, and the target table area after edge detection is divided into two parts, the background and the foreground, and the binarized target table area is obtained. The binarized target table area is shown in Figure 7 Shown in (b).

S2044: Erosion processing is performed on the binarized target table area, and vertical lines in the binarized target table area are removed to obtain a target table area from which the vertical lines have been removed.

Specifically, the morphological method is used to create structural elements for extracting vertical lines, and the vertical lines in the binarized target table area are removed to obtain the target table area after the vertical lines are removed, as shown in (c) in Figure 7 The vertical lines in the binarized target table area, (d) in FIG. 7 is the target table area after the vertical lines are removed.

S2045: Process the target table area after removing the vertical lines according to the predefined second expansion kernel and corrosion kernel, to obtain the target table area after corrosion processing.

Specifically, by applying morphological calculations, the target form area after removing the vertical lines is corroded and then expanded, and the height is set to control the degree of expansion in the up and down directions to obtain the target form area after corrosion treatment. Figure 7(e) is The target table area expanded by the second expansion kernel highlights the outline of the text. (f) in FIG. 7 is the target table area after corrosion, and some vertical table lines are removed again.

Exemplarily, the predefined dimensions of the second expansion core and corrosion core are (20, 10).

S2046: Perform dilation processing on the corroded target table area again according to the predefined third expansion kernel, to obtain the dilated target table area.

Specifically, according to the predefined second expansion kernel, the corroded target table area is horizontally expanded again to obtain the expanded target table area. The text outline of the expanded target table area is more obvious, and the expanded target table area The table area is shown in (g) in Figure 7.

Exemplarily, the size of the second expansion kernel is (10, 1).

S2047: Find the rectangle outline in the expanded target table area, and position the text.

Specifically, the position information of the text is found according to the position information of the rectangular outline in the target table area after the dilation process.

S2048: Filter the rectangular contours according to a predefined second screening rule.

Wherein, the second screening rule is a rectangular outline satisfying preset size standards such as area, height and width.

Exemplarily, the second screening rule satisfies a rectangular outline with an area less than 10 cm2, a rectangular outline with a height greater than twice the width, a rectangular outline with a height less than 40 cm, and a rectangular outline with a width less than 20 cm.

S2049: Extract an area containing the filtered rectangular outline from the target table area to obtain a text-positioned table image.

Specifically, an area containing a rectangular outline satisfying the second screening rule is extracted from the target table area to obtain a text-located table image, as shown in (h) in FIG. 7 .

To sum up, the table processing method provided in this embodiment extracts the table from the source table image by using the histogram statistics method, and locates the text part of the extracted target table area to obtain the table image with the text positioning. The recognition method does not Relying on the frame line of the table, it can also accurately identify the table with incomplete frame line.

In addition, the table area containing the target attribute is screened out, which reduces the recognition of unnecessary tables and further improves the efficiency and accuracy of table recognition.

In this embodiment of the present application, on the basis of the embodiment provided in FIG. 2 , the specific implementation method of performing horizontal dilation processing on the image after the statistical processing of the horizontal histogram after S2024 is described in detail. The method includes: performing horizontal dilation processing on the image after the statistical processing of the horizontal histogram through a predefined first dilation kernel to obtain the image after the horizontal dilation processing.

Specifically, the image highlight part after the horizontal histogram statistical processing is expanded by the predefined first expansion kernel, the gap between the image highlight parts is reduced, and the image after the horizontal expansion process is obtained. Figure 8 shows the horizontal expansion process after the image.

Exemplarily, the size of the first expansion kernel is set to (20, 10).

Correspondingly, the minimum circumscribed rectangle detection is performed on the image after the statistical processing of the horizontal histogram to determine the vertical boundary of the table area, including:

The minimum circumscribed rectangle detection is performed on the image processed by horizontal dilation to determine the vertical boundary of the table area.

Exemplarily, FIG. 9 is an image of the minimum circumscribed rectangle detection, and the uppermost and rightmost frame lines of the table area are determined, that is, the vertical boundary of the table area.

To sum up, the table processing method provided by this embodiment reduces the gap between the highlighted parts of the image by expanding the highlighted parts of the image, thereby enhancing the accuracy of the minimum rectangle detection and further enhancing the accuracy of table recognition.

In this embodiment of the present application, based on the embodiment provided in FIG. 2 , a specific implementation method for generating an electronic form after S204 is described in detail. The method includes:

Recognize text in table images and generate spreadsheets from the text.

Specifically, an intelligent algorithm is adopted to recognize the text in the form image, and the text is entered into the electronic form according to the form image located by the text.

In this embodiment, the intelligent algorithm may be a template matching algorithm, or a deep learning algorithm.

In summary, the form processing method provided by the embodiment of the present application recognizes the form by identifying the text in the form image, and solves the problems of heavy workload, cumbersome operation and low accuracy in the process of manually entering the electronic form.

FIG. 10 is a schematic structural diagram of a form processing device provided by an embodiment of the present application. As shown in FIG. 10 , the form processing apparatus includes: an acquisition module 1001 , a pre-extraction module 1002 , a screening module 1003 and a positioning module 1004 .

An acquisition module 1001, configured to acquire a source table image;

The pre-extraction module 1002 is used to extract the form area in the source form image by histogram statistics to obtain the pre-extracted form area;

A screening module 1003, configured to detect whether there is a target attribute in the pre-extracted form area, and filter out the target form area from the pre-extracted form area according to the detection result;

The positioning module 1004 is configured to locate the text part in the target form area, and obtain the form image of the text positioning.

In a possible implementation, the pre-extraction module 1002 is specifically configured to: perform grayscale binarization processing on the source table image to obtain an image after grayscale binarization processing; Perform statistical processing of the vertical histogram to obtain the image after the statistical processing of the vertical histogram; perform minimum external rectangle detection on the image after the statistical processing of the vertical histogram to determine the horizontal boundary of the table area; Horizontal histogram statistical processing to obtain the image after horizontal histogram statistical processing; perform minimum circumscribed rectangle detection on the image after horizontal histogram statistical processing to determine the vertical boundary of the table area; determine the rectangle of the table area according to the horizontal boundary and vertical boundary ; Filter the rectangle of the table area according to the predefined first screening rule to obtain the rectangle of the filtered table area; extract the rectangle of the table area based on the filtered table area from the source table image to obtain the pre-extracted table area.

In a possible implementation manner, the pre-extraction module 1002 is further configured to: perform horizontal dilation processing on the image after the statistical processing of the horizontal histogram through a predefined first dilation kernel to obtain the image after the horizontal dilation processing; correspondingly, the The minimum circumscribed rectangle detection is performed on the image after the horizontal histogram statistical processing to determine the vertical boundary of the table area, including: the minimum circumscribed rectangle detection is performed on the image after the horizontal expansion processing to determine the vertical boundary of the table area.

In a possible implementation manner, the screening module 1003 is specifically configured to: the target attribute is a horizontal straight line; correspondingly, detect whether the target attribute exists in the pre-extracted table area, and filter out from the pre-extracted table area according to the detection result The target table area includes: performing grayscale binarization on the pre-extracted table area to obtain the gray-scale binarized pre-extracted table area; using a morphological method to extract the gray-scale binarized pre-extracted table area the horizontal line segment; use the straight line extraction algorithm to reconstruct the horizontal line segment into a horizontal straight line; detect whether there is a horizontal line in each pre-extracted table area, and obtain the detection result; determine all the pre-extracted table areas where the detection result is yes is the target table area.

In a possible implementation manner, the positioning module 1004 is specifically configured to: grayscale the target table area to obtain the grayscaled target table area; perform edge detection on the grayscaled target table area , to obtain the target table area after edge detection; binarize the target table area after edge detection to obtain a binarized target table area; corrode the binarized target table area to remove the binarized target The vertical line in the table area obtains the target table area after the vertical line is removed; according to the predefined second expansion kernel and corrosion check, the target table area after the vertical line is removed is processed to obtain the target table area after the corrosion process; According to the predefined third expansion kernel, the corroded target table area is expanded again to obtain the expanded target table area; the rectangular outline in the expanded target table area is searched for, and the text is positioned; according to the predefined The second screening rule filters the rectangular outline; extracts the area containing the filtered rectangular outline from the target table area, and obtains the text-positioned table image.

In a possible implementation manner, with continued reference to FIG. 10 , the device further includes a generation module 1005 , specifically configured to: recognize text in the form image, and generate an electronic form according to the text.

The device provided in this embodiment can be used to implement the technical solutions of the above method embodiments, and its implementation principle and technical effect are similar, so this embodiment will not repeat them here.

The form processing system provided by the embodiment of this application includes:

A camera 101, configured to collect source form images;

The server 102 is configured to execute the above form processing method.

FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application. As shown in Figure 11, the server in this embodiment includes: a processor 1101 and a memory 1102;

memory 1102, for storing computer-executable instructions;

The processor 1101 is configured to execute the computer-executed instructions stored in the memory, so as to implement the various steps executed by the server in the foregoing embodiments. For details, refer to the related descriptions in the foregoing method embodiments.

Optionally, the memory 1102 can be independent or integrated with the processor 1101 .

When the memory 1102 is set independently, the server further includes a bus 1103 for connecting the memory 1102 and the processor 1101 .

The embodiment of the present application also provides a computer storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above table processing method is implemented.

An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above table processing method is realized. An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above table processing method is realized.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another A system, or some feature, can be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.

A module described as a separate component may or may not be physically separated, and a component shown as a module may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, each module may exist separately physically, or two or more modules may be integrated into one unit. The units formed by the above modules can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) or a processor execute some steps of the methods in various embodiments of the present application.

It should be understood that the above-mentioned processor may be a central processing unit (Central Processing Unit, referred to as CPU), and may also be other general-purpose processors, a digital signal processor (Digital Signal Processor, referred to as DSP), an application specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC) and so on. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in conjunction with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The storage may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage, and may also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

The bus may be an Industry Standard Architecture (Industry Standard Architecture, ISA for short) bus, a Peripheral Component Interconnect (PCI for short) bus, or an Extended Industry Standard Architecture (EISA for short) bus. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The above-mentioned storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable In addition to programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in application specific integrated circuits (Application Specific Integrated Circuits, ASIC for short). Of course, the processor and the storage medium can also exist in the electronic device or the main control device as discrete components.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application . The specification and examples are to be considered exemplary only, with a true scope and spirit of the application indicated by the following claims.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (12)

1. A form processing method, comprising:

acquiring a source form image;

extracting a table region in the source table image by adopting a histogram statistical method to obtain a pre-extracted table region;

detecting whether the target attribute exists in the pre-extracted form area or not, and screening the target form area from the pre-extracted form area according to a detection result;

and positioning the text part in the target table area to obtain a table image with text positioning.

2. The method of claim 1, wherein extracting the table regions in the source table image using histogram statistics results in pre-extracted table regions, comprising:

carrying out gray level binarization processing on the source table image to obtain an image after gray level binarization processing;

performing vertical histogram statistical processing on the image subjected to gray level binarization processing to obtain an image subjected to vertical histogram statistical processing;

Performing minimum external rectangle detection on the image subjected to the vertical histogram statistical processing, and determining the transverse boundary of a table area;

performing horizontal histogram statistical processing on the image subjected to gray level binarization processing to obtain an image subjected to horizontal histogram statistical processing;

performing minimum circumscribed rectangle detection on the image subjected to the transverse histogram statistical processing to determine a longitudinal boundary of a table area;

determining a rectangle of the form area according to the transverse boundary and the longitudinal boundary;

screening the rectangles of the table areas according to a predefined first screening rule to obtain the rectangles of the screened table areas;

and extracting the pre-extracted form area from the source form image based on the rectangle of the screened form area.

3. The method according to claim 2, further comprising, after said subjecting said gray-level binarized image to a lateral histogram statistical process, the step of:

performing transverse expansion processing on the image subjected to the transverse histogram statistical processing through a predefined first expansion check to obtain an image subjected to the transverse expansion processing;

Correspondingly, the detecting the minimum circumscribed rectangle of the image after the statistics processing of the transverse histogram to determine the longitudinal boundary of the table area includes:

and detecting the minimum circumscribed rectangle of the image after the transverse expansion processing, and determining the longitudinal boundary of the table area.

4. The method of claim 1, wherein the target property is a transverse straight line;

correspondingly, the detecting whether the target attribute exists in the pre-extracted form area, and screening the target form area from the pre-extracted form area according to the detection result includes:

performing gray level binarization on the pre-extracted form area to obtain a pre-extracted form area with gray level binarization;

extracting a transverse line segment in the pre-extracted form area of the gray level binarization by adopting a morphological method;

reconstructing the transverse line segment into a transverse straight line by adopting a straight line extraction algorithm;

detecting whether a transverse straight line exists in each pre-extracted table area to obtain a detection result;

and determining all the pre-extracted table areas with the detection result of yes as target table areas.

5. The method of claim 1, wherein locating the text portion in the target form area results in a text located form image, comprising:

Carrying out graying treatment on the target table area to obtain a target table area after graying treatment;

performing edge detection on the target table area subjected to the graying treatment to obtain a target table area subjected to edge detection;

binarizing the target table area after edge detection to obtain a binarized target table area;

performing corrosion treatment on the binarized target table area, and removing vertical lines in the binarized target table area to obtain a target table area with the vertical lines removed;

processing the target form area after the vertical line is removed according to a predefined second expansion core and a corrosion core to obtain a target form area after corrosion processing;

performing expansion treatment again on the target table area after the corrosion treatment according to a predefined third expansion check to obtain the target table area after the expansion treatment;

searching a rectangular outline in the expanded target table area, and positioning the characters;

screening the rectangular outline according to a predefined second screening rule;

and extracting the region containing the screened rectangular outline from the target table region to obtain a table image with text positioning.

6. The method according to any one of claims 1 to 5, further comprising, after said locating the text portion in the target form area to obtain a text located form image:

and recognizing characters in the form image, and generating a spreadsheet according to the characters.

7. A form processing apparatus, comprising:

the acquisition module is used for acquiring the source form image;

the pre-extraction module is used for extracting the table areas in the source table image by adopting a histogram statistical method to obtain pre-extracted table areas;

the screening module is used for detecting whether the target attribute exists in the pre-extracted form area or not, and screening the target form area from the pre-extracted form area according to a detection result;

and the positioning module is used for positioning the text part in the target table area to obtain a table image of text positioning.

8. The apparatus of claim 7, wherein the pre-extraction module is specifically configured to: carrying out gray level binarization processing on the source table image to obtain an image after gray level binarization processing; performing vertical histogram statistical processing on the image subjected to gray level binarization processing to obtain an image subjected to vertical histogram statistical processing; performing minimum external rectangle detection on the image subjected to the vertical histogram statistical processing, and determining the transverse boundary of a table area; performing horizontal histogram statistical processing on the image subjected to gray level binarization processing to obtain an image subjected to horizontal histogram statistical processing; performing minimum circumscribed rectangle detection on the image subjected to the transverse histogram statistical processing to determine a longitudinal boundary of a table area; determining a rectangle of the form area according to the transverse boundary and the longitudinal boundary; screening the rectangles of the table areas according to a predefined first screening rule to obtain the rectangles of the screened table areas; and extracting the pre-extracted form area from the source form image based on the rectangle of the screened form area.

9. The apparatus of claim 8, wherein the positioning module is specifically configured to: carrying out graying treatment on the target table area to obtain a target table area after graying treatment; performing edge detection on the target table area subjected to the graying treatment to obtain a target table area subjected to edge detection; binarizing the target table area after edge detection to obtain a binarized target table area; performing corrosion treatment on the binarized target table area, and removing vertical lines in the binarized target table area to obtain a target table area with the vertical lines removed; processing the target form area after the vertical line is removed according to a predefined second expansion core and a corrosion core to obtain a target form area after corrosion processing; performing expansion treatment again on the target table area after the corrosion treatment according to a predefined third expansion check to obtain the target table area after the expansion treatment; searching a rectangular outline in the expanded target table area, and positioning the characters; screening the rectangular outline according to a predefined second screening rule; and extracting the region containing the screened rectangular outline from the target table region to obtain a table image with text positioning.

10. A form processing system, comprising:

the camera is used for acquiring a source form image;

the server comprises: at least one processor and memory;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 6.

11. A computer storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any of claims 1 to 6.

12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the form processing method of any one of claims 1 to 6.