CN115546815A - Table identification method, device, equipment and storage medium - Google Patents

Abstract

The present application provides a form recognition method, device, equipment, and storage medium. The specific implementation plan is: detecting and determining the form area from the image to be tested; determining the corresponding text feature and position feature based on the form area; The feature is fused with the position feature to obtain a table feature; and the table feature is used to determine the table recognition result in the image to be tested. According to the technical solution of the present application, the form can be accurately identified.

Description

Form recognition method, device, equipment and storage medium

technical field

The present application relates to the technical field of artificial intelligence, and in particular to a form recognition method, device, equipment and storage medium.

Background technique

With the development of information technology, the importance of spreadsheets is beyond doubt. However, the tables faced in the process of business processing are not just simple Excel and Word documents, but also many table pictures in the form of PDF scans and pictures.

Usually, only the cell detection method is used to perform table recognition on pictures with tables, and the recognized tables are inaccurate, and manual post-adjustment of the recognized tables is required.

Contents of the invention

In order to solve the above problems, the present application proposes a form recognition method, device, equipment and storage medium, which can effectively improve the accuracy of form recognition.

According to the first aspect of the embodiments of the present application, a form identification method is provided, including:

Detect and determine the table area from the image to be tested;

determining corresponding text features and location features based on the table area;

Fusing the text feature and the position feature to obtain a form feature;

Using the table features to determine a table recognition result in the image to be tested.

According to a second aspect of the embodiments of the present application, a form recognition device is provided, including:

Determining a module, used to detect and determine the table area from the image to be tested;

A processing module, configured to determine corresponding text features and location features based on the table area;

A fusion module, used to fuse the text features and the position features to obtain table features;

A recognition module, configured to use the table features to determine a table recognition result in the image to be tested.

The third aspect of the present application provides an electronic device, including:

memory and processor;

The memory is connected to the processor for storing programs;

The processor realizes the above table recognition method by running the program in the memory.

The fourth aspect of the present application provides a storage medium, on which a computer program is stored, and when the computer program is run by a processor, the above table recognition method is realized.

An embodiment in the above application has the following advantages or beneficial effects:

When the technical solution proposed by this application recognizes a form from an image to be detected, the corresponding text feature and position feature are determined based on the form area in the image to be tested, and the text feature and position feature are fused to obtain the form feature, that is, multiple The dimension extracts the features of the table, so that the table in the image to be tested can be identified more accurately by using the table feature, so the recognition accuracy of the table can be improved.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present application, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.

FIG. 1 is a schematic flow chart of a form recognition method provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a full-line table provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a wireless watch provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a few-line table provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of cell division provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of text line division provided by an embodiment of the present application;

FIG. 7 is a schematic flowchart of a form recognition method provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of division of cells and text lines provided by an embodiment of the present application;

FIG. 9 is a specific schematic diagram of feature fusion provided by an embodiment of the present application;

FIG. 10 is a specific schematic diagram of form identification provided by an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a form recognition device provided by an embodiment of the present application;

FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

The technical solutions of the embodiments of the present application are applicable to various forms recognition scenarios, for example, smart office scenarios, text processing, and the like.

At present, with the increasing maturity of artificial intelligence-related technologies, smart office has become the demand of most users. Most of the table recognition only uses the cell detection method to process the picture with the table, and this method has a poor recognition effect on the table with few lines.

Therefore, it is necessary to provide a form identification method, device, equipment and storage medium, which can identify forms more accurately.

The technical solutions of the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware devices execute the processing process of the technical solutions of the embodiments of the present application, Or when the above software program is running, the text features and position features determined by the table area in the image to be tested are used to fuse to obtain the table feature, so as to achieve the purpose of using the table feature to generate the table in the image to be tested. The embodiment of the application only exemplifies the specific processing process of the technical solution of the application, and does not limit the specific implementation form of the technical solution of the application. Any technical implementation form that can execute the processing process of the technical solution of the application can be used by this application. Used in the application examples.

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

exemplary method

Fig. 1 is a flow chart of a form recognition method according to an embodiment of the present application. In an exemplary embodiment, the form identification method specifically includes:

S110. Detect and determine the table area from the image to be tested;

S120. Determine corresponding text features and location features based on the table area;

S130. Fusing the text feature and the position feature to obtain a table feature;

S140. Determine a table recognition result in the image to be tested by using the table features.

In step S110, for example, the image to be tested refers to an image containing a table, wherein the table is composed of one or more rows of cells, and the table is used to display numbers and other items, which is convenient for quick reference and analysis. Items are organized into rows and columns. As shown in Figure 2-4, there are various types of tables, which can be divided into wired tables and wireless tables according to the presence or absence of borders. Among them, wired tables can be divided into few-line tables (such as three-line tables) and full-line tables. Optionally, the image to be tested may be a screenshot of an article or a webpage, or an image obtained by taking a photo. The table area refers to the area where the table is located in the image to be tested. Optionally, the table area may be the entire table in the image to be tested, or a part of the table in the image to be tested, for example, the area where a row of the table is located, the area where each cell is located, and so on. Specifically, object detection or instance segmentation can be used to detect the image to be tested to obtain the table area. For example, YOLO (You Only Look Once), Faster RCNN (Faster Regions with CNNfeatures), Generative Adversarial Network (GAN).

In step S120, for example, the text feature is used to represent text information in the table. Optionally, the text features may include: text semantic features and text format features. Optionally, text semantic features are used to represent the meaning of the text. The text format feature is used to represent the visual information of the text. For example, the text format feature may include text color, text font size, text font, whether it is bold, whether it is underlined, font distribution, whether the font has a background color, etc. The location feature is used to represent the location of table regions. For example, coordinates, information about rows in a table, information about columns in a table, etc.

Optionally, when the table area is the entire table, the text features correspond to the text features of all texts in the entire table, and the position features correspond to the position features of the entire table. When the table area is the area where a row of tables is located, the text feature is the text feature corresponding to each row of the table in the entire table, and the location feature is the location feature corresponding to each row of the table in the entire table. In this way, there are several rows in the entire table, there is Several text features and location features. When the table area is the area where each cell is located, the text feature is the text feature corresponding to each cell in the entire table, and the position feature is the position feature corresponding to each cell in the entire table. In this way, how many cell, there are several text features and location features.

Specifically, the neural network can be trained according to table images, text features and location features, so as to obtain a trained neural network model. In this way, the text feature and position feature can be output by identifying the table area according to the trained neural network model. It is also possible to input the table area into a pre-trained text recognition model to obtain text features, and input the table area to a pre-trained position recognition model to obtain position features, thereby determining the text features and position features. The text recognition model can be a convolutional recurrent neural network (Convolutional RecurrentNeural Network, CRNN) model, and the position recognition model can be a U-net model, or any model that can recognize coordinates in an image, which is not limited here.

In step S130, for example, the table feature is used to reflect the location of the table and the specific content in the table. Optionally, the table features can be obtained by concatenating text features and position features. Optionally, it is also possible to splice text features and position features to obtain fusion features, and input the fusion features to an encoder or a neural network (for example, a feedforward neural network model (Convolutional Neural Network, CNN), a recurrent neural network (Recurrent Neural Network) NeuralNetworks, RNN)) get tabular features.

In step S140, for example, the table recognition result is used to represent the table structure and/or table content in the image to be tested. Optionally, the table features can be processed according to a pre-trained neural network to obtain a table recognition result. It is also possible to combine the form features according to the position information in the form features to obtain the form recognition result.

In the technical solution of the present application, when identifying a form from an image to be detected, the corresponding text feature and position feature are determined based on the form area in the image to be tested, and the text feature and position feature are fused to obtain the form feature, that is, from Feature extraction is performed on tables in multiple dimensions, so that the table features in the image to be tested can be used to more accurately identify the table in the image to be tested, so the recognition accuracy of the table can be improved.

In one embodiment, determining corresponding text features and location features based on the table area includes:

Determining text features and location features corresponding to each table element in the table area; the table elements include cells and/or text rows.

Exemplarily, the table area may include a plurality of table elements, and the table elements include cells and/or text rows. Among them, the cells are divided according to the lines in the table. For example, when the table is a full-line table, the cells are determined according to the row-column grid lines; when the table is a three-line table, the cells are determined according to the upper and lower row grid lines. Wherein, the text line can be a single character, vocabulary, phrase, sentence, etc., and the text line can also be a text composed of words and symbols, which is not limited here. Specifically, both cells and text lines can be determined through object detection.

Optionally, when the table elements include cells or text rows, determining the text features and position features corresponding to each table element in the table area includes:

Determining cell detection results or text line detection results respectively in the table area;

Determining corresponding text features and location features based on the cell detection result;

Or, determine the corresponding text feature and position feature based on the text line detection result.

In this embodiment, as shown in Figure 5, when the table element includes a cell, the table area is detected, that is, two cell detection results are determined according to the row table line, and the upper cell in Figure 5 is called The detection result of the first cell, the following cells are called the detection result of the second cell. Perform text recognition and position recognition on the detection result of the first cell to obtain the text feature and position feature corresponding to the detection result of the first cell, and perform text recognition and position recognition on the detection result of the second cell to obtain the detection result of the second cell Corresponding text features and location features. It can be understood that the text feature of the first cell detection result includes: Model type, Acc, BLEU-2, BLEU-4 features, and the location feature of the first cell detection result is the location information corresponding to the first cell , the text feature and position feature corresponding to the second cell detection result are the same. In this way, the positional relationship among the cells in the table and the textual relationship in the cells can be accurately determined, so that the table can be recognized more accurately.

In this embodiment, as shown in Figure 6, when the table element includes a text row, the table area is detected, that is, twelve text row detection results are determined according to the text content, and the text is determined for each text row detection result. Recognition and position recognition to obtain the text features and position features corresponding to each text line detection result. For example, determine the corresponding text features and location features for Model type, and determine the corresponding text features and location features for Vision only. By repeating this, twelve text features and twelve location features can be determined. In this way, the positional relationship between each text line in the table and the meaning of the text line can be accurately determined, so that the table can be recognized more accurately.

Preferably, as shown in Figure 7, in the case where the table elements include cells and text rows, the determination of the text features and position features corresponding to each table element in the table area includes:

S710. Determine cell detection results and text line detection results respectively in the table area;

S720. Determine a corresponding text feature and position feature based on the cell detection result;

S730. Determine corresponding text features and position features based on the text line detection result.

In this embodiment, as shown in Figure 8, when a table element includes a cell and a text row, the table area can be detected for the table element (token), specifically, the table area can be detected using Faster RCNN to determine the form Cell detection results and text line detection results in ranges. Perform text recognition and location recognition for each cell detection result, and obtain text features and location features corresponding to each cell detection result. For example, if there are Modeltype, Acc, BLEU-2, and BLEU-4 in the cell, then the text feature corresponding to the cell detection result is the feature containing Model type, Acc, BLEU-2, and BLEU-4. The position feature of the cell detection result is the position information corresponding to the cell, for example, the position of the center point of the cell or the position of the vertex of the cell.

For each text line detection result, it is determined to perform text recognition and position recognition, and the text feature and position feature corresponding to each text line detection result are obtained. For example, determine the corresponding text feature and location feature for Model type, and determine the corresponding text feature and location feature for Acc. By repeating this, multiple text features and location features can be determined. In this way, not only the positional relationship between the cells and the textual relationship in the cell can be determined, but also the positional relationship between each text line and the meaning of the text line can be determined, and the cell and text relationship in the multi-line table can be determined from the bottom up. relationship between lines of text, allowing for more accurate identification of lines-less tables.

In one embodiment, when the table element includes a cell, the fusion of the text feature and the position feature to obtain the table feature includes:

For each cell, the text feature and position feature corresponding to the cell are concatenated to obtain the table feature corresponding to the cell.

Exemplarily, each cell is recognized according to the text recognition model to obtain the content in the cell. If it is recognized that the cell contains multiple contents, the multiple contents can be spliced to obtain the text feature corresponding to the cell. According to the position identification model, the position information of each cell is identified to obtain the position feature. The text features and position features are concatenated to obtain the table features to realize the fusion of multi-modal features, so that the table features can reflect the characteristics of multiple dimensions.

In one embodiment, when the table element includes a text row, the fusion of the text feature and the position feature to obtain the table feature includes:

For each text line, the text feature and position feature corresponding to the text line are spliced to obtain the table feature corresponding to the text line.

Exemplarily, each text line is identified according to the text recognition model, and text features corresponding to each text line are obtained. According to the position recognition model, the position information of each text line is recognized to obtain the position feature. The text features and position features are concatenated to obtain the table features to realize the fusion of multi-modal features, so that the table features can reflect the characteristics of multiple dimensions.

In one embodiment, the text features include: text format features and semantic features;

Correspondingly, the fusion of the text features and the position features to obtain table features includes:

A table feature is obtained by fusing the text format feature, the semantic feature and the position feature.

Exemplarily, the table region can be identified according to the trained neural network model, and text format features, semantic features and position features can be output. It is also possible to input the table area into a pre-trained visual classification model to obtain text format features, input the table area to a pre-trained text recognition model to obtain semantic features, and input the table area to a pre-trained position recognition model to obtain location features , so as to determine the text format features, semantic features and location features.

In this embodiment, as shown in FIG. 9 , for each table element in the table area, CNN is used to extract text format features, and BERT (Bidirectional Encoder Representations from Transformers) is used to extract semantic features. Determine location characteristics. The above-mentioned text format features, semantic features and position features are spliced to obtain the table features corresponding to each table element, so that each table feature is a multi-dimensional feature, so that the table can be represented in multiple modes. In this way, when multiple table elements are included in the table area, multiple table features can be correspondingly obtained.

In one embodiment, the determining the table recognition result in the image to be tested by using the table features includes:

Determining the prediction sequence of the form by the form feature;

The table topology in the image to be tested is determined according to the prediction sequence.

Exemplarily, the prediction sequence may be a sequence defined by hypertext markup language, that is, a sequence of html annotations. It may be to input table features into a preset classification model to obtain a table prediction sequence in the image to be tested; wherein, the classification model is trained according to training data, and optionally, the training data is a table area of an image sample The tabular features in and the sequence labels corresponding to the tabular regions. The table feature in the table area is used as the input of the classification model, and the sequence label corresponding to the table area is used as the target of the classification model to train the classification model. Specifically, the classification model may adopt a transformer model.

Specifically, the table features are input into a preset classification model to output a prediction sequence, and the rows and columns of the table in the image to be tested can be determined according to the prediction series, thereby determining the topological structure of the table. In this way, the table structure recognition is transformed into sequence prediction, which increases the accuracy of table recognition. Furthermore, text features can also be added to the table topology to supplement the content in the table.

In this embodiment, as shown in FIG. 10 , token detection is first performed on the table area to determine cell detection results and text line detection results. Determine the text format feature, semantic feature and position feature corresponding to each cell detection result and text line detection result. For each cell detection result, the text format feature, semantic feature and position feature are spliced to obtain the table feature corresponding to each cell detection result. For each text line detection result, the text format feature, semantic feature and position feature are spliced to obtain the table feature corresponding to each text line detection result. In this way, bottom-up multimodal tabular features are obtained. Then input the above table features into the transformer model, and output the prediction sequence marked by html. According to the prediction sequence marked in html, the topological structure of the table can be directly output to realize top-down table recognition, and the structure of the table can be accurately recognized even in the face of a few-line table.

Exemplary device

Correspondingly, FIG. 11 is a schematic structural diagram of a form recognition device according to an embodiment of the present application. In an exemplary embodiment, the embodiment of the present application also proposes a form recognition device, which includes:

A determining module 1110, configured to detect and determine the table area from the image to be tested;

A processing module 1120, configured to determine corresponding text features and location features based on the table area;

A fusion module 1130, configured to fuse the text features and the location features to obtain table features;

A recognition module 1140, configured to determine a table recognition result in the image to be tested by using the table features.

In one embodiment, the processing module 1120 is also used to:

Determining text features and location features corresponding to each table element in the table area; the table elements include cells and/or text rows.

In one embodiment, in the case where the table elements include cells and text rows, the determining the text features and position features corresponding to each table element in the table area includes:

Determine the cell detection result and the text line detection result respectively in the table area;

Determining corresponding text features and location features based on the cell detection result;

The corresponding text feature and position feature are determined based on the text line detection result.

In one embodiment, when the table element includes a cell, the fusion module 1130 is further configured to:

For each cell, the text feature and position feature corresponding to the cell are concatenated to obtain the table feature corresponding to the cell.

In one embodiment, when the form element includes a text row, the fusion module 1130 is further configured to:

For each text line, the text feature and position feature corresponding to the text line are spliced to obtain the table feature corresponding to the text line.

In one embodiment, the text features include: text format features and semantic features;

Correspondingly, the pair of fusion modules 1130 is also used for;

A table feature is obtained by fusing the text format feature, the semantic feature and the position feature.

In one embodiment, the identification module 1140 is also used to:

Determining the prediction sequence of the form by the form feature;

The table topology in the image to be tested is determined according to the prediction sequence.

The form recognition device provided in this embodiment belongs to the same application concept as the form recognition method provided in the above-mentioned embodiments of the present application, can execute the form recognition method provided in any of the above-mentioned embodiments of the present application, and has corresponding functional modules for executing the form recognition method and beneficial effects. For technical details that are not described in detail in this embodiment, refer to the specific processing content of the table recognition method provided in the above embodiments of the present application, and will not be repeated here.

Exemplary electronic device

Another embodiment of the present application also proposes an electronic device, as shown in FIG. 12 , the device includes:

memory 1200 and processor 1210;

Wherein, the memory 1200 is connected to the processor 1210 for storing programs;

The processor 1210 is configured to execute the program stored in the memory 1200 to implement the form recognition method disclosed in any of the above embodiments.

Specifically, the above electronic device may further include: a bus, a communication interface 1220 , an input device 1230 and an output device 240 .

The processor 1210, the memory 1200, the communication interface 1220, the input device 1230, and the output device 1240 are connected to each other through a bus. in:

A bus may include a pathway that carries information between various components of a computer system.

The processor 1210 may be a general processor, such as a general central processing unit (CPU), a microprocessor, etc., or may be an application-specific integrated circuit (ASIC), or one or more integrated circuit for program execution. It can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

The processor 1210 may include a main processor, and may also include a baseband chip, a modem, and the like.

The memory 1200 stores the program for executing the technical solution of the present invention, and may also store the operating system and other key services. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 1200 may include read-only memory (read-only memory, ROM), other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), that can store information and Other types of dynamic storage devices, disk storage, flash, etc. for instructions.

The input device 1230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer or a gravity sensor, and the like.

Output devices 1240 may include devices that allow information to be output to a user, such as a display screen, printer, speakers, and the like.

Communication interface 1220 may include the use of any transceiver-like means to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), and the like.

The processor 1210 executes the program stored in the memory 1200, and calls other devices, which can be used to implement each step of any form recognition method provided by the above-mentioned embodiments of the present application.

Exemplary computer program product and storage medium

In addition to the above-mentioned methods and devices, embodiments of the present application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the above-mentioned "exemplary method" of this specification. The steps in the table recognition method according to various embodiments of the application described in the section.

The computer program product can be written in any combination of one or more programming languages to execute the program codes for performing the operations of the embodiments of the present application, and the programming languages include object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.

In addition, the embodiment of the present application may also be a storage medium, on which a computer program is stored, and the computer program is executed by a processor in the table recognition method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification. A step of.

For the specific working content of the above-mentioned electronic device, and the specific working content of the above-mentioned computer program product and the computer program on the storage medium being run by the processor, please refer to the content of the above-mentioned method embodiment, and will not be repeated here.

For the aforementioned method embodiments, for the sake of simple description, they are expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the described action sequence, because according to the application, Certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions and modules involved are not necessarily required by this application.

It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same and similar parts in each embodiment, refer to each other, that is, Can. As for the device-type embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to part of the description of the method embodiments.

The steps in the method of each embodiment of the present application can be adjusted, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

The modules and submodules in the devices and terminals in the various embodiments of the present application can be combined, divided and deleted according to actual needs.

In the several embodiments provided in this application, it should be understood that the disclosed terminal, device and method may be implemented in other ways. For example, the terminal embodiments described above are only illustrative. For example, the division of modules or sub-modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple sub-modules or modules can be combined Or it can be integrated into another module, or some features can be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.

The modules or sub-modules described as separate components may or may not be physically separated, and the components as modules or sub-modules may or may not be physical modules or sub-modules, that is, they may be located in one place, or may also be distributed to on multiple network modules or submodules. Part or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, each module or submodule may exist separately physically, or two or more modules or submodules may be integrated in one processing module. in a module. The above-mentioned integrated modules or sub-modules can be implemented in the form of hardware or in the form of software function modules or sub-modules.

Professionals can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the possible For interchangeability, in the above description, the composition and steps of each example have been generally described according to their functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be directly implemented by hardware, software units executed by a processor, or a combination of both. The software unit can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

Finally, it should also be noted that in this text, relational terms such as first and second etc. are only used to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A form identification method, characterized in that, comprising: Detect and determine the table area from the image to be tested; determining corresponding text features and location features based on the table area; Fusing the text feature and the position feature to obtain a table feature; Using the table features to determine a table recognition result in the image to be tested. 2. The method according to claim 1, wherein said determining corresponding text features and location features based on said table area comprises: Determining text features and location features corresponding to each table element in the table area; the table elements include cells and/or text rows. 3. The method according to claim 2, wherein, in the case where the form elements include cells and text rows, the determination of the text features and position features corresponding to each form element in the form area ,include: Determine the cell detection result and the text line detection result respectively in the table area; Determining corresponding text features and location features based on the cell detection result; The corresponding text feature and position feature are determined based on the text line detection result. 4. The method according to claim 2, wherein, in the case where the form element comprises a cell, the fusion of the text feature and the position feature to obtain the form feature comprises: For each cell, the text feature and position feature corresponding to the cell are concatenated to obtain the table feature corresponding to the cell. 5. The method according to claim 2, wherein, when the form element includes a text row, the fusion of the text feature and the position feature to obtain the form feature includes: For each text line, the text feature and position feature corresponding to the text line are spliced to obtain the table feature corresponding to the text line. 6. The method according to claim 1, wherein the text features include: text format features and semantic features; Correspondingly, the fusion of the text features and the position features to obtain table features includes: A table feature is obtained by fusing the text format feature, the semantic feature and the position feature. 7. The method according to claim 1, wherein said utilizing said form feature to determine the form recognition result in said image to be tested comprises: Determining the prediction sequence of the form by the form feature; The table topology in the image to be tested is determined according to the prediction sequence. 8. A form recognition device, characterized in that, comprising: Determining a module, used to detect and determine the table area from the image to be tested; A processing module, configured to determine corresponding text features and location features based on the table area; A fusion module, used to fuse the text features and the position features to obtain table features; A recognition module, configured to use the table features to determine a table recognition result in the image to be tested. 9. An electronic device, characterized in that it comprises: memory and processor; The memory is connected to the processor for storing programs; The processor implements the form recognition method according to any one of claims 1 to 7 by running the program in the memory. 10. A storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is run by a processor, the form recognition method according to any one of claims 1 to 7 is realized.

Images