CN117011877A - Financial contract auditing method and device, electronic equipment and storage medium - Google Patents

Abstract

The application provides a financial contract auditing method, a financial contract auditing device, electronic equipment and a storage medium, which comprise the steps of determining a pre-trained target segmentation model according to a template type corresponding to a financial contract to be audited; inputting the to-be-signed financial contract and the signed financial contract into a target segmentation model in a picture format respectively to output text segmentation information corresponding to the to-be-signed financial contract and the signed financial contract respectively; inputting text segmentation screenshots corresponding to segmentation screenshots of the financial contract to be signed and the signed financial contract respectively into a pre-trained optical character recognition model so as to output text recognition results corresponding to each text segmentation screenshot; and performing text matching on text recognition results corresponding to text segmentation screenshots with the same text category between the to-be-signed financial contract and the signed financial contract to determine whether the to-be-signed financial contract and the signed financial contract meet the contract signing standard.

Description

A method, device, electronic equipment and storage medium for auditing financial contracts

Technical field

This application relates to the field of data processing technology, specifically to a financial contract review method, device, electronic equipment and storage medium.

Background technique

A contract is a document that transactions in all walks of life take place. Due to the rich usage scenarios, a large amount of contract text data is generated in the physical world. The large amount of contract text increases the time required for contract review and also lengthens the waiting period for customers. Most solutions in the industry are to directly recognize the text of the contract text, then match the text lines, and compare the texts to confirm whether the contract has been modified.

However, when using the above method to compare contract texts, interlaced and wrong lines often occur, which ultimately leads to deviations in the comparison results, affects the review progress, and increases the workload of manual secondary review. Therefore, a contract review method with more accurate review results and higher efficiency is needed.

Contents of the invention

In view of this, the purpose of this application is to provide a financial contract review method, device, electronic equipment and storage medium to improve the efficiency and accuracy of contract review.

First, this application provides a method for reviewing financial contracts. The method includes determining a pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes both parties to the contract. The financial contract to be signed prepared by one party and the signed financial contract fed back by the other party to the contract; the financial contract to be signed and the financial contract signed are input into the target segmentation model in the format of pictures to output the financial contract to be signed. The corresponding text segmentation information of the financial contract and the signed financial contract. The text segmentation information at least includes text segmentation screenshots and the text category corresponding to each text segmentation screenshot segmentation screenshot; separate the financial contract to be signed and the signed financial contract. The text segmentation screenshots are input to the pre-trained optical character recognition model to output the text recognition results corresponding to each text segmentation screenshot; the text segmentation screenshots corresponding to the same text category between the financial contract to be signed and the financial contract that has been signed are The text recognition results are text matched to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards.

Preferably, the text segmentation information also includes the vertex coordinates of each text segmentation screenshot, and the text recognition results corresponding to the text segmentation screenshots with the same text category between the financial contract to be signed and the financial contract that has been signed are text matched to determine Before determining whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards, it also includes: For any of the financial contract to be signed and the financial contract that has been signed, split screenshots based on all texts of the contract Vertex coordinates, sort all the text-split screenshots, and combine the text-split screenshots of a financial contract to be signed and the text-split screenshots of a signed financial contract with the same serial number into a screenshot data group; for each screenshot data group, Match the text categories of the two text-segmented screenshots in the screenshot data group; if the text categories between each screenshot data group are matched successfully, then perform the text matching step; if there are groups whose text categories do not match, then Mark the two text-split screenshots within this group.

Preferably, the text recognition results include multiple fields and the coordinates of the recognition frame of each field, and the text recognition results corresponding to the text segmentation screenshots of the same text category between the financial contract to be signed and the financial contract that has been signed are text matched, so as to The steps to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards include: for any text segmentation screenshot, the field in the text recognition result corresponding to the text segmentation screenshot, according to the identification frame coordinates of the field Arrange and connect to obtain the text in the text segmentation screenshot; match the text of two text segmentation screenshots with the same text category word for word; if the words between the texts of each group of text segmentation screenshots with the same text category are If they match, it is determined that the financial contract to be signed and the financial contract that has been signed meet the contract signing standards, otherwise they do not meet the contract signing standards.

Preferably, if the words in each group of text segmented screenshots of the same text category do not completely match, mark the two text segmented screenshots; and generate a financial contract review based on the labeling of all text segmented screenshots. Report and output.

Preferably, the target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit and a detection unit, wherein the first feature The input of the extraction unit is used as the input of the target segmentation model, the output of the first feature extraction unit is connected to the input of the second feature extraction unit, the output of the second feature extraction unit is connected to the input of the third feature extraction unit, and the third feature extraction unit The output of the region generation network unit is connected to the input of the region generation network unit. The output of the region generation network unit is connected to the input of the fourth feature extraction unit. The output of the region generation network unit is also connected to the input of the segmentation unit. The output of the segmentation unit serves as the target seam model. The first output is used to output text segmentation screenshots. The output of the fourth feature extraction unit is connected to the input of the classification unit. The output of the classification unit is used as the second output of the target seam model and is used to output the text category corresponding to the text segmentation screenshots. The output of the four feature extraction units is also connected to the input of the detection unit. The output of the detection unit is used as the third output of the target seam model and is used to output the vertex coordinates of the text segmentation screenshot.

Preferably, each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit includes a plurality of sequentially connected attention sub-units, each attention sub-unit It includes multiple single-head attention blocks, and each single-head attention block includes multiple attention layers, pooling layers, direct connection layers and fully connected layers, where the number of attention layers in different feature extraction units is different.

Preferably, the target segmentation model is trained and generated in the following manner: for each training sample, the annotated training sample is input into the initial segmentation model to be trained to respectively determine the first loss value and segmentation unit corresponding to the output of the region generation network unit. The second loss value corresponding to the output of the classification unit, the third loss value corresponding to the output of the detection unit, and the fourth loss value corresponding to the output of the detection unit; combine the first loss value, the second loss value, the third loss value, the fourth loss value The sum between the values is used as the total loss value;

The parameters of the initial segmentation model are adjusted based on the total loss value to generate the target segmentation model.

Preferably, for each screenshot data group, the text category is matched in the following manner: determine whether the text categories of the two text-segmented screenshots in the screenshot data group are the same; if they are the same, determine that the screenshot data group is matched successfully; if If they are not the same, it is determined that the screenshot data group does not match.

Preferably, the marking situation includes text part matching and text category mismatch, and the financial contract review report is generated in the following way: when the text category does not match, the financial contract to be signed and the signed financial contract are completely added to the financial In the page of the contract review report; when the text does not match, screenshots of the mismatched text between the financial contract to be signed and the signed financial contract are added to the page of the financial contract review report respectively.

Preferably, when the text category mismatch and the text mismatch exist at the same time, the financial contract to be signed and the signed financial contract are completely added to the page of the financial contract review report, and the financial contract that is marked as a text mismatch is added in its entirety. The corresponding vertex coordinates of the text segmentation screenshot are generated, and the first screenshot frame is generated and added to the upper layer of the financial contract to be signed and the signed financial contract to generate a financial contract review report.

Preferably, for each group of text segmentation screenshots with the same text category, when there is mismatched text between the two, the vertex coordinates of the text recognition frame are determined and recorded.

Preferably, when the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation graph to generate a financial contract review report.

In the second aspect, this application provides a financial contract review device, which includes:

The pre-selection module is used to determine the pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes the financial contract to be signed drafted by one of the parties to the contract and the other party to the contract. A signed financial contract as reported by one party;

The segmentation module is used to input the financial contract to be signed and the financial contract that has been signed into the target segmentation model in the format of pictures to output the text segmentation information corresponding to the financial contract to be signed and the financial contract that has been signed. The text segmentation information At least include text segmentation screenshots and text categories corresponding to each text segmentation screenshot segmentation screenshot;

The recognition module is used to input the text segmentation screenshots corresponding to the respective segmentation screenshots of the financial contract to be signed and the signed financial contract into the pre-trained optical character recognition model to output the text recognition results corresponding to each text segmentation screenshot;

The verification module is used to match the text recognition results corresponding to the text segmentation screenshots of the same text category between the financial contract to be signed and the financial contract that has been signed, to determine whether the financial contract to be signed and the financial contract that has been signed meet the requirements. Contract signing standards.

In a third aspect, this application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. When the machine-readable instructions are executed by the processor, the steps of the above-mentioned method for auditing a financial contract are performed.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run by a processor, the computer program executes the above-mentioned steps of a financial contract review method. .

This application provides a method, device, electronic device and storage medium for reviewing financial contracts, including determining a pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The review of financial contracts includes the financial contract to be signed drafted by one of the two parties to the contract and the signed financial contract reported by the other party; input the financial contract to be signed and the signed financial contract in picture format respectively. The target segmentation model is to output text segmentation information corresponding to the financial contract to be signed and the financial contract that has been signed. The text segmentation information at least includes text segmentation screenshots and the text category corresponding to each text segmentation screenshot segmentation screenshot; The text segmentation screenshots corresponding to the respective segmentation screenshots of the contract and the signed financial contract are input into the pre-trained optical character recognition model to output the text recognition results corresponding to each text segmentation screenshot; the financial contract to be signed and the signed financial contract are The text recognition results corresponding to the text segmentation screenshots of the same text category between contracts are text matched to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards. By first segmenting the contract by area, and then extracting text from each segmented area, and then matching the text at the regional level, this reduces the probability of serialization in the text recognition process and improves the efficiency of contract review. and accuracy.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, preferred embodiments are given below and described in detail with reference to the attached drawings.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore do not It should be regarded as a limitation of the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a financial contract review method provided by the embodiment of this application;

Figure 2 is a flow chart of the comparison steps of text segmentation screenshots provided by the embodiment of the present application;

Figure 3 is a flow chart of the comparison steps of text recognition results provided by an embodiment of the present application;

Figure 4 is a schematic structural diagram of a financial contract review device provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only These are part of the embodiments of this application, but not all of them. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the application provided in the appended drawings is not intended to limit the scope of the claimed application, but rather to represent selected embodiments of the application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without any creative work shall fall within the scope of protection of this application.

First, the applicable application scenarios of this application are introduced. This application can be applied to the review of contracts before and after they are signed during the transaction.

A contract is a document that transactions in all walks of life take place. Due to the rich usage scenarios, a large amount of contract text data is generated in the physical world. The large amount of contract text increases the time required for contract review and also lengthens the waiting period for customers. Most solutions in the industry are to directly recognize the text of the contract text, then match the text lines, and compare the texts to confirm whether the contract has been modified. However, when using the above method to compare contract texts, interlaced and wrong lines often occur, which ultimately leads to deviations in the comparison results, affects the review progress, and increases the workload of manual secondary review. Therefore, a contract review method with more accurate review results and higher efficiency is needed.

Based on this, embodiments of this application provide a financial contract review method, device, electronic device and storage medium to improve the efficiency and accuracy of contract review.

Please refer to Figure 1, which is a flow chart of a financial contract review method provided by an embodiment of the present application. As shown in Figure 1, the embodiment of this application provides a method for auditing financial contracts, including:

S101. Determine the pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes the financial contract to be signed drafted by one of the two parties to the contract and the financial contract to be signed by the other party to the contract. The financial contract has been signed.

The financial contracts here can be transactions in financial fields such as wealth management, securities, and banks, payment agreements, transaction contracts, insurance policies, etc. generated during the payment process. It can also be a transaction contract signed in other fields. The financial contract to be signed here is a contract drawn up by one party to the transaction and transferred to the other party and requires the other party to sign and confirm. The signed financial contract here is a contract in which the other party to the transaction receives the financial contract to be signed from one party in the transaction, completes the signing, and feeds it back to the other party. The content of the financial contract to be signed and the financial contract that has been signed should be completely consistent. However, in order to prevent the content of the contract from being modified, the two financial contracts need to be proofread. The financial contracts here can be in electronic form or paper, but they need to be converted or scanned into image form before processing.

It is understandable that for different types of transactions, the financial contracts here usually have fixed templates, and the formats of the contracts are mostly similar. Therefore, for financial contracts of the same format, the corresponding segmentation can be trained. Model. Therefore, in step S101, before reviewing the financial contract, the corresponding model can be selected based on the template of the financial contract to ensure the accuracy of segmentation.

S102. Input the financial contract to be signed and the financial contract that has been signed into the target segmentation model in the format of pictures to output text segmentation information corresponding to the financial contract to be signed and the financial contract that has been signed. The text segmentation information at least includes text. Split screenshots and text categories for each split screenshot.

The segmentation model here can be segmented according to the section division and text content of the financial contract. For example, the text categories can be title, information about Party A and Party B, content column, list, terms and seals, etc. The text categories here can be determined based on the content of the financial contract.

Each text segmentation screenshot is an area corresponding to the text category taken from the financial contract.

S103. Input the text segmentation screenshots of the financial contract to be signed and the financial contract that has been signed into the pre-trained optical character recognition model to output the text recognition results corresponding to each text segmentation screenshot.

In step S103, an optical character recognition (OCR, Optical Character Recognition) model can be used to perform text extraction on each text-segmented screenshot to obtain all text in the screenshot.

S104. Perform text matching on the text recognition results corresponding to the text segmentation screenshots of the same text category between the financial contract to be signed and the financial contract that has been signed, to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards. .

Specifically, for each screenshot data group, text category matching is performed in the following manner: determining whether the text categories of the two text-segmented screenshots in the screenshot data group are the same. If they are the same, it is determined that the screenshot data group matches successfully; if they are not the same, it is determined that the screenshot data group does not match.

For example, the first text recognition result corresponding to the title extracted from the financial contract to be signed and the second text recognition result corresponding to the title extracted from the signed financial contract are matched word for word, and the two text recognition results are determined. Whether the titles of each contract are completely consistent, and so on, according to the layout order of the financial contract, compare the text in the information, content column, list, terms and seals of Party A and Party B in order to complete the review of the financial contract and determine Whether there are any changes in the content between the financial contract to be signed and the financial contract that has been signed. For the unmatched parts, the corresponding text can be divided into screenshots and sorted out, and an audit report can be generated uniformly to facilitate secondary review, or it can be directly proofread manually.

The method for auditing financial contracts provided by the embodiment of this application first divides the contract into regions, then extracts text from each divided region, and then matches the text at the regional level, thus reducing the need for text recognition. The probability of serialization in the process improves the efficiency and accuracy of contract review.

Please refer to Figure 2. Figure 2 is a flow chart of comparison steps of text segmentation screenshots provided by an embodiment of the present application. As shown in Figure 2, the text segmentation information also includes the vertex coordinates of each text segmentation screenshot. The text recognition results corresponding to the text segmentation screenshots with the same text category between the financial contract to be signed and the financial contract that has been signed are text recognition results. Before matching to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards, you can also compare the text split screenshots in the following ways:

S201. For any contract among the financial contract to be signed and the financial contract that has been signed, based on the vertex coordinates of all the text-divided screenshots of the contract, sort all the text-divided screenshots, and sort the financial contract to be signed with the same serial number. The text-segmented screenshots and the text-segmented screenshots of a signed financial contract are combined into a screenshot data group.

It can be understood that the text-divided screenshots are sorted according to their layout positions in the financial contract. For example, the text-divided screenshots can be sorted in the order of title, information of Party A and Party B, content column, list, terms and seals, where, For a financial contract, the number of text segmentation screenshots corresponding to each text category is at least one. For example, the information of Party A and Party B can be divided into one text segmentation screenshot, or the information of Party A can be divided into one text segmentation screenshot. , Party B's information is correspondingly divided into another text-divided screenshot, but there is also a fixed layout position between the two text-divided screenshots. Therefore, the order between the two text-divided screenshots is also fixed. For example, the information of Party A and Party B can be arranged in the order of Party A's information and Party B's information.

S202. For each screenshot data group, perform text category matching on the two text-divided screenshots in the screenshot data group.

S203. If the text categories between each screenshot data group are matched successfully, execute the text matching step.

For example, for the sequence of text segmentation screenshots of the financial contract to be signed and the financial contract that has been signed, the text segmentation screenshot of the first order in the financial contract to be signed is compared with the text segmentation screenshot of the first order in the financial contract that has been signed. Match the text categories of the text segmentation screenshots in sequence. If the text categories corresponding to the two text segmentation screenshots are consistent, for example, both are titles, then it is determined that this group of matches is successful. If all screenshot data groups are matched successfully, you can proceed. Step S104.

S204. If there is a group whose text categories do not match, mark the two text segmentation screenshots in the group.

For the two text-segmented screenshots in the screenshot data group with mismatched text categories, the two text-segmented screenshots need to be marked. They can be marked as the screenshot data group with mismatched text categories to facilitate subsequent manual review.

For example, the first text recognition result corresponding to the title extracted from the financial contract to be signed and the second text recognition result corresponding to the title extracted from the signed financial contract are matched word for word, and the two text recognition results are determined. Whether the titles of each contract are completely consistent, and so on, the corresponding third text recognition results from the information of Party A and Party B extracted from the financial contract to be signed and the corresponding third text recognition results from the information of Party A and Party B extracted from the signed financial contract The fourth text recognition results are matched word by word, and so on.

Figure 3 is a flow chart of the comparison steps of text recognition results provided by an embodiment of the present application. In one embodiment of the present application, the text recognition result includes multiple fields and the coordinates of the recognition frame of each field, and the text recognition corresponding to the text segmentation screenshots with the same text category between the financial contract to be signed and the financial contract that has been signed is The steps of text matching the results to determine whether the financial contract to be signed and the financial contract that has been signed meet the contract signing standards include:

S301. For any text segmentation screenshot, arrange and connect the fields in the text recognition results corresponding to the text segmentation screenshot according to the coordinates of the recognition frames of the fields to obtain the text in the text segmentation screenshot.

For multiple fields identified in any text split screenshot, they first need to be spliced in the order they are arranged in the screenshot to obtain a complete text paragraph.

S302. Match the text of two text segmentation screenshots with the same text category word by word.

S303. If the words in the texts of each group of text segmentation screenshots of the same text category match, it is determined that the financial contract to be signed and the financial contract that has been signed meet the contract signing standards, otherwise the contract signing standards are not met.

Here you need to compare whether the words in each sequence are exactly the same. If they are exactly the same, it can be determined that the contract signing standards are met, indicating that the contract content has not been modified between the financial contract to be signed and the financial contract that has been signed.

S304. If the words between the texts of each group of text segmentation screenshots of the same text category do not completely match, mark the two text segmentation screenshots.

Furthermore, a financial contract review report can be generated and output based on the annotation of all text segmented screenshots.

Screenshots of all marked texts can be sorted and generated to generate a financial contract review report. Staff can clearly determine whether there have been modifications between two financial contracts, conduct a secondary review, and discover problems in a timely manner to avoid losses.

In an embodiment of this application, a financial contract audit report can be generated through the following solutions:

First, the annotation situation includes text part matching and text category mismatch, and a financial contract audit report is generated in the following way: when the text category does not match, the financial contract to be signed and the signed financial contract are completely added to the financial contract. In the page of the audit report; when the text does not match, screenshots of the mismatched text between the financial contract to be signed and the signed financial contract are added to the page of the financial contract audit report respectively.

When text category mismatch and text mismatch exist at the same time, the financial contract to be signed and the financial contract that has been signed are completely added to the page of the financial contract review report, and the text corresponding to the text that is marked as text mismatch is added. Split the vertex coordinates of the screenshot, generate the first screenshot frame and add it to the upper layer of the financial contract to be signed and the signed financial contract to generate a financial contract review report.

In addition to financial contracts or screenshots, the report should also at least indicate specific error matters, such as which text category failed to match, or which text failed to match, etc.

Further, for each group of text segmentation screenshots with the same text category, when there is mismatched text between the two, the vertex coordinates of the text recognition frame are determined and recorded.

When the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation graph to generate a financial contract review report.

In this way, when the auditors get the financial contract audit report for secondary verification, they can see at a glance the points where the matching failed, thereby improving the overall efficiency of the audit.

In one embodiment of the present application, a segmentation model corresponding to the template type is constructed in the following manner:

The first is the selection of the initial segmentation model. Here, the Swin backbone network is selected as the prototype for the construction of the three branches of segmentation, detection, and classification.

The target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit and a detection unit, wherein the first feature extraction unit As input to the target segmentation model, the output of the first feature extraction unit is connected to the input of the second feature extraction unit, the output of the second feature extraction unit is connected to the input of the third feature extraction unit, and the output of the third feature extraction unit is connected to The input of the region generation network unit is connected, the output of the region generation network unit is connected with the input of the fourth feature extraction unit, the output of the region generation network unit is also connected with the input of the segmentation unit, and the output of the segmentation unit serves as the first output of the target seam model , used to output text segmentation screenshots, the output of the fourth feature extraction unit is connected to the input of the classification unit, the output of the classification unit is used as the second output of the target seam model, and is used to output the text category corresponding to the text segmentation screenshots, the fourth feature extraction The output of the unit is also connected to the input of the detection unit. The output of the detection unit is used as the third output of the target seam model and is used to output the vertex coordinates of the text segmentation screenshot.

Each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit includes a plurality of sequentially connected attention sub-units, and each attention sub-unit includes a plurality of Single-head attention block, each single-head attention block includes multiple attention layers, pooling layers, direct connection layers (Short cut) and fully connected layers, where the number of attention layers in different feature extraction units is different.

The attention layer here consists of key, query and value matrix operations. Multiple single-head attention blocks form an attention sub-unit with a multi-head attention mechanism. Among them, in the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit, the number of attention layers in each single-head attention block is in accordance with 3:6:6:9. Scale settings.

Based on MaskRCNN, a Region Proposal Networks (RPN) unit is added between the third feature extraction unit and the fourth feature extraction unit. The region generation network unit here consists of a convolution layer, an activation layer and a pooling layer. .

On the basis of the feature extraction backbone network, three parallel piplines are set up to realize the identification and detection of contract content. Among them, the feature vector output by the third feature extraction unit passes through the area generation network unit (outputs the coordinates of the detection frame ), and finally the corresponding text seam screenshot is cut out by the segmentation unit. The segmentation unit here is composed of a block structure composed of resnet (including convolution layer, pooling layer, activation layer and short cut layer) and deconvolution layer. Finally, the features are mapped into k-dimensional screenshots of image size through a 1x1 convolution.

The feature vector output by the fourth feature extraction unit is classified by the classification unit, thereby obtaining the text category of the corresponding text segmentation screenshot. The classification unit here uses MLP (Multi-Layer Perceptron, multi-layer perceptron).

The structure of the detection unit is similar to that of the classification unit. The detection unit is used to output the vertex coordinates of the text segmentation screenshot. The vertex coordinates here are used to indicate the relative position of the text segmentation screenshot in the corresponding financial contract picture.

In one embodiment of this application, the target segmentation model is trained in the following manner:

Step 1: For each training sample, input the labeled training sample into the initial segmentation model to be trained to determine the first loss value corresponding to the output of the region generation network unit, the second loss value corresponding to the output of the segmentation unit, and The third loss value corresponding to the output of the classification unit, and the fourth loss value corresponding to the output of the detection unit;

Step 2: Take the sum of the first loss value, the second loss value, the third loss value, and the fourth loss value as the total loss value;

Step 3: Adjust parameters of the initial segmentation model based on the total loss value to generate a target segmentation model.

Then, based on the two-stage segmentation model training method of the transformer (attention mechanism) backbone network, task definition, loss function design and iterative training are carried out:

Determine appropriate training samples. The training samples here are financial contracts with a unified template format. Financial contracts are different from other contracts in that they have stable structural characteristics. After analyzing the collected information, they can be divided into titles, information about Party A and Party B, content columns, lists, terms and seals, etc.

The collected training samples are annotated on the cloud annotation platform according to the detection frame, text segmentation area and corresponding text category, and divided into training set, verification set and test set to form a complete data set.

For the design part of the loss function, the loss function contains four parts. The output of the RPN layer and the real frame are calculated to correspond to the MSE Loss (the first loss value). The output of the detection layer and the real frame are also calculated to correspond to the MSE Loss (the second loss value). Classification The CE Loss (the third loss value) is obtained from the layer and the real category. The output of the segmentation layer calculates the FocalLoss (the fourth loss value) according to the pixel points and the real MASK. Finally, these four losses are combined to form the total loss.

The learning rate is set to 1e-3, the batch_size is set to 64, and the resolution is set to 1024 (the short sides are filled in according to the padding form). The optimizer uses the adam optimizer, and iteratively trains according to the gradient descent algorithm until the loss range of the model is very small. If it is small, the trained segmentation model is obtained.

Based on the same inventive concept, the embodiments of the present application also provide a financial contract review device corresponding to the financial contract review method. Since the problem-solving principle of the device in the embodiments of the present application is consistent with the above-mentioned financial contract in the embodiments of the present application, The audit methods are similar, so the implementation of the device can be referred to the implementation of the method, and repeated details will not be repeated.

Please refer to FIG. 4 , which is a schematic structural diagram of a financial contract review device provided by an embodiment of the present application. As shown in Figure 4, the audit device 400 includes:

The pre-selection module 410 is used to determine the pre-trained target segmentation model based on the template type corresponding to the financial contract to be reviewed. The financial contract to be reviewed includes the financial contract to be signed drafted by one of the two parties to the contract and the financial contract to be signed by both parties to the contract. The signed financial contract as reported by the other party;

The segmentation module 420 is used to input the financial contract to be signed and the financial contract that has been signed into the target segmentation model in the format of pictures, so as to output the text segmentation information corresponding to the financial contract to be signed and the financial contract that has been signed. Text segmentation The information at least includes text segmentation screenshots and text categories corresponding to each text segmentation screenshot;

The recognition module 430 is used to input the text segmentation screenshots corresponding to the respective segmentation screenshots of the financial contract to be signed and the signed financial contract into the pre-trained optical character recognition model to output the text recognition results corresponding to each text segmentation screenshot;

The verification module 440 is used to perform text matching on the text recognition results corresponding to the text segmentation screenshots of the same text category between the financial contract to be signed and the financial contract that has been signed, to determine whether the financial contract to be signed and the financial contract that have been signed are Meet contract signing criteria.

In a preferred embodiment, the text segmentation information also includes the vertex coordinates of each text segmentation screenshot, and also includes a matching module (not shown in the figure) for targeting any of the financial contracts to be signed and the financial contracts that have been signed. Contract, based on the vertex coordinates of all text-split screenshots of the contract, sort all the text-split screenshots, and combine the text-split screenshots of a financial contract to be signed and the text-split screenshots of a signed financial contract with the same serial number into one Screenshot data group; for each screenshot data group, match the text categories of the two text-divided screenshots in the screenshot data group; if the text categories between each screenshot data group are matched successfully, perform the text matching step ; If there is a group with mismatched text categories, mark the two text-split screenshots in the group.

In a preferred embodiment, the text recognition result includes multiple fields and the recognition frame coordinates of each field. The verification module 440 is specifically configured to segment any text screenshot and segment the field in the text recognition result corresponding to the text segmentation screenshot. Arrange and connect according to the coordinates of the recognition frame of the field to obtain the text in the text segmentation screenshot; match the text of the two text segmentation screenshots with the same text category word for word; if each group of text categories has the same text segmentation screenshot, If the words between them all match, it is determined that the financial contract to be signed and the financial contract that has been signed meet the contract signing standards, otherwise they do not meet the contract signing standards.

In a preferred embodiment, the checking module 440 is also used to mark the two text segmentation screenshots if the words between the texts of each group of text segmentation screenshots of the same text category do not completely match; and according to all texts Divide the annotation status of the screenshot, generate a financial contract review report and output it.

In a preferred embodiment, the target segmentation model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit and a detection unit, where , the input of the first feature extraction unit is used as the input of the target segmentation model, the output of the first feature extraction unit is connected with the input of the second feature extraction unit, the output of the second feature extraction unit is connected with the input of the third feature extraction unit, the The output of the three feature extraction units is connected to the input of the region generation network unit, the output of the region generation network unit is connected to the input of the fourth feature extraction unit, the output of the region generation network unit is also connected to the input of the segmentation unit, and the output of the segmentation unit is as The first output of the target seam model is used to output text segmentation screenshots. The output of the fourth feature extraction unit is connected to the input of the classification unit. The output of the classification unit is used as the second output of the target seam model and is used to output the corresponding text segmentation screenshots. For the text category, the output of the fourth feature extraction unit is also connected to the input of the detection unit. The output of the detection unit is used as the third output of the target seam model and is used to output the vertex coordinates of the text segmentation screenshot.

In a preferred embodiment, each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit includes a plurality of sequentially connected attention sub-units, each of which The attention sub-unit includes multiple single-head attention blocks. Each single-head attention block includes multiple attention layers, pooling layers, direct connection layers and fully connected layers. Among them, the attention layers in different feature extraction units Quantities vary.

In a preferred embodiment, it also includes a training module (not shown in the figure), which is used to train and generate the target segmentation model in the following manner: for each training sample, input the labeled training sample into the initial segmentation model to be trained, To respectively determine the first loss value corresponding to the output of the region generation network unit, the second loss value corresponding to the output of the segmentation unit, the third loss value corresponding to the output of the classification unit, and the fourth loss value corresponding to the output of the detection unit; The sum of the first loss value, the second loss value, the third loss value, and the fourth loss value is used as the total loss value; parameters of the initial segmentation model are adjusted based on the total loss value to generate a target segmentation model.

In a preferred embodiment, for each screenshot data group, the matching module performs text category matching in the following manner: determine whether the text categories of the two text-segmented screenshots in the screenshot data group are the same; if they are the same, determine whether the screenshot The data group matches successfully; if they are not the same, it is determined that the screenshot data group does not match.

In a preferred embodiment, the marked situations include text part matching and text category mismatch, and the financial contract review report is generated in the following manner: when the text category does not match, complete the financial contract to be signed and the signed financial contract. Added to the page of the financial contract review report; when the text does not match, screenshots of the mismatched text segments between the financial contract to be signed and the signed financial contract are added to the page of the financial contract review report respectively.

In a preferred embodiment, when text category mismatch and text mismatch exist at the same time, the financial contract to be signed and the financial contract that has been signed are completely added to the page of the financial contract review report, and the financial contract is marked as If the text does not match the vertex coordinates of the corresponding text segmentation screenshot, the first screenshot frame is generated and added to the upper layer of the financial contract to be signed and the signed financial contract to generate a financial contract review report.

In a preferred embodiment, for each group of text segmentation screenshots with the same text category, when there is mismatched text between the two, the vertex coordinates of the text recognition frame are determined and recorded.

In a preferred embodiment, when the text does not match, a second screenshot frame is generated based on the vertex coordinates of the text recognition frame and added to the upper layer of the corresponding text segmentation graph to generate a financial contract review report.

Please refer to FIG. 5 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 5 , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate through the bus 530, and the machine-readable instructions are When the processor 510 is executed, the steps of the financial contract review method in the method embodiment shown in the above-mentioned method embodiment shown in FIG. 1 can be executed. For specific implementation methods, please refer to the method embodiment, which will not be described again here.

An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run by a processor, it can execute the financial contract in the method embodiment shown in Figure 1. For the steps of the audit method, please refer to the method embodiments for specific implementation methods, and will not be described again here.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code.

Finally, it should be noted that the above-mentioned embodiments are only specific implementation modes of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although refer to the foregoing The embodiments describe the present application in detail. Those of ordinary skill in the art should understand that any person familiar with the technical field can still modify the technical solutions recorded in the foregoing embodiments within the technical scope disclosed in the present application. It is possible to easily think of changes, or to make equivalent substitutions for some of the technical features; and these modifications, changes or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and they should all be covered by this application. within the scope of protection. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (15)

1. A method of auditing financial contracts, the method comprising:

determining a pre-trained target segmentation model according to the template type corresponding to the financial contract to be checked, wherein the financial contract to be checked comprises a financial contract to be signed, which is drawn by one of the two parties of the contract, and a signed financial contract fed back by the other party of the two parties of the contract;

inputting the financial contract to be signed and the signed financial contract into the target segmentation model in a picture format respectively to output text segmentation information corresponding to the financial contract to be signed and the signed financial contract, wherein the text segmentation information at least comprises text segmentation screenshots and text categories corresponding to each text segmentation screenshot;

inputting respective text segmentation screenshots of the financial contract to be signed and the signed financial contract to a pre-trained optical character recognition model so as to output a text recognition result corresponding to each text segmentation screenshot;

and performing text matching on text recognition results corresponding to text segmentation screenshots with the same text category between the financial contract to be signed and the signed financial contract to determine whether the financial contract to be signed and the signed financial contract meet the contract signing standard.

2. The method of claim 1, wherein the text segmentation information further includes vertex coordinates for each text segmentation screenshot, and wherein prior to the step of performing text matching of text recognition results corresponding to text segmentation shots having the same text category between the to-be-signed financial contract and the signed financial contract to determine whether the to-be-signed financial contract and the signed financial contract meet contract signing criteria, further comprising:

aiming at any contract in the to-be-signed financial contract and the signed financial contract, sequencing all text segmentation screenshots based on vertex coordinates of all text segmentation screenshots of the contract, and combining the text segmentation screenshots of one to-be-signed financial contract and the text segmentation screenshots of one signed financial contract with the same serial numbers into a screenshot data set;

aiming at each screenshot data group, matching text categories of two text segmentation screenshots in the screenshot data group;

if the text category between each screenshot data group is successfully matched, executing a text matching step;

if a group with the text category not matched exists, marking two text segmentation screenshots in the group.

3. The method according to claim 2, wherein the text recognition result includes a plurality of fields and recognition frame coordinates of each field, and the step of text matching text recognition results corresponding to text segmentation shots having the same text category between the to-be-signed fund contract and the signed fund contract to determine whether the to-be-signed fund contract and the signed fund contract satisfy a contract signing standard specifically includes:

aiming at any text segmentation screenshot, arranging and connecting fields in a text recognition result corresponding to the text segmentation screenshot according to the recognition frame coordinates of the fields to obtain texts in the text segmentation screenshot;

performing word-by-word matching on texts of the text segmentation screenshots with the same two text categories;

if the words between the texts of the text segmentation screenshots with the same text category in each group are matched, determining that the financial contract to be signed and the signed financial contract meet the contract signing standard, or else, not meeting the contract signing standard.

4. A method according to claim 3, wherein if the words between the text of the text segmentation shots of the same text category of each group are not perfectly matched, the two text segmentation shots are marked;

And generating and outputting a financial contract auditing report according to the labeling conditions of all the text segmentation screenshots.

5. The method of claim 1, wherein the object segmentation model comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a region generation network unit, a segmentation unit, a classification unit, and a detection unit,

the input of the first feature extraction unit is used as the input of the target segmentation model, the output of the first feature extraction unit is connected with the input of the second feature extraction unit, the output of the second feature extraction unit is connected with the input of the third feature extraction unit, the output of the third feature extraction unit is connected with the input of the region generation network unit, the output of the region generation network unit is connected with the input of the fourth feature extraction unit, the output of the region generation network unit is further connected with the input of the segmentation unit, the output of the segmentation unit is used as the first output of the target seam model and used for outputting a text segmentation screenshot, the output of the fourth feature extraction unit is connected with the input of the classification unit, the output of the classification unit is used as the second output of the target seam model and used for outputting a text category corresponding to the text segmentation screenshot, the output of the fourth feature extraction unit is further connected with the input of the detection unit, and the output of the detection unit is used as the third output of the target seam model and used for outputting the vertex coordinates of the text segmentation screenshot.

6. The method of claim 5, wherein each of the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit comprises a plurality of sequentially connected attention sub-units, each attention sub-unit comprises a plurality of single-head attention blocks, each single-head attention block comprises a plurality of attention layers, a pooling layer, a direct connection layer, and a full connection layer, wherein the number of attention layers in different feature extraction units is different.

7. The method of claim 5, wherein the generating the target segmentation model is trained by:

inputting the marked training samples into an initial segmentation model to be trained for each training sample, so as to respectively determine a first loss value corresponding to the output of the regional generation network unit, a second loss value corresponding to the output of the segmentation unit, a third loss value corresponding to the output of the classification unit and a fourth loss value corresponding to the output of the detection unit;

taking the sum of the first loss value, the second loss value, the third loss value and the fourth loss value as a total loss value;

and carrying out parameter adjustment on the initial segmentation model based on the total loss value to generate a target segmentation model.

8. The method of claim 2, wherein for each screenshot data set, matching of text categories is performed by: determining whether text categories of two text segmentation screenshots in the screenshot data group are the same;

if the screenshot data sets are the same, determining that the screenshot data sets are successfully matched;

if not, the screenshot data sets are determined to be not matched.

9. The method of claim 4, wherein labeling includes text portion matches and text category mismatches, and wherein the financial contract audit report is generated by:

when the text categories are not matched, the financial contract to be signed and the signed financial contract are completely added into a page of a financial contract audit report;

and when the texts are not matched, respectively adding the text segmentation screenshot of the mismatch between the financial contract to be signed and the signed financial contract into a page of a financial contract audit report.

10. The method of claim 9, wherein when a text category mismatch exists together with a text mismatch, the to-be-signed financial contract and the signed financial contract are added in their entirety in a page of a financial contract audit report, and a first screenshot box is generated and added at an upper layer of the to-be-signed financial contract and the signed financial contract according to vertex coordinates of text segmentation shots corresponding to the text mismatch marked to generate a financial contract audit report.

11. The method of claim 10, wherein for each set of text-segmentation shots of the same text category, when there is a mismatch between the text, vertex coordinates of a text recognition box are determined and recorded.

12. The method of claim 10, wherein when text does not match, generating a second screenshot box according to vertex coordinates of the text recognition box and adding the second screenshot box to an upper layer of a corresponding text segmentation map to generate a financial contract audit report.

13. An audit device for financial contracts, said device comprising:

the pre-selection module is used for determining a pre-trained target segmentation model according to the template type corresponding to the financial contract to be checked, wherein the financial contract to be checked comprises a financial contract to be checked drawn by one of the two parties of the contract and a signed financial contract fed back by the other party of the two parties of the contract;

the segmentation module is used for inputting the financial contract to be signed and the signed financial contract into the target segmentation model in a picture format respectively so as to output text segmentation information corresponding to the financial contract to be signed and the signed financial contract respectively, wherein the text segmentation information at least comprises text segmentation screenshots and text categories corresponding to each text segmentation screenshot;

The recognition module is used for inputting text segmentation screenshots corresponding to the segmentation screenshots of the financial contract to be signed and the signed financial contract respectively into a pre-trained optical character recognition model so as to output text recognition results corresponding to each text segmentation screenshot;

and the checking module is used for carrying out text matching on text recognition results corresponding to text segmentation screenshots with the same text category between the financial contract to be signed and the signed financial contract so as to determine whether the financial contract to be signed and the signed financial contract meet the contract signing standard.

14. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating over the bus when the electronic device is running, said processor executing said machine readable instructions to perform the steps of the method of auditing a financial contract according to any of claims 1 to 12.

15. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, which computer program, when being executed by a processor, performs the steps of the method of auditing financial contracts according to any one of claims 1 to 12.