CN111832423A - A kind of bill information identification method, device and system - Google Patents

Abstract

Embodiments of the present invention provide a method, device and system for identifying bill information. The scheme is as follows: receiving an information identification request for the bill image to be identified; preprocessing the bill image to be identified; using a pre-trained bill information recognition model to identify bill information in the preprocessed bill image to be identified to obtain the bill image to be identified The bill information; wherein, the bill information recognition model is obtained by training according to the preset training set, the preset training set includes a plurality of sample bill images of different bill types, and sample bill information of each sample bill image; The first response message of the request, where the first response message includes the ticket information of the ticket image to be recognized. The technical solutions provided by the embodiments of the present invention effectively solve the problems existing in the existing bill information identification methods, thereby reducing the dependence on the quantity of human resources and improving the generality of the bill identification methods.

Description

A kind of bill information identification method, device and system

technical field

The present invention relates to the technical field of image recognition, in particular to a method, device and system for recognizing bill information.

Background technique

In production and life, there are various forms of bills. For example, bills can include drafts, promissory notes, checks, bills of lading, certificates of deposit, etc. Each bill contains a large amount of bill information. Taking the invoice in the bill as an example, the invoice may include bill information such as the invoice code, the invoice number, the billing date, and the buyer's name.

At present, when recognizing the bill information contained in the above-mentioned bills, the commonly used bill recognition methods include manual identification and traditional Optical Character Recognition (OCR) technology. Among them, the traditional OCR technology identifies each character in the bill image to determine the bill information contained in the bill image, that is, the bill image corresponds to the bill information contained in the bill. An important part of the traditional OCR technology is character recognition, that is, to match each character in the bill image with each character in the preset candidate template, identify the characters contained in the bill image, or extract the characters in the bill image. Character feature information, and use a Support Vector Machine (SVM) to identify the characters contained in the bill image according to the character feature information.

For the above manual identification method, due to the limitation of the number of human resources, this method will not be suitable for scenarios with a large number of bills. For the traditional OCR technology, the preset candidate templates and character feature information used in the character recognition process have certain limitations, that is, the preset candidate templates limit the bill types of the recognized bills, and the extraction of character feature information limits the available options. The number of characters recognized, which would make conventional OCR techniques less versatile.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a bill information identification method, device and system to solve the problems existing in the existing bill information identification methods, thereby reducing the dependence on the quantity of human resources and improving the versatility of the bill identification method. The specific technical solutions are as follows:

An embodiment of the present invention provides a method for identifying bill information, which is applied to an OCR device in a bill information identifying system, and the method includes:

Receive an information identification request for the image of the ticket to be identified;

Based on the information identification request, preprocessing the to-be-identified bill image to obtain a pre-processed to-be-identified bill image;

Using the pre-trained bill information recognition model to identify the bill information in the preprocessed bill image to be identified, to obtain bill information of the bill image to be identified; wherein the bill information identification model is based on preset training The preset training set includes a plurality of sample bill images of different bill types, and sample bill information of each sample bill image;

A first response message to the information identification request is returned, where the first response message includes bill information of the bill image to be identified.

The embodiment of the present invention also provides a bill information identification device, which is applied to OCR equipment in a bill information identification system, and the device includes:

The receiving module is used for receiving the information identification request of the bill image to be identified;

a preprocessing module, configured to preprocess the bill image to be identified based on the information identification request to obtain a preprocessed bill image to be identified;

The first identification module is used to identify the bill information in the preprocessed bill image to be identified by using a pre-trained bill information identification model to obtain bill information of the bill image to be identified; wherein the bill information The recognition model is obtained by training according to a preset training set, and the preset training set includes a plurality of sample bill images of different bill types, and sample bill information of each sample bill image;

A first return module, configured to return a first response message to the information identification request, where the first response message includes bill information of the bill image to be identified.

The embodiment of the present invention also provides a bill information identification system, the bill information identification system includes an OCR device and a storage device;

The OCR device is used for receiving an information identification request for the bill image to be identified; based on the information identification request, preprocessing the bill image to be identified to obtain a preprocessed bill image to be identified; using the pre-trained bill image A bill information recognition model, which identifies bill information in the preprocessed bill image to be identified, and obtains bill information of the bill image to be identified; wherein, the bill information identification model is obtained by training according to a preset training set, The preset training set includes a plurality of sample bill images of different bill types, and sample bill information of each sample bill image; and returns a first response message for the information identification request, where the first response message includes the The bill information of the bill image to be identified;

The storage device is used for storing bill information of the bill image to be identified.

An embodiment of the present invention further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

memory for storing computer programs;

The processor is configured to implement any one of the above-mentioned steps of the bill information identification method when executing the program stored in the memory.

Embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any one of the above-mentioned steps of the bill information identification method is implemented.

Embodiments of the present invention also provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute any one of the above-mentioned methods for identifying ticket information.

Beneficial effects of the embodiment of the present invention:

In the bill information identification method, device and system provided by the embodiments of the present invention, the OCR device in the bill information identification system, after receiving the information identification request for the bill image to be identified, uses the pre-trained bill information identification model to identify the preprocessing The bill information in the subsequent bill image to be recognized is obtained, thereby obtaining bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

Of course, it is not necessary for any product or method of the present invention to achieve all of the advantages described above at the same time.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other embodiments can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a method for training a ticket information recognition model according to an embodiment of the present invention;

2 is a schematic structural diagram of a preset deep learning network provided by an embodiment of the present invention;

3 is a first schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention;

4 is a schematic flowchart of a method for preprocessing an image of a bill to be recognized according to an embodiment of the present invention;

5 is a schematic diagram of an image white border cropping process provided by an embodiment of the present invention;

6 is a schematic diagram of character recognition provided by an embodiment of the present invention;

FIG. 7 is a second schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention;

8 is a third schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention;

9 is a fourth schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention;

10 is a signaling diagram of a ticket information identification process provided by an embodiment of the present invention;

11 is a schematic structural diagram of a bill identification device provided by an embodiment of the present invention;

12 is a schematic structural diagram of a bill identification system provided by an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In order to solve the problems existing in the existing bill information identification methods, an embodiment of the present invention provides a bill information identification method. The method is applied to the OCR equipment in the ticket information identification system. The OCR device can be any electronic device used for ticket information identification. In the method provided by the embodiment of the present invention, an information identification request for a bill image to be identified is received; based on the information identification request, the bill image to be identified is preprocessed to obtain a preprocessed bill image to be identified; the pre-trained bill information is used The recognition model recognizes the bill information in the preprocessed bill image to be identified, and obtains bill information of the bill image to be identified; wherein, the bill information identification model is trained according to a preset training set, and the preset training set includes a plurality of Sample bill images of different bill types, and sample bill information of each sample bill image; return a first response message to the information identification request, where the first response message includes bill information of the bill image to be identified.

In the embodiment of the present invention, the above-mentioned ticket information identification system may include the above-mentioned OCR device and storage device. Wherein, the number of OCR devices in the ticket information identification system is at least one.

With the method provided by the embodiment of the present invention, after receiving the information identification request for the bill image to be identified, the OCR device in the bill information identification system uses the pre-trained bill information identification model to identify the preprocessed bill image to be identified to obtain the bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

The embodiments of the present invention will be described below through specific embodiments.

As shown in FIG. 1 , FIG. 1 is a schematic flowchart of a method for training a bill information recognition model according to an embodiment of the present invention. The method includes the following steps.

Step S101, obtaining a preset training set.

The above-mentioned preset training set includes a plurality of sample bill images of different bill types, and sample bill information of each sample bill image.

The above-mentioned sample bill image may be an image obtained by photographing the sample bill, or an image obtained by scanning the sample bill, or an image obtained by photographing or scanning the sample bill after preprocessing. The bill types of the sample bills include, but are not limited to, drafts, promissory notes, checks, bills of lading, and certificates of deposit. For the method of preprocessing, reference may be made to the following description, which will not be described in detail here.

According to the different types of bills corresponding to the sample bills in the preset training set, the sample bill information corresponding to each sample bill image is also different. For example, when the above-mentioned sample bill is an invoice, the bill information of the invoice includes, but is not limited to, the invoice code, the invoice number, the billing date, the buyer's name and other information. For another example, when the sample bill is a bank draft, the bill information of the bank draft includes, but is not limited to, the payee, the billing amount, the actual calculated amount, and the billing date.

In the embodiment of the present invention, the number of sample bill images corresponding to each bill type in the preset training set, the bill type of the sample bill, the sample bill image, and the bill information of the sample bill image are not specifically limited.

Step S102 , for each sample bill image, use a preset deep learning network to identify bill information in the sample bill image, and obtain predicted bill information of the sample bill image.

In this step, for each sample bill image, the OCR device can input the sample bill image into a preset deep learning network, so as to use the preset deep learning network to identify the bill information included in the sample bill image, and use the preset deep learning network to identify the bill information included in the sample bill image. The recognized ticket information is output as predicted ticket information for the sample ticket image.

In an optional embodiment, in order to improve the accuracy of the ticket information recognition model obtained by training, the above-mentioned preset deep learning network may be obtained based on multiple deep learning algorithms. That is, the above-mentioned preset deep learning network may be composed of deep learning networks corresponding to multiple deep learning algorithms.

For example, the above-mentioned preset deep learning network may be a deep learning network obtained according to a continuous text region proposal network (Connectionist Text Proposal Network, CTPN) algorithm and a convolutional recurrent neural network (Convolutional Recurrent Neural Network, CRNN) algorithm. That is, the above-mentioned preset deep learning network can be composed of CTPN network and CRNN network. Among them, the CTPN algorithm/network can be used to determine the character region where the characters in the sample bill image are located. The CRNN algorithm/network can be used to identify the characters in the character region, resulting in a collection of strings in the sample ticket image.

For another example, the above-mentioned preset deep learning network may also be a deep learning network obtained according to the Shape Robust Text Detection with Progressive Scale Expansion Network (PSENET) algorithm and the Convolutional Recurrent Neural Network (CRNN) algorithm. . That is, the above-mentioned preset deep learning network can be composed of a PSENET network and a CRNN network. Among them, the PSENET algorithm/network can be used to determine the character area where the characters in the sample bill image are located. The CRNN algorithm/network can be used to identify the characters in the character region, resulting in a collection of strings in the sample ticket image.

For ease of understanding, the above preset deep learning network will be described by taking as an example that the preset deep learning network may be a deep learning network obtained based on the CTPN algorithm and the CRNN algorithm. As shown in FIG. 2 , FIG. 2 is a schematic structural diagram of a preset deep learning network provided by an embodiment of the present invention. The preset deep learning network shown in FIG. 2 may mainly include a convolution layer 201 , a logistic regression layer 202 , a fully connected layer 203 , a convolution layer 204 , a recurrent layer 205 and a transcription layer 206 . Among them, the convolutional layer 201, the logistic regression layer 202 and the fully connected layer 203 are the components of the above-mentioned CTPN network, and the convolutional layer 204, the recurrent layer 205 and the transcription layer 206 are the components of the above-mentioned CRNN network. In addition to this, the above-mentioned preset deep learning network may also include an input layer, an output layer (not shown in FIG. 2 ), etc., which will not be described in detail here.

The above-mentioned convolutional layer 201 can perform feature extraction on the input sample bill image. That is, the feature of all channels at the corresponding position of each pixel point in the sample bill image is extracted, such as the feature of the red green blue (Red Green Blue, RGB) channel.

The above-mentioned logistic regression layer 202 can perform logistic regression processing on the features extracted by the convolution layer 201 , for example, perform a bi-directional long short-term memory network (Bi-directional Long Short-Term Memory, BLSTM) regression processing on the features extracted by the convolution layer 201 .

The fully connected layer 203 performs fully connected processing on the received output of the logistic regression layer 202, and can obtain 2k vertical coordinates, 2k scores, and k side-refinements. Among them, k is the number of anchor points, generally 10 by default. The vertical coordinates contain two parameter values, the vertical offset y and the area height h. Scores contains two scores for the foreground and background of the sample bill image. The side-refinement contains the offset x in the horizontal direction.

The convolution layer 204 performs convolution processing on the output of the fully connected layer 203 to extract feature information in the image to obtain a feature sequence.

The recurrent layer 205 is a BLSTM network, which can further predict the feature sequence extracted by the convolution layer 204 to obtain a preliminary prediction result. Compared with the traditional recurrent neural network (Recurrent Neural Network, RNN), the recurrent layer 205 includes three gate structures of forget gate, input gate, and output gate to process the cell state of the RNN. Among them, the forget gate mainly uses the logistic regression (sigmoid, also known as sigmoid growth curve) function to decide to retain the information of the previous cell state, and the input gate adds new information to the cell state through the sigmoid and hyperbolic tangent (tanh) functions, and Combined with the old cell state to update the current cell state, the input gate determines the output characteristics through the sigmoid function. By forgetting gates, input gates, and output gates, each character can be connected to the previous character.

In the embodiment of the present invention, a two-layer BLSTM network is used to associate each character in the sample bill image with its adjacent characters (that is, the characters before and after the character). After two layers of LSTM, the recurrent layer 205 outputs each feature sequence. The predicted character probability value, that is, the above-mentioned preliminary prediction result.

The transcription layer 206 performs transcription based on the above-mentioned preliminary prediction result to obtain the final prediction result.

In this embodiment of the present invention, the above-mentioned transcription layer 206 may perform transcription in the following two ways:

The first method is based on the transcription in the form of a dictionary. That is, when transcribing, the output result is compared with the preset dictionary, and the predicted value with the maximum probability is determined as the final predicted character string, that is, the above-mentioned final predicted result.

The second method is based on the transcription without dictionary form. That is, there is no character string or preset dictionary for comparison, and when transcribing, the character with the highest probability is directly determined as the final predicted character string, that is, the above-mentioned final predicted result.

In the embodiment of the present invention, the above-mentioned convolutional layer 201 and convolutional layer 204 include a plurality of convolutional units and pooling units. Taking the convolutional layer 201 as an example for illustration, the convolutional layer 201 can perform multiple convolution operations and pooling operations on the input sample bill image, so as to obtain the features of the full channel. Here, the above-mentioned convolution operation and pooling operation process will not be described in detail.

In an optional embodiment, before the convolution layer 204 performs convolution processing on the output of the fully connected layer 203, the height of the sample bill image may be proportionally scaled to a preset height according to the size of the sample bill image. For example, the width of the sample bill image is W_0, the height is H_0, denoted as (W_0, H_0), and the preset height is 32, then the size of the sample bill image after proportional scaling can be expressed as (W, 32)

By adjusting the size of the sample bill image, the receptive field of the feature sequence extracted by the convolution layer 204 can be a rectangular area. Specifically, the kernel size in the modified convolutional layer 204 may be changed from a square size to a rectangular size, so that the convolutional layer 204 obtains a feature sequence corresponding to the rectangular area. This will facilitate later character recognition and improve the accuracy of the recognized characters

According to different deep learning algorithms adopted by the above-mentioned preset deep learning network, the structure of the above-mentioned preset deep learning network is also different. Here, the deep learning algorithm adopted by the preset deep learning network and the structure of the preset deep learning network will not be described in detail.

Step S103: Calculate the loss value of the preset deep learning network based on the sample bill information and the predicted bill information corresponding to each sample bill image.

In this step, the OCR device may use the preset loss function to calculate the loss value of the deep learning network according to the sample bill information and predicted bill information corresponding to each sample bill image in the preset training set.

The above-mentioned preset loss functions include but are not limited to Softmax-cross-entropy loss functions, logarithmic loss functions, and L2 regression loss functions. Among them, Softmax in the Softmax-cross entropy loss function represents the Softmax function, which can also be called a normalized exponential function. The L2 regression loss function can also be called the Mean Square Error (MSE) loss function. Here, the above preset loss function is not specifically limited.

In the embodiment of the present invention, since the above-mentioned preset deep learning network may be a deep learning network formed based on different algorithms, when calculating the loss value of the above-mentioned deep learning network, the OCR device may calculate the corresponding corresponding loss value. Taking the above preset deep learning network based on the deep learning network obtained by the CTPN algorithm and the CRNN algorithm as an example, the OCR device can use the above L2 regression loss function to calculate the loss value of the deep learning network corresponding to the CTPN algorithm, using the above Softmax-cross entropy The loss function calculates the loss value of the CRNN algorithm corresponding to the deep learning network. Here, the calculation process of the above-mentioned loss value will not be described in detail.

Step S104, when the loss value is greater than the preset loss value threshold, adjust the parameters of the preset deep learning network, and return to the above-mentioned step S102.

In this step, the OCR device may compare the loss value of the preset deep learning network with the preset loss value threshold, so as to determine whether the preset deep learning network has converged. When the loss value of the preset deep learning network is greater than the preset loss value threshold, the OCR device may determine that the preset deep learning network has not converged. At this time, the OCR device may adjust the parameters of the preset deep learning network, and return to performing the above step S102. That is, returning to the above step of recognizing the bill information in the sample bill image by using the preset deep learning network for each sample bill image, and obtaining the predicted bill information of the sample bill image.

In the embodiment of the present invention, the above-mentioned adjustment of parameters in the preset deep learning network includes, but is not limited to, adjustment of the offset and weight in the preset deep learning network. The parameter adjustment methods adopted by the OCR equipment include, but are not limited to, the gradient descent method and the reverse adjustment method.

Step S105, when the loss value is not greater than the preset loss value threshold, determine the current preset deep learning network as the trained bill information recognition model.

In this step, when the loss value of the preset deep learning network is not greater than the preset loss value threshold, the OCR device may determine that the preset deep learning network is converged. At this time, the OCR device can determine the current preset deep learning network as the trained bill information recognition model.

Using the method shown in FIG. 1, the bill information recognition model trained by using the above preset training set can be applied to the identification of bills of different bill types, which improves the versatility of the trained bill information identification model, thereby improving bills. Generality of identification methods.

Based on the bill information identification model trained by the method shown in FIG. 1 , an embodiment of the present invention provides a bill information identification method. As shown in FIG. 3 , FIG. 3 is a first schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention. The method is applied to the OCR equipment in the ticket information identification system, and specifically includes the following steps.

Step S301, receiving an information identification request for the image of the bill to be identified.

The to-be-recognized bill image of the to-be-identified bill may be included in the above information identification request. Regarding the bill image to be identified, reference may be made to the above description of the sample bill image, which will not be described in detail here.

In an optional embodiment, the user may send an information identification request for the image of the bill to be identified to the OCR device through the client. The OCR device receives the information identification request sent by the user.

In another optional embodiment, the above-mentioned OCR device may include multiple OCR computing instance units (ie, service instances, Service Instances). A request proxy middleware can be set, such as a reverse proxy (Reverse Proxy). When a user sends an information identification request for a ticket image to be identified to the ticket information identification system, the request proxy middleware can receive the information identification request sent by the user, so that according to the computing resources of each OCR calculation instance unit in the OCR device, the The received information identification request is forwarded to the OCR computing instance unit whose computing resources are not occupied. The OCR calculation instance unit in the OCR device receives the information identification request forwarded by the request proxy middleware.

The above-mentioned request proxy middleware may be integrated in the above-mentioned OCR device, and may also be integrated in other electronic devices. For the convenience of description, in the embodiment of the present invention, the above-mentioned request proxy middleware is integrated in the OCR device as an example for description, which does not play any limiting role.

Step S302 , based on the information identification request, preprocess the image of the bill to be identified to obtain the preprocessed bill image to be identified.

In this step, the above-mentioned preprocessing includes, but is not limited to, image contrast enhancement processing, image grayscale processing, image binarization processing, image white border cropping processing, and skew correction processing. That is, when the OCR device receives the above information identification request, it can preprocess the bill image to be identified carried in the information identification request according to the preset preprocessing sequence, and obtain the preprocessed bill image to be identified. For example, the OCR device can sequentially perform image contrast enhancement processing, image grayscale processing, image binarization processing, image white edge cropping processing, and tilt correction processing on the image of the bill to be recognized, to obtain a preprocessed bill image to be identified. The specific process of preprocessing can be found in the following description, which will not be described in detail here.

In the embodiment of the present invention, according to the different types of bills of the bills to be identified and the different bill images to be identified, the preprocessing process of the bill images to be identified is also different.

For example, in a common invoice, the upper left area is the area where the QR code is located. When the above-mentioned bill to be identified is the invoice, the above-mentioned preprocessing process may further include a text direction correction process. That is, the OCR device can use a preset filter, such as a Gaussian filter, to perform convolution processing on the image of the bill to be identified corresponding to the invoice, determine the position of the maximum value, and determine the position of the maximum value as the upper left corner of the invoice. , so as to determine the direction of the image text of the bill to be recognized.

For another example, when the above-mentioned image of the bill to be identified is an image obtained by photographing the bill to be identified, that is, when the image of the bill to be identified is a color image, the above-mentioned preprocessing process may include an image grayscale processing process. When the above-mentioned image of the bill to be identified is an image of the bill to be identified, that is, the image is a grayscale image, the above-mentioned preprocessing process may not include the above-mentioned image grayscale processing process.

In the embodiment of the present invention, the preprocessing process of the above-mentioned bill image to be recognized is not specifically limited.

In an optional embodiment, when the above-mentioned two-dimensional code is included in the above-mentioned bill image to be recognized, the OCR device can directly identify the two-dimensional code in the to-be-recognized bill image, obtain the bill information of the bill image to be identified, and store the bill. information. When the above two-dimensional code is not included in the to-be-recognized bill image, the OCR device executes the above step S302, that is, the step of preprocessing the to-be-recognized bill image based on the information identification request to obtain the pre-processed to-be-recognized bill image.

Step S303, using the pre-trained bill information recognition model to identify the bill information in the pre-processed bill image to be identified, to obtain bill information of the bill image to be identified; wherein, the bill information identification model is obtained by training according to a preset training set Yes, the preset training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image.

For the identification process of the bill information of the bill image to be identified, reference may be made to the following description, which will not be described in detail here.

Step S304, returning a first response message to the information identification request, where the first response message includes the bill information of the bill image to be identified.

In this step, the OCR device may return a first response message for the information identification request according to the identification result output by the above-mentioned bill identification model, that is, the bill information of the above-mentioned bill image to be identified.

In an optional embodiment, if the above-mentioned information identification request is directly sent by the user to the OCR device, the OCR device may send a first response message including the ticket information of the ticket image to be identified to the client used by the user.

In another optional embodiment, if the above-mentioned information identification request is forwarded by the above-mentioned request proxy middleware, the OCR device may send a first response message including bill information of the bill image to be identified to the request proxy middleware. The request proxy middleware sends the first message to the client corresponding to the above-mentioned information identification request, that is, the client that sends the above-mentioned information identification request.

Using the method shown in Fig. 3, after receiving the information identification request for the bill image to be identified, the OCR device in the bill information identification system uses the pre-trained bill information identification model to identify the preprocessed bill image to be identified. to obtain the bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

In an optional embodiment, due to the influence of factors such as the central processing unit (Central Processing Unit, CPU), memory, and hardware characteristics of the graphics processing unit (Graphics Processing Unit, GPU) in the above-mentioned OCR device, it is impossible to use the OCR device on the OCR device. Start multiple processes at the same time for service. Therefore, in the embodiment of the present invention, the above-mentioned OCR device may process the above-mentioned information identification request in a single-process multi-thread mode, which enables the OCR device to process multiple information identification requests synchronously.

In an optional embodiment, for the above step S302, based on the information identification request, the bill image to be identified is preprocessed to obtain a preprocessed bill image to be identified. The embodiment of the present invention provides a preprocessing of the bill image to be identified. Approach. As shown in FIG. 4 , FIG. 4 is a schematic flowchart of a method for preprocessing an image of a bill to be recognized according to an embodiment of the present invention. Specifically, the above step S302 can be refined into the following steps.

Step S3021, using a preset gamma coefficient to perform gamma transformation on the image of the bill to be identified in the information identification request to obtain a first image.

In this step, when the to-be-recognized bill image in the above information identification request is a color image, the to-be-identified bill image includes not only bill information, but also some other information such as border information. For example, the borders of common bills are generally brown, and the text is generally blue or red. In order to make the difference between the bill information in the bill image to be recognized and other information more obvious, the OCR device can perform image contrast enhancement processing on the bill image to be recognized, that is, the OCR device can use the preset gamma coefficient to identify the information in the request. The image of the bill to be recognized is subjected to gamma transformation to obtain the first image, that is, the image after the image contrast is enhanced. Among them, gamma transform can also be called Gamma transform.

In an optional embodiment, the above-mentioned preset gamma coefficient may be 2.5. In addition to this, the above-mentioned preset gamma coefficient may also be other values. For example, the value range of the above-mentioned preset gamma coefficient may be between 1 and 3, wherein the preset gamma coefficient of 1 indicates that no image contrast is performed. Enhancement processing, that is, the preset gamma coefficient is 1 is the above-mentioned bill image to be recognized.

In an optional embodiment, when the to-be-recognized bill image in the above information identification request is a grayscale image, the OCR device may use other methods to perform image comparison and enhancement processing on the to-be-identified bill image to obtain the first image. For example, the OCR device may use a grayscale transformation method to perform image contrast and enhancement processing on the image of the bill to be recognized to obtain the first image.

In the embodiment of the present invention, the method used for the above-mentioned image contrast enhancement processing is not specifically limited.

Step S3022, converting the first image into a grayscale image to obtain a second image.

In this step, the OCR device may perform image grayscale processing on the first image, that is, convert the first image into a grayscale image to obtain a second image. By converting the RGB value of each pixel in the first image into a grayscale value, the color information in the first image is removed, and the text information is enhanced, that is, the bill information is enhanced. The process of image grayscale processing will not be described in detail here.

Step S3023: Perform image binarization processing on the second image based on the average value of pixel values corresponding to each pixel in the second image to obtain a third image.

In this step, in order to improve the accuracy of the preprocessed bill image to be recognized, the OCR device may perform image binarization processing on the second image, thereby distinguishing the foreground and background of the bill image to be identified. That is, based on the average value of pixel values corresponding to each pixel in the second image, image binarization processing is performed on the second image to obtain a third image, that is, an image obtained by performing image binarization processing on the second image.

Specifically, the OCR device may calculate the average value of the pixel values in the second image according to the pixel value of each pixel in the second image, that is, the gray value. Taking the 5 pixel points included in the second image, that is, the pixel points 1-5 as an example, it is assumed that the pixel values corresponding to the pixel points 1-5 are grayscales 1-5. The average Z of pixel values in the second image can be expressed as:

The OCR device may compare the pixel value of each pixel in the second image with the average value. For each pixel, when the pixel value of the pixel is less than the above average value, the OCR device can determine that the pixel value of the pixel is 0, that is, the pixel is represented as black, that is, the foreground of the above-mentioned bill image to be recognized; When the pixel value of the pixel point is not less than the above average value, the OCR device can determine that the pixel value of the pixel point is 255, that is, the pixel point is represented as white, which is the background of the above-mentioned bill image to be recognized, thereby obtaining the third image.

Step S3024, according to the pixel value of each pixel included in each row in the horizontal direction in the third image, the pixel value of each pixel included in each column in the vertical direction, and the pixel number threshold of the preset pixel value, determine the first pixel value. The position of the outer border of the three images, and the image within the position of the outer border is cropped to obtain a fourth image.

For ease of understanding, a pixel point in any horizontal direction in the above third image is taken as an example for description with reference to FIG. 5 . FIG. 5 is a schematic diagram of an image white border cropping process provided by an embodiment of the present invention.

In Fig. 5, 10 pixel points are included, namely pixel points a-j. These 10 pixel points are all the pixel points included in a certain horizontal direction in the above-mentioned third image. Among them, the pixel value of pixel point a, pixel point c, pixel point d, pixel point h, pixel point i and pixel point j is 0, and the pixel value of pixel point b, pixel point e, pixel point f and pixel point g is 255.

For each row in the horizontal direction in the above-mentioned third image, the OCR device may traverse each pixel in the order from left to right, and count the number of pixels whose pixel value is a preset pixel value. When the number of pixels whose pixel value is the preset pixel value is equal to the above-mentioned threshold of the number of pixels, the above-mentioned threshold of the number of pixels is traversed in reverse, and the position of the last traversed pixel is determined as the row of pixels. the left border position of . By analogy, the OCR device can determine the right border position of each row of pixels, the upper border position of each column of pixels, and the lower border position of each column of pixels according to the method for determining the left outer border position.

For ease of understanding, the above-mentioned predetermined pixel value is 0, and the pixel number threshold is 4, and the determination of the above-mentioned left border position is illustrated with reference to the above-mentioned FIG. 5 . For each line in the horizontal direction in the above third image, the OCR device may traverse each pixel in the order from left to right, and when the traversal reaches the pixel h, the number of pixels with a pixel value of 0 is 4. At this time, the OCR device will traverse 4 pixel points in reverse, that is, traverse from pixel point h to pixel point d. The OCR device may determine the position of the left boundary of the pixel point of the row by the left boundary of the pixel point d.

The OCR device may determine the position of the outer border of the third image according to the determined left and right border positions of each row in the horizontal direction and the upper and lower border positions of each column in the vertical direction. For example, the OCR device may determine each left border position and right border position in the horizontal direction and the smallest rectangular area or the smallest polygon area corresponding to each upper border position and bottom border position in the vertical direction as the third image the position of the outer border.

The OCR device may crop the image within the position of the outer frame determined in the third image to obtain the fourth image, that is, to obtain the image after the white border cropping process of the image.

In the embodiment of the present invention, the above-mentioned step S3024 is an image white border cropping processing process for the third image. While discarding irrelevant white border information in the third image, it is possible to reduce the image white border cropping process in the fourth image. Noise information, such as the influence of the pixel point a and the pixel point c in the above-mentioned FIG. 5 on the identification of the bill information in the later stage, improves the accuracy of the bill information identification.

Step S3025, performing tilt correction on the fourth image to obtain a preprocessed image of the bill to be recognized.

In this embodiment of the present invention, during the process of photographing or scanning the bill to be recognized, it may be used for the shooting angle or the position when scanning, which may cause the bill to be recognized in the image of the bill to be recognized to be inclined, such as the direction of the text. Tilt occurs. In order to reduce the influence of image tilt on the recognition and recognition of bill information in the later stage, the OCR device can perform tilt correction on the above fourth image, so as to obtain the image after tilt correction processing, that is, the pre-processed image of the bill to be recognized. Accuracy of information identification.

In the embodiment shown in FIG. 4 , the above-mentioned preprocessing process of the bill image to be recognized includes, but is not limited to, the above-mentioned image contrast enhancement processing, image grayscale processing, image binarization processing, image white border cropping processing, and skew correction processing. . The pre-processed image of the bill to be recognized is the image obtained after the last step of the pre-processing process, and is not limited to the image after the tilt correction process.

Using the method shown in FIG. 4 , the OCR device improves the recognition accuracy of the preprocessed bill image to be recognized in the later stage through the preprocessing process of the bill image to be recognized, thereby improving the accuracy of the bill information identification method.

In an optional embodiment, in the above step S303, the pre-trained bill information recognition model is used to identify bill information in the pre-processed bill image to be identified, and obtain bill information of the bill image to be identified, which may specifically include the following steps: .

Step 1: Based on the pre-trained bill information recognition model, determine the region where each character in the preprocessed bill image to be recognized is located as the character region, and identify the characters in the character region to obtain a string set.

For ease of understanding, the bill information recognition model is a deep neural network formed by the CTPN algorithm and the CRNN algorithm as an example for description in conjunction with FIG. 6 . FIG. 6 is a schematic diagram of character recognition provided by an embodiment of the present invention.

The deep network corresponding to the above CTPN algorithm is obtained by combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM). According to the characteristics of the characters in the preprocessed bill image to be recognized, each character can be regarded as a character, a part of a character or a sequence composed of multiple characters. As shown in FIG. 6 , the character S is a character in the preprocessed bill image to be recognized, and the CTPN algorithm can be used to determine that the character S may be composed of some characters included in the area 601-605. Therefore, the area formed by the area 601 - the area 605 is the character area where the character S is located in the preprocessed bill image to be recognized. By analogy, the OCR device can identify the region where each character is located in the preprocessed bill image to be recognized, so as to obtain the character region in the preprocessed bill image to be recognized.

In the embodiment of the present invention, because the above-mentioned LSTM has the context inference ability, it can extract the features between characters in the preprocessed bill image to be recognized. Therefore, when the OCR device determines the character area, it can divide the characters with contextual relationship. In the same character area, for example, there is a contextual relationship between each character in the buyer's name in the above invoice, and the buyer's name can be divided into the same character area.

In the embodiment of the present invention, the character area in the pre-processed image of the bill to be recognized may include multiple character areas. Each character area includes at least one character.

In an optional embodiment, the above-mentioned character area may be recorded as the coordinate position where the upper left corner vertex is located and the width and height values of the character area. Still take FIG. 6 as an example for description. It is assumed that the region where the character S shown in FIG. 6 is located in the preprocessed bill image to be recognized is a character region. The character area can be recorded as the coordinates of the upper left corner vertex where the area 601 is located, the width and value corresponding to the area 601 to the area 605, and the height value corresponding to any one of the area 601 and the area 605.

In an optional embodiment, since the preprocessed bill image to be recognized may contain more text information, the OCR device can screen the determined multiple character areas according to the aspect ratio corresponding to each character area. . Taking the area where each character is located as a square area (that is, the aspect ratio is 1:1) as an example, the OCR device can filter out the character area with an aspect ratio greater than 2, that is, discarding the area with an aspect ratio not greater than 2 character area. That is, the character area that contains at least two characters is filtered out.

In the embodiment of the present invention, the above-mentioned CTPN algorithm can greatly reduce the constraint on the above-mentioned bill image to be recognized, thereby improving bill information Generality of identification methods.

In an optional embodiment, after determining the character area in the preprocessed bill image to be recognized, the OCR device can use the CRNN algorithm to identify the characters in the character area, and obtain a character string corresponding to each character area, and then A string set corresponding to the preprocessed bill image to be recognized is obtained.

In the embodiment of the present invention, the deep learning network corresponding to the above CRNN algorithm is end-to-end training during model training, which can effectively reduce the complexity of model training and save labor costs. In addition, the CRNN algorithm can recognize character sequences of any length, and without involving character segmentation or horizontal scale normalization, it retains the contextual features between characters in the bill to be recognized, and improves the bill recognition model for bill information recognition. accuracy. In addition, the CRNN algorithm is suitable for character recognition tasks without dictionary scenarios, as well as text recognition tasks based on dictionary scenarios, which improves the versatility of character recognition and thus the universality of bill information recognition methods.

In step 2, regularized matching is performed on the string set to obtain the bill information of the bill image to be recognized.

In this step, the OCR device may perform regularized matching on the character string set according to the preset matching rules to obtain the bill information of the bill image to be recognized. For example, a known rule for invoice codes in VAT invoices is typically 10 or 12 digits. The OCR device may determine the rule as one of the preset matching rules, so as to identify the invoice code in the VAT invoice according to the rule.

In the implementation of the present invention, the above-mentioned preset matching rules may be set according to the characteristics of bill information included in bills of different bill types. Here, the above preset matching rules are not specifically limited.

Through the above steps 1 and 2, the OCR device can accurately identify the bill information in the preprocessed bills to be identified, which improves the accuracy of bill information identification and the generality of the bill information identification method.

In an optional embodiment, according to the method shown in FIG. 3 , an embodiment of the present invention further provides a method for identifying bill information. As shown in FIG. 7 , FIG. 7 is a schematic flowchart of a second type of bill information identification method provided by an embodiment of the present invention. The method includes the following steps.

Step S701, receiving an information identification request for the image of the bill to be identified.

The above-mentioned step S701 is the same as the above-mentioned step S301.

In step S702, the information identification request is authenticated, and an authentication result is obtained.

In this step, after receiving the above information identification request, the OCR device may authenticate the information identification request according to other information carried in the information identification request except the above-mentioned bill image to be identified.

In this embodiment of the present invention, the above-mentioned information identification request is transmitted in the form of a message in the network. Therefore, the above information identification request may also include other information. Taking the information identification request as a HyperText Transfer Protocol (HTTP) message as an example, the information identification request may also include the address information of the sender, the header field of the message, and the content length (Content-Length) of the message. ) and other information.

For ease of understanding, the authentication process of the information identification request is described with the above content length. When the content length of the information identification request is greater than the preset content length threshold, the OCR device may determine that the information identification request is an illegal request. When the content length of the information identification request is not greater than the preset content length threshold, the OCR device may determine that the information identification request is a legitimate request.

In the embodiment of the present invention, the legal authentication process of the above-mentioned information identification request is not specifically limited.

Step S703, when the authentication result indicates that the information identification request is a legitimate request, based on the information identification request, preprocess the to-be-recognized bill image to obtain a pre-processed to-be-identified bill image.

In the embodiment of the present invention, through the legal authentication process of the above-mentioned information identification request, the OCR device can be prevented from identifying the bill information to be identified in the illegal information identification request in time, and the computing resources of the OCR device are effectively saved, thereby reducing the OCR device. The number of bill images that the device needs to identify improves the identification efficiency of the bill information by the OCR device.

Step S704, using the pre-trained bill information recognition model to identify the bill information in the preprocessed bill image to be identified, and obtain bill information of the bill image to be identified; wherein, the bill information identification model is obtained by training according to a preset training set Yes, the preset training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image.

Step S705, returning a first response message to the information identification request, where the first response message includes the bill information of the bill image to be identified.

The above steps S704 to S705 are the same as the above steps S303 to S304.

In an optional embodiment, according to the method shown in FIG. 3 , an embodiment of the present invention further provides a method for identifying bill information. As shown in FIG. 8 , FIG. 8 is a third schematic flowchart of a method for identifying bill information provided by an embodiment of the present invention. The method includes the following steps.

Step S801, receiving an information identification request for the image of the bill to be identified.

The above-mentioned step S801 is the same as the above-mentioned step S301.

Step S802, in the case that the current computing resources are occupied, return a second response message for the information identification request; the second response message includes an identification code indicating that the computing resources are occupied.

In this step, when the OCR device receives the above information identification request, it can determine whether the current computing resources are occupied, that is, determine whether the OCR device is currently processing other received identification requests. When it is determined that the current computing resources are occupied, the OCR device may return a second response message for the above-mentioned information identification request. For the method for sending the second response message, reference may be made to the method for sending the first response message, which will not be described in detail here.

In traditional OCR technology, request messages are generally processed in the form of message queues. However, due to the OCR service characteristics, long delays will lead to long message queues and unpredictable errors, such as request message overflow. and so on. In the embodiment of the present invention, the OCR device can ensure that each OCR computing instance unit in the OCR device rejects other information identification requests in time according to the current occupation of computing resources, and ensures the orderliness of responses to the information identification requests.

Step S803 , based on the information identification request, preprocess the image of the bill to be identified to obtain a preprocessed image of the bill to be identified.

In this step, when it is determined that the current computing resources are not occupied, the OCR device may preprocess the image of the bill to be identified based on the information identification request to obtain the preprocessed bill image to be identified.

Step S804, using the pre-trained bill information identification model to identify bill information in the pre-processed bill image to be identified, and obtain bill information of the bill image to be identified; wherein, the bill information identification model is obtained by training according to a preset training set Yes, the preset training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image.

Step S805, returning a first response message to the information identification request, where the first response message includes the bill information of the bill image to be identified.

The above steps S804 to S805 are the same as the above steps S303 to S304.

In an optional embodiment, according to the method shown in FIG. 3 , an embodiment of the present invention further provides a method for identifying bill information. As shown in FIG. 9 , FIG. 9 is a fourth schematic flowchart of a method for identifying bill information according to an embodiment of the present invention. The method includes the following steps.

Step S901, receiving an information identification request for the image of the bill to be identified.

Step S902 , based on the information identification request, preprocess the image of the bill to be identified to obtain a preprocessed image of the bill to be identified.

Step S903, using the pre-trained bill information recognition model to identify bill information in the pre-processed bill image to be identified, and obtain bill information of the bill image to be identified; wherein, the bill information identification model is obtained by training according to a preset training set Yes, the preset training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image.

The above steps S901 to S903 are the same as the above steps S301 to S303.

Step S904, the bill image to be recognized and the bill information of the bill image to be identified are stored in the storage device in the bill information identification system.

In this step, the OCR device may send the above-mentioned bill image to be identified and bill information of the bill image to be identified to the storage device in the bill information identification system. After receiving the bill image to be identified and the bill information of the bill image to be identified sent by the OCR device, the storage device can calculate the bill image to be identified and perform Hash calculation, obtain the Hash value of the bill image to be identified, and store the bill image to be identified. The image of the ticket to be recognized. Further, the storage device can store the Hash value, the absolute address and the bill information corresponding to the bill image to be identified.

In an optional embodiment, the above-mentioned storage device may use a preset Hash algorithm to perform Hash calculation on the above-mentioned bill image to be recognized, to obtain a Hash value of the bill image to be recognized. The preset Hash algorithm includes but is not limited to a message digest algorithm (Message-Digest Algorithm, MD5) and a secure hash (Secure Hash Algorithm, SHA) algorithm.

In an optional embodiment, when the storage device stores the Hash value, the absolute address and the ticket information corresponding to the above-mentioned ticket image to be recognized, the corresponding relationship between the Hash value, the absolute address and the ticket information can be established, so as to store the corresponding relationship between the Hash value, the absolute address and the ticket information. The corresponding relationship is stored in the preset corresponding relationship list in the above-mentioned storage device. As shown in Table 1, Table 1 is a manner for providing correspondence storage in an embodiment of the present invention.

Table 1

Hash value absolute address Ticket information Hash value 1 Address 1 Ticket Information 1 Hash value 2 address 2 Ticket Information 3

The storage device may store the Hash value, absolute address and bill information corresponding to the bill image to be recognized in the above-mentioned Table 1 according to the manner shown in Table 1. In the embodiment of the present invention, there is no specific limitation on the storage manner of the above-mentioned hash value, absolute address, and bill information corresponding to the bill image to be identified.

In the embodiment of the present invention, after the bill information of the bill image to be identified is determined, the bill information is stored persistently, which can facilitate later operations such as bill information query and statistics.

In an optional embodiment, after receiving the bill image to be identified and the bill information of the bill image to be identified sent by the OCR device, the storage device can also directly perform the processing on the bill image to be identified and the bill information of the bill image to be identified. storage. Here, the storage of the bill information of the bill image to be recognized is not specifically limited.

Step S905, returning a first response message to the information identification request, where the first response message includes the bill information of the bill image to be identified.

The above-mentioned step S905 is the same as the above-mentioned step S304.

In this embodiment of the present invention, the execution order of the foregoing steps S904 and S905 is not specifically limited.

For ease of understanding, the method for identifying bill information provided by the above embodiments of the present invention will be described with reference to FIG. 10 . FIG. 10 is a signaling diagram of a ticket information identification process provided by an embodiment of the present invention. Specifically, the following steps may be included.

Step S1001, the client sends an information identification request including an image of the bill to be identified to the OCR device.

Step S1002, the OCR device receives an information identification request.

Step S1003, the OCR device authenticates the information identification request to obtain an authentication result.

Step S1004, when the authentication result indicates that the information identifies the request as a legitimate request, the OCR device determines whether the current computing resources are occupied. If yes, step S1005 is executed. If not, step S1006 is executed.

When the above authentication result indicates that the information identification request is an illegal request, the OCR device may not respond to the information identification request, such as discarding the information identification request.

Step S1005, the OCR device returns a second response message for the information identification request.

Wherein, the above-mentioned second response message includes an identification code indicating that the computing resource is occupied.

In this embodiment of the present invention, the foregoing steps S1002 to S1005 may be performed by the request proxy middleware in the foregoing OCR device.

Step S1006, the OCR device preprocesses the bill image to be identified based on the information identification request to obtain a preprocessed bill image to be identified.

Step S1007, the OCR device uses the pre-trained bill information identification model to identify bill information in the preprocessed bill image to be identified, and obtain bill information of the bill image to be identified.

Step S1008, the OCR device sends the bill image to be recognized and the bill information of the bill image to be identified to the storage device.

Step S1009, the storage device correspondingly stores the received bill image to be identified and bill information of the bill image to be identified.

Step S1010, the OCR device returns a first response message for the information identification request to the client.

Wherein, the above-mentioned first response message includes bill information of the bill image to be identified.

In this embodiment of the present invention, the execution order of the foregoing steps S1008 and S1010 is not specifically limited.

Based on the same inventive concept, and according to the method for identifying bill information provided by the above embodiments of the present invention, the embodiment of the present invention further provides a device for identifying bill information. As shown in FIG. 11 , FIG. 11 is a schematic structural diagram of a bill identification device provided by an embodiment of the present invention. The device is applied to the OCR equipment in the information identification system, and specifically includes the following modules.

A receiving module 1101, configured to receive an information identification request for an image of a bill to be identified;

The preprocessing module 1102 is configured to preprocess the bill image to be identified based on the information identification request to obtain the preprocessed bill image to be identified;

The first identification module 1103 is used to identify the bill information in the preprocessed bill image to be identified by using the pre-trained bill information identification model, and obtain bill information of the bill image to be identified; Assuming that the training set is obtained by training, the preset training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image;

The first returning module 1104 is configured to return a first response message to the information identification request, where the first response message includes the ticket information of the ticket image to be identified.

Optionally, the above-mentioned first recognition module 1103 can be specifically used to determine the region where each character in the preprocessed bill image to be recognized is located as the character region based on the pre-trained bill information recognition model, and identify the characters in the character region. character to obtain a string set; regularized matching is performed on the string set to obtain the bill information of the bill image to be recognized.

Optionally, the above-mentioned preprocessing module 1102 can be specifically configured to use a preset gamma coefficient to perform gamma transformation on the image of the bill to be identified in the information identification request to obtain the first image;

Convert the first image to a grayscale image to obtain a second image;

Based on the average value of the corresponding pixel values of each pixel in the second image, image binarization processing is performed on the second image to obtain a third image;

According to the pixel value of each pixel included in each row in the horizontal direction in the third image, the pixel value of each pixel included in each column in the vertical direction, and the pixel number threshold of the preset pixel value, determine the pixel value of the third image the position of the outer border, and crop the image within the position of the outer border to obtain a fourth image;

Tilt correction is performed on the fourth image to obtain a preprocessed image of the bill to be recognized.

Optionally, the above-mentioned ticket information equipment device may further include:

The authentication module is used to authenticate the information identification request and obtain the authentication result;

The above-mentioned preprocessing module 1102 can be specifically configured to preprocess the image of the bill to be identified based on the information identification request when the authentication result of the authentication module indicates that the information identification request is a legitimate request, and obtain a preprocessed bill image to be identified.

Optionally, the above-mentioned ticket information equipment device may further include:

The second return module is configured to return a second response message for the information identification request if the current computing resources are occupied after receiving the information identification request for the bill image to be identified; the second response message includes an identifier indicating that the computing resources are occupied code.

Optionally, the above-mentioned ticket information equipment device may further include:

The sending module is configured to store the to-be-recognized ticket image and the ticket information of the to-be-recognized ticket image to the storage device in the ticket information identification system before returning the first response message to the information identification request.

Optionally, the above-mentioned ticket information equipment device may further include:

The acquisition module is used to acquire the preset training set;

The second identification module is configured to, for each sample bill image, use a preset deep learning network to identify bill information in the sample bill image, and obtain the predicted bill information of the sample bill image;

a calculation module, configured to calculate the loss value of the preset deep learning network based on the sample bill information and the predicted bill information corresponding to each sample bill image;

The adjustment module is used to adjust the parameters of the preset deep learning network when the loss value is greater than the preset loss value threshold, and return to execute the preset deep learning network for each sample bill image to identify the bill information in the sample bill image. , the steps of obtaining the predicted bill information of the sample bill image;

The determining module is configured to determine the current preset deep learning network as the trained bill information recognition model when the loss value is not greater than the preset loss value threshold.

With the device provided by the embodiment of the present invention, the OCR device in the bill information identification system, after receiving the information identification request for the bill image to be identified, uses the pre-trained bill information identification model to identify the preprocessed bill image to be identified to obtain the bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

Based on the same inventive concept, and according to the bill information identification method provided by the above embodiments of the present invention, the embodiments of the present invention further provide a bill information identification system. As shown in FIG. 12 , FIG. 12 is a schematic structural diagram of a bill identification system provided by an embodiment of the present invention. The OCR device 1201 and the storage device 1202 included in the ticket information identification system;

The above-mentioned OCR device 1201 is used to receive an information identification request for a bill image to be identified; based on the information identification request, preprocess the bill image to be identified to obtain a preprocessed bill image to be identified; using the pre-trained bill information identification model, Recognize the bill information in the preprocessed bill image to be identified, and obtain bill information of the bill image to be identified; wherein, the bill information recognition model is trained according to a preset training set, and the preset training set includes a plurality of different bill types Sample bill images, and sample bill information for each sample bill image; return a first response message to the information identification request, where the first response message includes the bill information of the bill image to be identified;

The above-mentioned storage device 1202 is used for storing bill information of the bill image to be recognized.

With the system provided by the embodiment of the present invention, after receiving the information identification request for the bill image to be identified, the OCR device in the bill information identification system uses the pre-trained bill information identification model to identify the preprocessed bill image to be identified to obtain the bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

Based on the same inventive concept, according to the bill information identification method provided by the above embodiments of the present invention, the embodiments of the present invention further provide an electronic device, which is the above OCR device, as shown in FIG. 13 , including a processor 1301, A communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304;

The memory 1303 is used to store computer programs;

When the processor 1301 is used to execute the program stored in the memory 1303, the following steps are implemented:

Receive an information identification request for the image of the ticket to be identified;

Based on the information identification request, preprocess the bill image to be identified to obtain the preprocessed bill image to be identified;

The pre-trained bill information recognition model is used to identify bill information in the preprocessed bill image to be identified, and the bill information of the bill image to be identified is obtained; wherein, the bill information identification model is trained according to a preset training set, and the pre- Suppose the training set includes multiple sample bill images of different bill types, and sample bill information of each sample bill image;

A first response message to the information identification request is returned, where the first response message includes ticket information of the ticket image to be identified.

With the electronic device provided by the embodiment of the present invention, the OCR device in the bill information identification system, after receiving the information identification request for the bill image to be identified, uses the pre-trained bill information identification model to identify the preprocessed bill to be identified The bill information in the image, so as to obtain the bill information of the bill image to be processed. Compared with the prior art, the use of OCR equipment to identify bill information overcomes the dependence of manual identification on the number of human resources, and since the bill information identification model is trained based on sample bill images of different bill types, the bill information identification model is The bill information of bill images of different bill types can be directly identified, which effectively reduces the dependence on preset candidate templates and character feature information, overcomes the limitations of character recognition in traditional OCR technology, and effectively solves the problem of existing bill information. The problems existing in the identification method are reduced, thereby reducing the dependence on the number of human resources and improving the generality of the bill identification method.

The communication bus mentioned in the above electronic device may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the above electronic device and other devices.

The memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

The above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Based on the same inventive concept, according to the bill information identification method provided by the above embodiments of the present invention, the embodiments of the present invention also provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program When executed by the processor, the steps of any one of the above-mentioned methods for recognizing bill information are realized.

Based on the same inventive concept, according to the bill information identification method provided by the above embodiments of the present invention, the embodiments of the present invention also provide a computer program product including instructions, which, when run on a computer, enables the computer to execute the above embodiments Any ticket information identification method.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), among others.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Each embodiment in this specification is described in a related manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for embodiments such as apparatuses, systems, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the description is relatively simple, and for related details, please refer to the partial descriptions of the method embodiments. That's it.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (10)

1. A bill information recognition method is applied to an Optical Character Recognition (OCR) device in a bill information recognition system, and the method comprises the following steps:

receiving an information identification request of a bill image to be identified;

preprocessing the bill image to be recognized based on the information recognition request to obtain a preprocessed bill image to be recognized;

identifying bill information in the preprocessed bill image to be identified by utilizing a pre-trained bill information identification model to obtain the bill information of the bill image to be identified; the bill information recognition model is obtained by training according to a preset training set, wherein the preset training set comprises a plurality of sample bill images of different bill types and sample bill information of each sample bill image;

and returning a first response message aiming at the information identification request, wherein the first response message comprises the bill information of the bill image to be identified.

2. The method according to claim 1, wherein the step of identifying the bill information in the preprocessed bill image to be identified by using the pre-trained bill information identification model to obtain the bill information of the bill image to be identified comprises:

determining the area where each character is located in the preprocessed bill image to be recognized as a character area based on a pre-trained bill information recognition model, and recognizing the characters in the character area to obtain a character string set;

and performing regularization matching on the character string set to obtain bill information of the bill image to be identified.

3. The method according to claim 1, wherein the step of preprocessing the ticket image to be recognized based on the information recognition request to obtain a preprocessed ticket image to be recognized comprises:

performing gamma transformation on the bill image to be identified in the information identification request by using a preset gamma coefficient to obtain a first image;

converting the first image into a gray image to obtain a second image;

performing image binarization processing on the second image based on the average value of the pixel values corresponding to all the pixel points in the second image to obtain a third image;

determining the outer frame position of the third image according to the pixel values of all pixel points included in each row in the horizontal direction of the third image, the pixel values of all pixel points included in each column in the vertical direction and the pixel number threshold of a preset pixel value, and cutting the image in the outer frame position to obtain a fourth image;

and performing inclination correction on the fourth image to obtain a preprocessed bill image to be recognized.

4. The method according to claim 1, before preprocessing the to-be-recognized bill image based on the information recognition request to obtain a preprocessed to-be-recognized bill image, further comprising:

authenticating the information identification request to obtain an authentication result;

the step of preprocessing the bill image to be recognized based on the information recognition request to obtain a preprocessed bill image to be recognized comprises the following steps:

and when the authentication result indicates that the information identification request is a legal request, preprocessing the to-be-identified bill image based on the information identification request to obtain a preprocessed to-be-identified bill image.

5. The method according to any one of claims 1 to 4, further comprising, after receiving the information identification request for the ticket image to be identified:

returning a second response message to the information identification request if the current computing resource is occupied; the second response message includes an identification code indicating that the computing resource is occupied.

6. The method of claim 1, further comprising, prior to returning the first response message to the information identification request:

and storing the bill image to be identified and the bill information of the bill image to be identified to a storage device in the bill information identification system.

7. The method of claim 1, wherein the bill information recognition model is trained by the following steps:

acquiring the preset training set;

for each sample bill image, utilizing a preset deep learning network to identify bill information in the sample bill image to obtain predicted bill information of the sample bill image;

calculating a loss value of the preset deep learning network based on sample bill information and predicted bill information corresponding to each sample bill image;

when the loss value is larger than a preset loss value threshold value, adjusting parameters of the preset deep learning network, and returning to execute the step of identifying bill information in the sample bill image by using the preset deep learning network aiming at each sample bill image to obtain predicted bill information of the sample bill image;

and when the loss value is not greater than the preset loss value threshold value, determining the current preset deep learning network as a trained bill information recognition model.

8. A bill information recognition apparatus, applied to an Optical Character Recognition (OCR) device in a bill information recognition system, comprising:

the receiving module is used for receiving an information identification request of a bill image to be identified;

the preprocessing module is used for preprocessing the bill image to be recognized based on the information recognition request to obtain a preprocessed bill image to be recognized;

the first identification module is used for identifying the bill information in the preprocessed bill image to be identified by utilizing a pre-trained bill information identification model to obtain the bill information of the bill image to be identified; the bill information recognition model is obtained by training according to a preset training set, wherein the preset training set comprises a plurality of sample bill images of different bill types and sample bill information of each sample bill image;

and the first returning module is used for returning a first response message aiming at the information identification request, and the first response message comprises the bill information of the bill image to be identified.

9. The device according to claim 8, wherein the first recognition module is specifically configured to determine, based on a pre-trained bill information recognition model, an area where each character in the preprocessed bill image to be recognized is located as a character area, and recognize the character in the character area to obtain a character string set; and performing regularization matching on the character string set to obtain bill information of the bill image to be identified.

10. A bill information recognition system is characterized by comprising an Optical Character Recognition (OCR) device and a storage device;

the OCR equipment is used for receiving an information identification request of a bill image to be identified; preprocessing the bill image to be recognized based on the information recognition request to obtain a preprocessed bill image to be recognized; identifying bill information in the preprocessed bill image to be identified by utilizing a pre-trained bill information identification model to obtain the bill information of the bill image to be identified; the bill information recognition model is obtained by training according to a preset training set, wherein the preset training set comprises a plurality of sample bill images of different bill types and sample bill information of each sample bill image; returning a first response message aiming at the information identification request, wherein the first response message comprises bill information of the bill image to be identified;

and the storage equipment is used for storing the bill information of the bill image to be identified.

Images