KR101771524B1 - Method for extracting price information from a receipt and method for providing price information - Google Patents

Abstract

There is provided a method of extracting information on a store and a commodity price from a receipt image taken by users and a method of processing information extracted from a receipt image provided by users to provide price information for each commodity for each commodity. A method of providing price information extracted from a receipt according to an embodiment of the present invention includes receiving a receipt image from a terminal, extracting text from the receipt image, extracting store information from the extracted text, Extracting product information including a product name and a price from the extracted text, storing the retrieved store information in association with product information in a product price database, receiving a product price search request And a search step of extracting a search result including price information matching the search condition included in the goods price search request from the goods price database and providing the search result to the user. According to the present invention, since the price information of the goods is extracted from the receipt, it is possible not only to obtain prices for a plurality of goods at once, but also to obtain the actually sold prices.

Description

TECHNICAL FIELD The present invention relates to a method for extracting price information from a receipt and a method for providing price information using the method.

The present invention relates to a method for extracting price information from a receipt and a method for providing price information using the method. More particularly, the present invention relates to a method for extracting information on a store and a commodity price from a receipt image taken by users, The present invention relates to a method for processing information extracted from a receipt image and providing price information for each specific product.

An important factor in deciding to purchase a product is price. Therefore, a lot of price comparison sites that compare prices at various sales points of various products are being operated. For example, in the Korean Patent Registration No. 10-1336151, it is possible to easily make a price comparison using information stored as an item of interest of a buyer in an online shopping mall, A method for updating the information is disclosed.

However, existing price comparison sites are aimed at online shopping malls where price information can be acquired easily. The reason why it is difficult to provide price comparison service for offline stores is that it is difficult to reliably acquire price information in the offline shops which is changing every moment.

On the other hand, in the "online / offline integrated price comparison service method and system" of Patent No. 10-1137541, price information of a specific product photographed by a user who visited a local store is stored in a bar code of a specific product Tag image information. However, it is difficult for users to expect to upload and record barcodes and price tag images of all the products in the in-store, and the information recorded in the barcode often uses different management codes for the same products in each store, It is difficult to obtain information. Also, even if the user uploads the product price image, there is no way to confirm whether this information is the latest information, and therefore, it is difficult to obtain useful information in a situation where the price changes instantaneously. In addition, there are many cases where settlement is made by applying the discount at the price described in the price tag, so that there is no guarantee that the price described in the price tag is the actual selling price.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a method of extracting information such as store information, date information, and price information of goods on a receipt so as to accurately acquire price information of each store, And a method for providing price information for each store using the extracted information.

A method for extracting receipt price information according to an embodiment of the present invention includes receiving a receipt image from a terminal, extracting text from a receipt image, extracting store information from the extracted text, A product information extracting step of extracting product information including a product name and a price from the extracted text, and storing the extracted store information in association with the product information in a product price database.

A method of providing price information extracted from a receipt according to an embodiment of the present invention includes receiving a receipt image from a terminal, extracting text from the receipt image, extracting store information from the extracted text, Extracting product information including a product name and a price from the extracted text, storing the retrieved store information in association with product information in a product price database, receiving a product price search request And a search step of extracting a search result including price information matching the search condition included in the goods price search request from the goods price database and providing the search result to the user. When providing search results to a user, they may also provide advertisements related to search conditions. In addition, it is also possible to configure the point to be accumulated to the user who uploaded the price information included in the search result.

The step of extracting the store information includes a step of extracting the business registration number from the extracted text and extracting information on the store of the business in the business entity database using the extracted business registration number . Extracts at least one of an address and a telephone number from the extracted text and searches for at least one of an address pattern and a telephone number pattern and extracts at least one of an address and a telephone number from the extracted text, The information on the matching stores in the business entity database can be extracted using at least one of the numbers.

When receiving the receipt image from the terminal, it is also desirable to receive the location information of the terminal as well. If a store matching at least one of the extracted address and telephone number is not present in the business entity database, the business entity database is searched in the business entity database for stores that are located within a predetermined radius from coordinates corresponding to the location information and extracted from the receipt image It is possible to extract information on the store with the highest matching rate.

In the product information extracting step, only a line having a pattern of a string (referred to as a "second string") + space + number ' among the strings of one line (hereinafter referred to as "first string" Obtaining a product name using the product name dictionary database for the second character string of each of the remaining lines, and obtaining a price for the product using the number. At this time, it is preferable to remove a row including an abbreviation not related to a product name including a claim amount, a total amount, and a sum in the second character string among the lines having the pattern of 'second character string + space + number'.

According to an embodiment, when date information is extracted from the extracted text, and the extracted store information and product information are linked and stored in the product price database, the date information may be linked together and stored in the product price database.

According to the present invention, since the price information of the goods is extracted from the receipt, it is possible not only to obtain prices for a plurality of goods at once, but also to obtain the actually sold prices. Further, not only the latest price information can be grasped by using the date information described in the receipt, but also the flow of price fluctuation can be grasped.

Figure 1 is a network diagram for illustrating the method of the present invention.
2 is a block diagram showing an internal configuration of a price information providing server.
FIG. 3 is a flowchart illustrating a procedure for extracting and storing store information, product information, and date information from a receipt.
4 is a flowchart showing a method of extracting store information.
5 is a flowchart showing a method of extracting product information.
FIG. 6 is a flowchart showing a procedure of displaying product price information and advertisements according to a product price search request and providing a reward.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

Figure 1 is a network diagram for illustrating the method of the present invention.

A user who purchases a product (or uses a service) in an offline store takes a receipt by using the mobile terminal 200 and transmits the receipt image to the price information providing server 100. As the mobile terminal 200, a mobile information terminal device including a camera such as a smart phone, a tablet PC, and a notebook computer can be used. Preferably, the mobile terminal 200 is provided with a dedicated application for performing the receipt photographing and transmission using a dedicated application. However, the present invention is not limited to this, and an email or an SNS (Social Network Service) It is also possible to configure to transmit the photographed image using a tool. In addition, the dedicated application may transmit the photographed receipt image as it is, or may perform preprocessing such as binarization and segmentation to reduce transmission data or to reduce processing time in the server As shown in FIG. According to an embodiment, it is also possible to transmit only the recognized text information after character recognition is performed in the mobile terminal.

The price information providing server 100 extracts the information of the shop, the product information including the product name and the price, the purchased date information and the like from the receipt image received from the mobile terminal 200 and converts it into a database. When the price information providing server 100 receives a search request for price information on a specific product from the search terminal 300, the price information providing server 100 searches the database for data satisfying the condition and provides the data to the search terminal 300. As the search terminal 300, various information terminal devices such as a desktop PC, a smart phone, a tablet PC, and a notebook computer can be used.

In order to facilitate the uploading of the receipt image of the user, in providing the price information for the search request, an advertisement related to the requested product or area is provided together, and a reward is provided to the user who uploaded the receipt image used for providing the price information Can be configured.

Next, the configuration of the price information providing server 100 will be described with reference to FIG.

The price information providing server 100 includes a receipt registration interface 110 for receiving a receipt image from a user. The receipt registration interface 110 may be an interface for communicating with a dedicated application installed in the user's mobile terminal 200 or an interface for receiving a receipt image transmitted via e-mail or SNS. In order to obtain more accurate information about the store, the price information providing server 100 may receive the location information of the user when receiving the receipt image from the user. A dedicated application may be configured so that the user can display the position when the user transfers the receipt image to the store location.

The receipt recognizing unit 120 extracts a character string from the receipt image received from the user and extracts a character string from the extracted character string and stores the store information, And product information including the price, information on the date of purchase, and the like. Specific methods of extracting the store information, product information, etc. will be described later.

The extracted store information and product information are linked and stored in the goods price database 130. [ That is, information on how much a specific product is sold at a particular store is stored. Preferably, the information on the date when the commodity is sold in the store is stored in association with the commodity. From this, the user can ascertain the most recent sales price and the change in sales price.

The search user interface 160 requests the search engine 140 to search according to a search request from a user connected through the Internet or the like and transmits a search result from the search engine 140 to the search terminal 300 And provides it to the search terminal 300. The search request from the user may be price information for a specific product in a few stores located closest to the user's current location. For example, it may be "the gasoline price of the three closest gas stations at the location of the user ". To this end, the search user interface 160 may receive the location information of the user when receiving the search request from the user.

The search engine 140 extracts price information corresponding to the search conditions included in the search request from the goods price DB 130 according to the search request from the search user interface 160 and provides the extracted price information to the search user interface 160. The search condition may include a condition regarding a position, a condition relating to a price range, a condition relating to a product name, a condition relating to a store, and the like.

The search user interface 160 may insert and provide advertisements when providing search results to the search terminal 300. The advertisement insertion unit 150 extracts advertisements corresponding to the search conditions from the advertisement DB 151 and provides the extracted advertisements to the search user interface 160. The search user interface 160 searches the search results And advertisement information from the advertisement inserting unit 150, and provides the generated page to the search terminal 300.

On the other hand, the reward can be provided to the user who uploaded the price information used in the search result. To this end, the commodity price DB also stores information about the user who has uploaded the price information (receipt image). The advertisement settlement unit 170 gives a predetermined point or amount according to the advertisement exposure to the user who provided the price information.

Next, the operation of the receipt recognizing unit 120 will be described with reference to FIG. 3 to FIG.

FIG. 3 is a flowchart illustrating a procedure for extracting and storing store information, product information, and date information from a receipt.

The receipt recognition unit 120 receives the receipt image through the receipt registration interface 110 (S310), and recognizes the character in the received receipt image (S320). To increase the character recognition rate, you can also use a dictionary database for commonly used strings in receipts.

The receipt has a commonly used idiom, regardless of store information, product information, or date information. Examples of abbreviations include 'credit approval', 'thank you', 'lump sum', 'card number', 'approval number', 'purchase', 'for members', 'for customers', 'terminal NO'. And removes such an abbreviation from the character string extracted in step S320 (step S330). On the other hand, the removal of the insoluble words can be performed in the store information extracting step (S340), the product information extracting step (S350), and the like. It is also possible to remove the numeral strings and the like, which are used in the abbreviated terms, when removing the stopwords. For example, the credit card number used for settlement is written next to the abbreviation of 'card number', and when the abbreviation of 'card number' is removed, the credit card number beside it is also removed.

The receipt recognizing unit 120 extracts information on the store (store information), information on the name of the commodity, information on the price of the commodity (commodity information), and date information from the string in which the abbreviations are removed (steps S340, S350, S360 ). The order of extracting store information, product information and date information can be arbitrarily designed. That is, the information may be extracted in order from the beginning to the end of the character string, or the product information may be extracted after extracting the store information and the date information, and the present invention is not limited to the specific order.

The receipt recognizer 120 stores the extracted store information, product information, and date information in the product price database 130 in association with each other.

One example of extracting the store information from the receipt will be described with reference to FIG.

First, it is confirmed whether there is a business registration number pattern in the entire string extracted from the receipt (step S410). The business registration number may appear after the string "business", and the business registration number may appear directly without such a string. The business registration number consists of three digits + '-' + two digits + '-' + five digits', so make sure you have a string with this pattern.

If there is a business registration number pattern, it is confirmed in step S420 whether the business registration number exists in the store information DB 121. [ If there is a corresponding business registration number in the store information DB 121, information (store ID, business name, store address, part or all of the store telephone number) of the store is temporarily stored in step S470.

If it is unsuccessful to extract the business registration number or if there is no corresponding business registration number in the store information DB 121, it is confirmed in step S430 whether address information exists in the entire character string. The address information may include an address and a telephone number. If there is a character string having an address pattern or a telephone number pattern in the entire character string, it is confirmed whether a store having the corresponding address or telephone number exists in the store information DB 121 (step S440). If there is such address information in the store information DB 121, information (store ID, business name, store address, part or all of the store telephone number) of the store is temporarily stored in step S470.

If address information has not been extracted or there is no matching address information in the store information DB 121, it is determined in step S450 that the location information of the store located within a predetermined radius from the location of the terminal Is registered in the store information DB (121). If there is a store within a predetermined radius and there is information matching the store information and the entire string, the information (store ID, business name, store address, part or all of the store telephone number) of the store is temporarily stored I will. According to an embodiment of the present invention, the stores located within a predetermined radius may be searched in the store information DB and compared with the texts extracted from the receipt image, thereby selecting the store with the highest matching rate.

If the store information can not be obtained through such a process, the store information may be separately classified, and the manager may later display the receipt image and input the store information by hand (step S460). Alternatively, it is possible to omit step S460 and discard the receipt image which failed to obtain the store information.

One example of extracting product information will be described with reference to FIG.

First, a row having a price pattern in the entire character string is extracted (step S510). A row having a price pattern means a line having a pattern of a string (referred to as "second string") + space + number 'among the strings of one row (hereinafter referred to as "first string" It says. That is, only the row having the price pattern is extracted by deleting the row having no price pattern in the entire character string.

Subsequently, a line including a product name inscription is removed (step S520). Examples of abbreviated product names include 'bill amount', 'total amount', and 'total'. That is, it removes rows that are not rows representing the price of an individual commodity among rows having a price pattern.

After extracting only the row representing the price of the product, the product name is acquired using the product name dictionary DB 122 for the second character string for each remaining row, and the price for the product is obtained using the number. The product name dictionary DB 122 is a database in which words or strings that are likely to be used as product names are mapped to standard product names. In addition, the product name dictionary DB 122 may be a database in which a product name used for each store is converted into a database. In this case, a product name can be obtained using an appropriate database according to the store information acquired in step S340. The acquired product information is stored in the goods price database 130 in association with the store information temporarily stored in step S470.

Next, referring to FIG. 6, a procedure for displaying product price information and advertisements and providing rewards according to a product price search request from a user will be described.

Upon receiving a goods price search request from the search terminal 300 (step S610), the search user interface 160 extracts search conditions from the received search request and transmits the search conditions to the search engine 140 to obtain price information corresponding to the search conditions (Step S620). The search conditions may include conditions relating to the location of the store, conditions relating to the price range of the goods, conditions relating to the goods, conditions relating to the stores, and the like.

In addition, the search user interface 160 transmits the search condition to the advertisement inserting unit 150 to inquire the advertisement related to the search condition (S630). Examples of the advertisement related to the search condition include an advertisement related to a product requested by a user, an advertisement related to a location requested by the user, an advertisement related to a store requested by the user, and the like.

The search user interface 160 receiving the search results from the search engine 140 and receiving the advertisement information from the advertisement inserting unit 150 constructs a price information page including the advertisement to fit the search terminal 300, To the terminal 300 (step S640).

The search user interface 160 acquires information on the user who has uploaded the price information used in the search result from the goods price database 130 and transmits the acquired information to the advertisement settlement unit 170. The advertisement settlement unit 170 And a predetermined point or reward is accumulated for the user (step S650).

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art. Such a computer program can be stored in a computer-readable storage medium, readable and executed by a computer, thereby realizing an embodiment of the present invention. The storage medium of the computer program may include a semiconductor recording medium, a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100 Price information server,
120 Receipt Recognition Department,
130 Product Price DB,
140 search engine,
150 advertisement insertion unit,
170 Advertisement settlement part,
200 mobile terminal,
300 search terminal.

Claims (20)

Receiving a receipt image and location information of the terminal from the terminal,
A text extracting step of extracting text from the receipt image,
A store information extracting step of extracting store information from the extracted text,
A product information extracting step of extracting product information including a product name and a price from the extracted text,
And storing the extracted store information and product information in association with each other in a product price database,
The step of extracting the store information may include extracting a business registration number from the extracted text and extracting store information having a business registration number from the store information database using the extracted business registration number And extracts at least one of an address and a telephone number from the extracted text and finds out at least one of the address pattern and the telephone number pattern and extracts at least one of the address and the telephone number if the business registration number matches the extracted business registration number, Extracting store information of a store matching an address or a telephone number in the store information database by using at least one of the extracted address and the telephone number; If it is not in the information database, From the corresponding coordinates in the beam compared to the buried existing within a predetermined radius and the text extracted from the receipt of the image search in the store information database, comprising the step of matching the rate of extraction stores information of the highest store,
In the product information extracting step, only a line having a pattern of a string (referred to as a "second string") + space + number 'among the strings of one line (hereinafter referred to as "first string" Removing the remaining lines; obtaining a product name using the product name dictionary database, which is a database in which words or strings that are likely to be used as product names are mapped to standard product names for the second strings in each of the remaining rows; And obtaining a price for the product using the number,
How to extract receipt pricing information. delete delete delete The method according to claim 1, wherein the store information is extracted only when the matching rate of the store with the highest matching rate is equal to or greater than a predetermined reference value. delete delete The method according to claim 1,
Removing a line including an abbreviation not related to a product name including a bill amount, a total amount, and a sum in a second character string among the lines having a pattern of 'second character string + space + number'. The method according to any one of claims 1, 5, and 8,
And a date information extracting step of extracting date information from the extracted text,
When the extracted store information and the merchandise information are linked and stored in the merchandise price database, date information is also linked together and stored in the merchandise price database. The method of any one of claims 1, 5, and 8, further comprising removing an insoluble word that is not explicitly extracted from the extracted text after the text extracting step. Receiving a receipt image and location information of the terminal from the terminal,
A text extracting step of extracting text from the receipt image,
A store information extracting step of extracting store information from the extracted text,
A product information extracting step of extracting product information including a product name and a price from the extracted text,
Storing the extracted store information and product information in a product price database in association with each other;
And a retrieval step of extracting, from the commodity price database, the retrieval result including the price information corresponding to the retrieval condition included in the retrieval request for the commodity price, upon receiving the commodity price retrieval request,
The step of extracting the store information may include extracting a business registration number from the extracted text and extracting store information having a business registration number from the store information database using the extracted business registration number And extracts at least one of an address and a telephone number from the extracted text and finds out at least one of the address pattern and the telephone number pattern and extracts at least one of the address and the telephone number if the business registration number matches the extracted business registration number, Extracting store information of a store matching an address or a telephone number in the store information database by using at least one of the extracted address and the telephone number; If it is not in the information database, From the corresponding coordinates in the beam compared to the buried existing within a predetermined radius and the text extracted from the receipt of the image search in the store information database, comprising the step of matching the rate of extraction stores information of the highest store,
In the product information extracting step, only a line having a pattern of a string (referred to as a "second string") + space + number 'among the strings of one line (hereinafter referred to as "first string" Removing the remaining lines; obtaining a product name using the product name dictionary database, which is a database in which words or strings that are likely to be used as product names are mapped to standard product names for the second strings in each of the remaining rows; , And obtaining the price for the product using the number. delete delete delete delete 12. The method of claim 11,
Removing a line including an illegal word not related to a product name including a charge amount, a total amount, and a sum in a second character string of a line having a pattern of 'second character string + space + number' Way. 12. The method of claim 11,
And a date information extracting step of extracting date information from the extracted text,
When the extracted store information and the merchandise information are linked and stored in the merchandise price database, the date information is also linked together and stored in the merchandise price database. 18. The method according to any one of claims 11, 16, and 17,
Wherein the search condition includes at least one of a condition regarding a location of a store, a condition relating to a price range of a commodity, a condition relating to a commodity, and a condition relating to a commodity. 18. The method according to any one of claims 11, 16, and 17,
And providing an advertisement related to the search condition when providing the search result to the user. 20. The method of claim 19,
Further comprising the step of: accumulating points to the user who uploaded the price information included in the search result.

Images