JP2013246790A - Recognition dictionary creation device and program - Google Patents

Abstract

To reduce labor and time required for creating a recognition dictionary.
A recognition dictionary creation device stores an image of an article captured by an imaging unit in a storage unit. In addition, the recognition dictionary creation apparatus processes the image stored in the storage unit into an image obtained by capturing the articles included in the image from different directions with the imaging unit. Furthermore, the recognition dictionary creation device extracts feature amount data of articles included in the image from the processed image. The recognition dictionary creation device registers the extracted feature data in the recognition dictionary file.
[Selection] Figure 4

Description

  Embodiments described herein relate generally to a recognition dictionary creation device used in a system for recognizing an article included in an image from an image, and a program for causing a computer to function as the recognition dictionary creation device.

  The feature quantity of the article is extracted from the image data obtained by imaging the target article (object) with the imaging unit, and the similarity is calculated by comparing with the feature quantity data registered in the recognition dictionary. There is a technique for recognizing the type of the article. Such a technique for recognizing an article included in an image is called generic object recognition. Regarding this general object recognition technique, various recognition techniques are described in the following documents.

Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>
Further, a technique for performing general object recognition by dividing an image into regions for each object is described in the following document.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [Search August 10, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = repl & type = pdf>
In recent years, for example, in an accounting system (POS system) of a retail store, it has been proposed to apply a general object recognition technique to a device that recognizes a product purchased by a customer.

JP 2010-237886 A

  In general object recognition technology, it is important to enhance the recognition dictionary in order to improve recognition accuracy. In the recognition dictionary, feature amount data that represents surface information such as the external shape, color, pattern, and unevenness of the target article as parameters is stored. However, even for the same product, the surface information varies depending on the orientation of the product. In addition, it is not certain in which direction the user points the article toward the imaging unit. Therefore, it is necessary to extract feature amounts of surface information from image data obtained by photographing an article in advance from various angles, and register these feature quantities in the recognition dictionary as feature quantity data of the article. For this reason, it takes a lot of labor and time to create a recognition dictionary.

  The problem to be solved by the present invention is to provide a recognition dictionary creation device that can reduce the labor and time required to create a recognition dictionary.

  In one embodiment, the recognition dictionary creation device includes a file access unit, a storage unit, a processing unit, an extraction unit, and a data registration unit. The file access means accesses a recognition dictionary file in which feature amount data representing the surface information of each article is stored for each article to be recognized. The storage means stores an image of the article imaged by the imaging means. The processing means processes the image stored in the storage means into an image obtained by imaging the article included in the image from different directions by the imaging means. The extraction unit extracts feature amount data of an article included in the image from the image processed by the processing unit. The data registration unit registers the feature amount data extracted by the extraction unit in the recognition dictionary file via the file access unit.

The external view of the store accounting system which is one Embodiment. The block diagram which shows the hardware constitutions of the goods reading apparatus and POS terminal of a store accounting system. The schematic diagram which shows the data structure of the recognition dictionary file used with a store accounting system. The block diagram which shows the function structure which CPU of the goods reading apparatus implement | achieves when the recognition dictionary creation mode is selected by the goods reading apparatus in 1st Embodiment. FIG. 5 is a schematic diagram showing an area configuration of a storage unit necessary for realizing each function shown in FIG. 4 in the first embodiment. The figure for demonstrating the rotation direction of an image in 1st Embodiment. The flowchart which shows the procedure of the information processing which CPU of a goods reading apparatus performs according to a recognition dictionary creation program in 1st Embodiment. The flowchart which shows the procedure of the front image process of step ST107 in FIG. 7 concretely. 8 is a flowchart specifically showing a procedure of processed image processing in step ST108 in FIG. The figure for demonstrating the rotation direction of an image in 2nd Embodiment. FIG. 5 is a schematic diagram showing an area configuration of a storage unit necessary for realizing each function shown in FIG. 4 in the second embodiment. The flowchart which shows the procedure of the information processing which CPU of a goods reading apparatus performs according to a recognition dictionary creation program in 2nd Embodiment. FIG. 13 is a flowchart specifically showing a procedure of a photographing process in step ST4 in FIG. FIG. 13 is a flowchart specifically showing the procedure of front image processing in step ST6 in FIG. 12; The flowchart which shows concretely the procedure of the process image process of step ST9 and step ST14 in FIG. The flowchart which shows the procedure of the back surface image process of step ST11 in FIG. 7 concretely. The block diagram which shows the function structure which CPU of the goods reading apparatus implement | achieves when the recognition dictionary creation mode is selected by goods reading apparatus in 3rd Embodiment. 10 is a flowchart specifically showing a procedure of processed image processing in the third embodiment. The block diagram which shows the function structure which CPU of the product reading apparatus implement | achieves in 4th Embodiment, when the recognition dictionary creation mode is selected by the product reading apparatus. The schematic diagram which shows the data structure of the flag table used in 4th Embodiment. The flowchart which shows only the principal part of the procedure of the information processing which CPU of a goods reading apparatus performs according to the recognition dictionary creation program in 4th Embodiment. The flowchart which shows only the principal part about the procedure of the process image process in 4th Embodiment.

  Hereinafter, an embodiment according to a recognition dictionary creation device will be described with reference to the drawings. In the present embodiment, the product reading device 1 used in the store accounting system (POS system) is provided with the function of a recognition dictionary creation device.

(First embodiment)
First, a first embodiment will be described with reference to FIGS.
FIG. 1 is an external view of a store accounting system. This system includes a product reading device 1 as a registration unit for registering products purchased by a customer, and a POS (Point Of Sales) terminal 2 as a settlement unit for processing payment for the customer. The commodity reading device 1 is attached on the accounting counter 3. The POS terminal 2 is installed on the cash register 4 via the drawer 5. The product reading device 1 and the POS terminal 2 are electrically connected by a communication cable (not shown).

  The merchandise reading apparatus 1 includes a keyboard 11, a touch panel 12, and a customer display 13. These display / operation devices (keyboard 11, touch panel 12, customer display 13) are attached to a thin rectangular housing 1 </ b> A constituting the main body of the commodity reading apparatus 1.

  The housing 1A incorporates an imaging unit 14 as imaging means. A rectangular reading window 1B is formed in front of the housing 1A. The imaging unit 14 includes a CCD (Charge Coupled Device) imaging device that is an area image sensor and its drive circuit, and an imaging lens for forming an image of the imaging region on the CCD imaging device. The imaging region refers to a region of a frame image that forms an image on the area of the CCD imaging device from the reading window 1B through the imaging lens. The imaging unit 14 outputs an image of the imaging region formed on the CCD imaging device through the imaging lens.

  The POS terminal 2 includes a keyboard 21, an operator display 22, a customer display 23, and a receipt printer 24 as devices necessary for payment.

  The checkout counter 3 has an elongated shape along the customer passage on the back side. The cash register 4 is placed substantially perpendicular to the transaction counter 3 on the front side of the end of the transaction counter 3 on the downstream side with respect to the moving direction of the customer moving along the transaction counter 3. The front side of the accounting counter 3 and the front side of the cash register 4 serve as a space for so-called cashiers in charge of accounting.

  In the approximate center of the accounting counter 3, the housing 1A of the commodity reading apparatus 1 is erected with the keyboard 11, the touch panel 12, and the reading window 1B facing the cashier side on the front side. The customer display 13 of the commodity reading apparatus 1 is attached to the housing 1A so as to face the customer passage.

  The receiving surface on the upstream side of the customer movement direction across the commodity reading device 1 of the accounting counter 3 is a space for placing a shopping basket 6 in which unregistered commodity M to be purchased by the shopper is placed. In addition, the downstream receiving surface is a space for placing a shopping basket 7 into which the product M registered by the product reading device 1 is placed.

  FIG. 2 is a block diagram illustrating a hardware configuration of the commodity reading device 1 and the POS terminal 2. The commodity reading apparatus 1 includes a CPU (Central Processing Unit) 101 as a control unit main body. A ROM 103 and a RAM 104 are connected to the CPU 101 via a bus line 102 such as an address bus or a data bus. The ROM 103 stores a program executed by the CPU 101.

  In addition, the imaging unit 14 is connected to the bus line 102 via an input / output circuit (not shown). In addition, the keyboard 11, the touch panel 12, and the customer display 13, which are display / operation devices, are connected to the bus line 102 via the connection interface 105 and the connection interface 106. The touch panel 12 includes a panel display unit 121 using, for example, a liquid crystal display, and a touch panel sensor 122 arranged on the screen of the display unit.

  The POS terminal 2 also includes a CPU 201 as a control unit main body. The CPU 201 is connected to a ROM 203, a RAM 204, an HDD (Hard Disk Drive) device 205, a communication interface 206, and a connection interface 207 via a bus line 202. Further, the keyboard 21, the operator display 22, the customer display 23, the printer 24, and the drawer 5 are connected to the bus line 202 through input / output circuits (not shown).

  The communication interface 206 is connected to a store computer SC serving as the center of the store via a network such as a LAN (Local Area Network). With this connection, the POS terminal 2 can transmit / receive data to / from the store computer SC. For example, the POS terminal 2 receives a product data file (not shown) and a recognition dictionary file 8 from the store computer SC. The received product data file and recognition dictionary file 8 are stored in the HDD device 205.

  The connection interface 207 is connected to both connection interfaces 105 and 106 of the commodity reading apparatus 1 via a communication cable. With this connection, the POS terminal 2 receives the product information read by the product reading device 1. Further, the product reading device 1 can access the product data file and the recognition dictionary file 8 stored in the HDD device 205.

  FIG. 3 is a schematic diagram showing the structure of dictionary data stored in the recognition dictionary file 8. As shown in FIG. 3, the recognition dictionary file 8 stores N (N is a plurality) feature amount data in association with a product ID and a product name for identifying a product for each product to be recognized. The The feature amount data represents the feature amount of the surface information of the product identified by the corresponding product ID as a parameter.

  A unique product ID is assigned to each product sold in the store. In the product data file, product information such as product name, unit price, and classification code is preset in association with the product ID of each product.

  The product reading device 1 has at least a product recognition mode and a recognition dictionary creation mode as business modes. The product recognition mode is a mode for recognizing a product purchased by a customer and outputting the recognition result to the POS terminal 2. In the recognition dictionary creation mode, a feature dictionary data of an unregistered new product is added to the recognition dictionary file 8 or a feature dictionary data of an existing product registered in the recognition dictionary file 8 is updated. This is a mode for creating (editing) 8. These business modes can be selected from a menu screen displayed on the touch panel 12, for example. Regarding the recognition dictionary creation mode, it is preferable to limit the operators that can be selected by providing authentication means such as password entry.

  When the product recognition mode is selected, the CPU 101 outputs an imaging on signal to the imaging unit 14 to image the imaging area. Then, the CPU 101 captures an image of a product from the image data of the imaging area, and extracts a feature amount related to the surface information of the product from the captured product image. Further, the CPU 101 collates the extracted feature quantity with the feature quantity data registered in the recognition dictionary file 8 to calculate the similarity. If feature amount data whose similarity exceeds a predetermined threshold is registered in the recognition dictionary file 8, the CPU 101 recognizes the product ID associated with the feature amount data as a customer's purchased product, and performs POS. Output to terminal 2.

  On the other hand, when the recognition dictionary creation mode is selected, the CPU 101 performs the functions shown in the block diagram of FIG. 4, that is, the angle reception unit 41, the storage unit 42, the processing unit 43, the extraction unit 44, and the data registration unit 45. Is realized. All of these functions are realized according to a recognition dictionary creation program (not shown) stored in the ROM 103 which is a program storage unit.

  The angle receiving unit 41 receives a rotation angle input via the input unit 31 such as the keyboard 11 or the touch panel 12. The storage unit 42 causes the storage unit 32 to store the product image captured by the imaging unit 14. The processing means 43 processes the product image stored in the storage unit 32 into an image obtained by capturing the product included in the image from different directions with the imaging unit 14. The extraction unit 44 extracts feature amount data related to the surface information of the product included in the image from the image processed by the processing unit 43. The data registration unit 45 registers the feature amount data extracted by the extraction unit 44 in the recognition dictionary file 8 via the file access unit 46. The file access means 46 is means for accessing the recognition dictionary file 8 stored in the HDD device 205 of the POS terminal 2, and corresponds to the connection interface 105 in this embodiment.

  The storage unit 32 is formed in the RAM 104. As shown in FIG. 5, the storage unit 32 includes a reference image memory 321, a processed image memory 322, and a feature amount data buffer memory 323.

  The reference image memory 321 stores a frame image serving as a reference for processing. In the present embodiment, an image captured by the imaging unit 14 is used as a reference image.

  The processed image memory 322 stores an image processed from the reference image. In the present embodiment, as shown in FIG. 6, an image obtained by rotating the reference image 51 at a predetermined angle in the right direction r around the center O of the reference image (frame image) 50 is stored in the processed image memory 322. Is done. Note that the rotation direction of the image is not limited to the right direction r, and may be the opposite left direction.

  The buffer memory 323 stores feature amount data of the same product extracted from the reference image and the plurality of processed images.

  FIG. 7 is a flowchart showing the main information processing procedure of the CPU 101 when the recognition dictionary creation mode is selected. When the recognition dictionary creation mode is selected, the recognition dictionary creation program is activated. By starting this program, the CPU 101 causes the touch panel 12 to display a reception screen for the product ID, the rotation angle θ, and the number of samples n (ST101). Then, the CPU 101 waits for the product ID, the rotation angle θ, and the sample number n to be input from the reception screen (ST102: angle reception unit 41).

  The rotation angle θ is a rotation angle per process when the processing unit 43 processes an image obtained by rotating the reference image in the right direction r. In this embodiment, the processing means 43 processes the image obtained by rotating the reference image in the right direction r by the rotation angle θ by the number n of samples, and each time the extraction means 44 uses the processed image to produce a product. Are extracted and stored in the buffer memory 323.

  The operator who has confirmed the reception screen operates the touch panel 12 or the keyboard 11 to input the product ID of the dictionary creation target product, the desired rotation angle θ, and the number of samples n. A product name may be input instead of the product ID. Moreover, the input of the sample number n is not essential. A setting table in which the rotation angle θ and the sample number n are associated in advance is prepared, and if the rotation angle θ is selected, the sample number n may be uniquely determined. Also, a default angle may be set in advance for the rotation angle θ, and the default angle may be set as the rotation angle θ when the rotation angle θ is not input.

  If product ID, rotation angle θ, and sample number n are input by touch input on touch panel 12 or key input on keyboard 11 (YES in ST102), CPU 101 receives the input product ID, rotation angle θ, and sample. The number n is stored in the work area of the RAM 104 (ST103). Next, CPU 101 waits for an imaging key to be input (ST104).

  The operator who has input the product ID, the rotation angle θ, and the number of samples n of the dictionary creation target product holds the product surface over the reading window 1B. Then, the operator inputs a photographing key provided on the keyboard 11 or the touch panel 12.

  When the photographing key is input (YES in ST104), CPU 101 outputs an imaging on signal to imaging unit 14 (ST105). The imaging unit 14 images the imaging area in accordance with the imaging on signal. As a result, the surface of the product that the operator holds over the reading window 1 </ b> B is imaged by the imaging unit 14. The CPU 101 captures the frame image captured by the imaging unit 14. The CPU 101 stores the frame image in the reference image memory 321 (ST106: storage means 42).

  Thereafter, the CPU 101 executes reference image processing specifically shown in FIG. 8 (ST107). First, the CPU 101 resets the first counter i to “0” (ST201). Next, the CPU 101 extracts feature amount data of products included in the image from the frame image in the reference image memory 321 (ST202).

  Specifically, the CPU 101 first detects the presence or absence of a skin color region from the frame image. When the skin color area is detected, that is, when the reflection of the operator's hand is detected, the CPU 101 extracts a contour line or the like from the binarized image of the frame image. As a result, the CPU 101 attempts to extract the contour of the product M that is assumed to be held by the operator's hand. When the contour of the product M is extracted, the CPU 101 divides the image in the contour into grid-like patches, and reads the feature amount such as the shape of the product, the color of the surface, the pattern, and the unevenness state for each patch.

  When the feature amount data for each patch of the product is extracted from the frame image of the reference image memory 321 in this way, the CPU 101 uses the extracted feature amount data as the feature amount data i (i is the value of the first counter i) of the buffer memory 323. Stored in the area corresponding to (value) (ST203). Thus, the reference image processing ends. Therefore, the area corresponding to the feature amount data 0 of the buffer memory 323 stores feature amount data for each patch of the product extracted from the frame image obtained by capturing the dictionary creation target product by the imaging unit 14.

  When the reference image processing ends, the CPU 101 executes processed image processing specifically shown in FIG. 9 (ST108).

  First, the CPU 101 resets the second counter j to “0” (ST301). Next, the CPU 101 counts up the second counter j by “1” (ST302). Then, the CPU 101 confirms whether or not the second counter j has exceeded the sample number n (ST303).

  If second counter j does not exceed sample number n (NO in ST303), CPU 101 increments first counter i by “1” (ST304). Further, the CPU 101 processes the image in the reference image memory 321 into an image rotated in the right direction r by an angle jθ (j is a value of the second counter j and θ is a rotation angle) (ST305: processing unit 43).

  The CPU 101 stores the processed image in the processed image memory 322. Then, the CPU 101 extracts the feature amount data of the product included in the image from the image stored in the processed image memory 322 in the same manner as the process of step ST202 (ST306: extraction means 44). CPU 101 stores the extracted feature data in an area corresponding to feature data i (i is the value of first counter i) in buffer memory 323 (ST307).

  When the process of step ST307 ends, the CPU 101 returns to the process of step ST302. That is, the CPU 101 further increments the second counter j by “1”. Then, the CPU 101 confirms whether or not the second counter j has exceeded the sample number n (ST303).

  Until the second counter j exceeds the number of samples n, the CPU 101 repeatedly executes the processes of steps ST304 to ST307. If second counter j exceeds sample number n (YES in ST303), the processed image processing ends.

  When the processed image processing is completed, the CPU 101 executes dictionary update processing (ST109). That is, the CPU 101 associates the product ID stored in the work memory with the feature data 0 to N (N = n + 1) stored in the buffer memory 323. Then, the CPU 101 additionally registers the product ID and the feature data 0 to N in the recognition dictionary file 8 stored in the HDD device 205 of the POS terminal 205 via the connection interface 106 (data registration unit 45). .

  At this time, if data of the same product ID has already been registered in the recognition dictionary file 8, the CPU 101 deletes the existing data and then recognizes the current product ID and the feature data 0 to N as a recognition dictionary. It may be additionally registered in the file 8. Alternatively, the CPU 101 may overwrite existing data with the current feature data 0 to N.

  The store computer SC periodically accesses the POS terminal 2 and checks whether the recognition dictionary file 8 stored in the POS terminal 2 has been updated. If it has been updated, the store computer SC takes in the updated recognition dictionary file 8 and recreates its own recognition dictionary file 8. At this time, if there is data without a product name, the store computer SC refers to the product data file and adds the product name associated with the product ID to the data. Then, the store computer SC downloads the recreated recognition dictionary file 8 to the POS terminal 2.

  As described above, according to the present embodiment, the product reading device 1 is used to add feature quantity data of a new product that is not registered in the recognition dictionary file 8, or features of existing products that are registered in the recognition dictionary file 8. The quantity data can be updated.

  In this case, all the work required for the operator is to hold the dictionary creation target product over the reading window 1B. The operator is not required to perform the troublesome work of gradually shifting the direction of the product over which the product is held so that the product is photographed from all angles. Therefore, compared with the conventional case where image data obtained by photographing an article from various angles in advance is required, labor and time required for creating a recognition dictionary are greatly reduced. In addition, since the operator simply holds the product for which the dictionary is to be created over the reading window 1B, it is possible to create a dictionary that does not depend on the level of proficiency of the operator.

  In this embodiment, the rotation angle θ and the sample number n are input together with the product ID of the dictionary creation target product. For example, for products (such as gift certificates) that are composed of simple rectangles or simple patterns, there are few features related to surface information (shape, pattern, color, etc.) obtained from the appearance of the product. There is a possibility that another product may be erroneously recognized due to a slight difference in the angle of the product facing the reading window 1B of the reading device. Therefore, by reducing the rotation angle θ and increasing the number of samples n, images of various angles can be obtained for one product, and feature amount data including more feature amounts can be registered in the recognition dictionary file 8. Therefore, misrecognition can be reduced. On the other hand, for products with complex surface information (shape, pattern, color, etc.), feature data including more feature values can be obtained from one image, so the number n of samples can be reduced. Operation such as saving the number of feature amount data registered in the recognition dictionary file 8 can be performed.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIGS. In this second embodiment, an external view of a store accounting system, a block diagram showing a hardware configuration of a product reading device and a POS terminal, a schematic diagram showing a data structure of a recognition dictionary file, and a recognition dictionary creation mode in the product reading device The block diagram showing the functional configuration realized by the CPU of the commodity reading apparatus when the item is selected is the same as that of the first embodiment, so FIGS. 1 to 4 are used as they are. Common parts are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the first embodiment, the case where the processing unit 43 processes an image obtained by rotating the reference image 51 in the two-dimensional direction with the center O of the reference image 51 as the rotation center is illustrated. In the second embodiment, a case where the processing unit 43 processes an image obtained by rotating the reference image 51 in the three-dimensional direction is illustrated. That is, as shown in FIG. 10A, the processing unit 43 uses the perpendicular line 52 passing through the center of the reference image 51 as a rotation axis, and the reference image 51 is defined in one direction r1 (hereinafter, defined as the right direction r1). Alternatively, an image rotated at a predetermined angle in the opposite direction r2 (hereinafter defined as the left direction r2) is processed. Alternatively, as shown in FIG. 10B, the reference image 51 is defined as one direction r3 (hereinafter, defined as the forward direction r3) with the horizontal line 53 passing through the center of the reference image (frame image) 51 as the rotation axis. Alternatively, the image rotated at a predetermined angle in the reverse direction r4 (hereinafter defined as the backward direction r4) is processed.

  In the second embodiment, as shown in FIG. 11, the storage unit 32 includes a reference image memory 321, a processed image memory 322, a feature amount data buffer memory 323, a front image memory 324, a back image memory 325, A left side image memory 326, a right side image memory 327, a top image memory 328, and a bottom image memory 329 are provided.

  The front image memory 324 stores a frame image when the imaging unit 14 images the front of the dictionary creation target product. Similarly, the back side image memory 325, the left side image memory 326, the right side image memory 327, the top side image memory 328, and the bottom side image memory 329 include the back side, left side, right side, top side, and bottom side of the dictionary creation target product, respectively. A frame image obtained by the imaging unit 14 is stored.

  The reference image memory 321 stores a frame image serving as a reference for processing among the frame images of the front surface, the back surface, the left side surface, the right side surface, the top surface, and the bottom surface. In the present embodiment, a front or back frame image is used as a reference image.

  The processed image memory 322 stores an image rotated from the reference image in the right direction r1, the left direction r2, the front direction r3, or the rear direction r4 by a predetermined angle. The buffer memory 323 stores feature amount data of the same product extracted from the reference image and the plurality of processed images.

  FIG. 12 is a flowchart showing the main information processing procedure of the CPU 101 when the recognition dictionary creation mode is selected. When the recognition dictionary creation mode is selected, the recognition dictionary creation program is activated. By starting this program, the CPU 101 causes the touch panel 12 to display a reception screen for the product ID, the rotation angle θ, and the number of samples n (ST1). Then, the CPU 101 waits for the product ID, the rotation angle θ, and the sample number n to be input from the reception screen (ST2: angle reception means 41).

  The rotation angle θ is a rotation angle per process when the processing unit 43 processes an image obtained by rotating the reference image in the left, right, front, and rear directions, respectively. In the present embodiment, the processing means 43 processes the image obtained by rotating the reference image by one rotation angle θ in one direction and the opposite direction by the number n of samples, and each time the extraction means 44 performs processing. The feature amount data of the product is extracted from the obtained image and stored in the buffer memory 323.

  The operator who has confirmed the reception screen operates the touch panel 12 or the keyboard 11 to input the product ID of the dictionary creation target product, the desired rotation angle θ, and the number of samples n. A product name may be input instead of the product ID. Moreover, the input of the sample number n is not essential. A setting table in which the rotation angle θ and the sample number n are associated in advance is prepared, and if the rotation angle θ is selected, the sample number n may be uniquely determined. Also, a default angle may be set in advance for the rotation angle θ, and the default angle may be set as the rotation angle θ when the rotation angle θ is not input.

  If product ID, rotation angle θ, and number of samples n are input by touch input on touch panel 12 or key input on keyboard 11 (YES in ST2), CPU 101 receives the input product ID, rotation angle θ, and sample. The number n is stored in the work area of the RAM 104 (ST3). Next, the CPU 101 executes a photographing process specifically shown in FIG. 13 (ST4).

  First, the CPU 101 resets the third counter k to “0” (ST21). Next, the CPU 101 displays a shooting direction instruction screen on the touch panel 12 (ST22). On the shooting direction instruction screen, an image for instructing the operator of the shooting direction of the dictionary creation target product is displayed. The shooting direction is determined by the value of the third counter k. In the present embodiment, when the third counter k is “0”, the front side is “1”, the left side is “2”, the right side is “3”, the top side is “4”, and “5”. In this case, the bottom is used.

  The operator who has confirmed the photographing direction instruction screen holds the surface of the commodity designated as the photographing direction over the reading window 1B. Then, the operator inputs a photographing key provided on the keyboard 11 or the touch panel 12.

  CPU 101 waits for an imaging key to be input (ST23). If the photographing key is input (YES in ST23), CPU 101 outputs an imaging on signal to imaging unit 14 (ST24). The imaging unit 14 images the imaging area in accordance with the imaging on signal. As a result, the surface of the product that the operator holds over the reading window 1 </ b> B is imaged by the imaging unit 14. The CPU 101 captures the frame image captured by the imaging unit 14. Then, the CPU 101 stores the frame image in the image memories 321, 322, 323, 324, 325, or 326 corresponding to the direction specified on the shooting direction instruction screen (ST25: storage means 42).

  Thereafter, the CPU 101 counts up the third counter k by “1” (ST26). Then, the CPU 101 confirms whether or not the third counter k has exceeded the maximum value “5” (ST27). If third counter k does not exceed maximum value “5” (NO in ST27), CPU 101 returns to the process in step ST22. That is, the CPU 101 causes the touch panel 12 to display a shooting direction instruction screen determined by the value of the third counter k. And CPU101 performs the process of step ST23-ST27 again.

  Thus, every time the third counter k is incremented, the CPU 101 repeatedly executes the processes of steps ST22 to ST27. If third counter k exceeds maximum value “5” (YES in ST27), the photographing process is terminated.

  Thus, the frame image obtained by imaging the front of the dictionary creation target product is stored in the front image memory 324, the frame image obtained by imaging the back is stored in the back image memory 325, and the frame image obtained by imaging the left side is stored in the left side image memory 326. The frame image obtained by imaging the right side is stored in the right side image memory 327, the frame image obtained by imaging the upper surface is stored in the upper image memory 328, and the frame image obtained by imaging the lower surface is stored in the bottom image memory 329. The

  When the photographing process is completed, the CPU 101 resets the fourth counter r to “0” (ST5). Then, the CPU 101 executes front image processing specifically shown in FIG. 14 (ST6).

  First, the CPU 101 resets the first counter i to “0” (ST31). Next, the CPU 101 copies the frame image data stored in the front image memory 324 to the reference image memory 321 (ST32). Thereafter, the CPU 101 extracts the feature amount data of the product included in the image from the frame image in the reference image memory 321 (ST33). Also in this case, as described in the process of step ST202, feature amount data for each patch is extracted.

  If the feature amount data of the product is extracted from the frame image of the reference image memory 321, the CPU 101 corresponds the extracted feature amount data to the feature amount data i (i is the value of the first counter i) of the buffer memory 323. Store in the area (ST34). This completes the front image processing. Therefore, in the area corresponding to the feature amount data 0 of the buffer memory 323, the feature amount data of the product extracted from the frame image obtained by capturing the front of the dictionary creation target product is stored.

  When the front image processing is completed, the CPU 101 counts up the fourth counter r by “1” (ST7). Then, the CPU 101 confirms whether or not the fourth counter r has exceeded the maximum value “4” (ST8). When the fourth counter r does not exceed the maximum value “4” (NO in ST8), the CPU 101 executes processed image processing specifically shown in FIG. 15 (ST9).

  First, the CPU 101 resets the second counter j to “0” (ST41). Next, the CPU 101 counts up the second counter j by “1” (ST42). Then, the CPU 101 confirms whether or not the second counter j exceeds a value n / 2 that is ½ of the number of samples n (ST43).

  If second counter j does not exceed value n / 2 (NO in ST43), CPU 101 counts up first counter i by “1” (ST44). Further, the CPU 101 processes the image in the reference image memory 321 into an image rotated in the r direction by an angle jθ (j is a value of the second counter j, θ is a rotation angle) (ST45: processing unit 43).

  Here, “r” indicating the rotation direction is a value of the fourth counter r. In the present embodiment, when the fourth counter r is “1”, the right direction r1; when “4”, the left direction r2, “ When 3 ", the forward direction r3, and when" 4 ", the backward direction r4 is the rotational direction.

  Specifically, when the rotation direction is the right direction r1 or the left direction r2, the front and right sides of the dictionary creation target products stored in the reference image memory 321, the left side image memory 326, and the right side image memory 327, respectively. The CPU 101 creates an image obtained by rotating the product from the front in the right direction r1 or the left direction r2 by the angle jθ. When the rotation direction is the forward direction r3 or the backward direction r4, from the front, top, and bottom images of the dictionary creation target product stored in the reference image memory 321, the top image memory 328, and the bottom image memory 326, respectively. The CPU 101 creates an image in which the product is rotated from the front in the forward direction r3 or the backward direction r4 by an angle jθ.

  The CPU 101 stores the processed image in the processed image memory 322. Then, the CPU 101 extracts the feature amount data of the product included in the image from the image stored in the processed image memory 322 (ST46: extraction means 44). Then, the CPU 101 stores the extracted feature data in an area corresponding to the feature data i (i is the value of the first counter i) in the buffer memory 323 (ST47).

  When the process of step ST47 ends, the CPU 101 returns to the process of step ST42. That is, the CPU 101 further increments the second counter j by “1”. Then, the CPU 101 confirms whether or not the second counter j has exceeded the value n / 2 (ST43).

  Until the second counter j exceeds the value n / 2, the CPU 101 repeatedly executes the processes of steps ST44 to ST47. If second counter j exceeds value n / 2 (YES in ST43), the processed image processing ends.

  When the processed image process ends, the CPU 101 returns to the process of step ST7. That is, the CPU 101 further increments the fourth counter r by “1”. Then, the CPU 101 confirms whether or not the fourth counter r has exceeded the maximum value “4” (ST8). The CPU 101 repeatedly executes the processed image processing in step ST9 until the fourth counter r exceeds the maximum value “4”.

  Therefore, when the fourth counter r is “1”, the feature amount data of the product extracted from the images rotated by the angles θ2, 2θ, 3θ,..., (N / 2) θ in the right direction r1 from the front image. Are stored in areas corresponding to the feature data 1 to n / 2 of the buffer memory 323, respectively. When the fourth counter r is “2”, the feature amount data of the product respectively extracted from the image rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ in the left direction r2 from the front image, It is stored in areas corresponding to the feature data {(n / 2) +1} to n in the buffer memory 323, respectively. When the fourth counter r is “3”, the feature amount data of the product extracted from the images rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ in the forward direction r3 from the front image are respectively The data is stored in areas corresponding to the feature data (n + 1) to (3/2) n of the buffer memory 323, respectively. When the fourth counter r is “4”, the feature amount data of the product respectively extracted from the images rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ from the front image in the backward direction r4, The data is stored in areas corresponding to the feature data {(3/2) n + 1} to 2n in the buffer memory 323, respectively.

  If fourth counter r exceeds maximum value “4” (YES in ST8), CPU 101 resets fourth counter r to “0” (ST10). Then, the CPU 101 executes back surface image processing specifically shown in FIG. 16 (ST11).

  First, the CPU 101 counts up the first counter i by “1” (ST51). Next, the CPU 101 copies the frame image data stored in the back image memory 325 to the reference image memory 321 (ST52). Thereafter, the CPU 101 extracts the feature amount data of the product included in the image from the frame image of the reference image memory 321 in the same manner as in the process of step ST33 (ST53). Then, the CPU 101 stores the extracted feature amount data in an area corresponding to the feature amount data i (i is the value of the first counter i) in the buffer memory 323 (ST54). This is the end of the back surface image processing. Therefore, in the area corresponding to the feature amount data (2n + 1) of the buffer memory 323, the feature amount data of the product extracted from the frame image obtained by capturing the front of the dictionary creation target product is stored.

  When the back surface image processing is completed, the CPU 101 counts up the fourth counter r by “1” (ST12). Then, the CPU 101 confirms whether or not the fourth counter r has exceeded the maximum value “4” (ST13). If fourth counter r does not exceed maximum value “4” (NO in ST13), CPU 101 executes processed image processing (ST14). Since the processed image process in step ST14 is the same procedure as the image processed process in step ST10 described with reference to FIG. 15, the description thereof is omitted here.

  When the processed image process ends, the CPU 101 returns to the process of step ST12. That is, the CPU 101 further increments the fourth counter r by “1”. Then, the CPU 101 confirms whether or not the fourth counter r has exceeded the maximum value “4” (ST13). Until the fourth counter r exceeds the maximum value “4”, the CPU 101 repeatedly executes the processed image processing in step ST14.

  Therefore, when the fourth counter r is “1”, the feature amount data of the product extracted from the images rotated by the angles θ2, 2θ, 3θ,..., (N / 2) θ in the right direction r1 from the back image. Are stored in areas corresponding to the feature amount data (2n + 2) to {(5/2) n + 1} of the buffer memory 323, respectively. When the fourth counter r is “2”, the feature quantity data of the product extracted from the images rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ in the left direction r2 from the back image, The data is stored in areas corresponding to the feature data {(5/2) n + 2} to (3n + 1) in the buffer memory 323, respectively. When the fourth counter r is “3”, the feature quantity data of the product extracted from the images rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ in the forward direction r3 from the back image, The data is stored in areas corresponding to the feature data (3n + 2) to {(7/2) n + 1} in the buffer memory 323, respectively. When the fourth counter r is “4”, the feature amount data of the product extracted from the images rotated by the angles θ, 2θ, 3θ,..., (N / 2) θ in the backward direction r4 from the back image, It is stored in the areas corresponding to the feature data {(7/2) n + 2} to (4n + 1) in the buffer memory 323, respectively.

  If fourth counter r exceeds maximum value “4” (YES in ST13), CPU 101 executes a dictionary update process (ST15). That is, the CPU 101 associates the product ID stored in the work memory with the feature data 0 to N (N = 4n + 1) stored in the buffer memory 323. Then, the CPU 101 additionally registers the product ID and the feature data 0 to N in the recognition dictionary file 8 stored in the HDD device 205 of the POS terminal 205 via the connection interface 106 (data registration unit 45). .

  At this time, if data of the same product ID has already been registered in the recognition dictionary file 8, the CPU 101 deletes the existing data and then recognizes the current product ID and the feature data 0 to N as a recognition dictionary. It may be additionally registered in the file 8. Alternatively, the CPU 101 may overwrite existing data with the current feature data 0 to N.

  As described above, in the present embodiment as well, in the same way as in the first embodiment, by using the product reading device 1, feature quantity data of a new product that has not been registered is added to the recognition dictionary file 8, or the recognition dictionary file 8 It is possible to update the feature amount data of an existing product registered in.

  In this case, all the work required for the operator is to hold each surface of the dictionary creation target product over the reading window 1B according to the instruction on the shooting direction instruction screen. The operator is not required to perform the troublesome work of gradually shifting the direction of the product over which the product is held so that the product is photographed from all angles. Therefore, compared with the conventional case where image data obtained by photographing an article from various angles in advance is required, labor and time required for creating a recognition dictionary are greatly reduced. In addition, since the operator simply has to hold each side of the dictionary creation target product over the reading window 1B, it is possible to create a dictionary independent of the level of proficiency of the operator.

(Third embodiment)
In the first or second embodiment, product feature data is extracted from all images obtained by rotating the reference image by the rotation angle θ and reflected in the recognition dictionary file 8. However, depending on the product, there is a case where the feature amount data is hardly changed only by shifting the rotation angle θ. In the general object recognition technique, the presence of a plurality of feature amount data similar to the recognition dictionary file 8 not only causes an increase in the amount of data but also causes a variation in recognition rate.

  Therefore, as a third embodiment, a recognition dictionary creation apparatus that can not only reduce the labor and time required to create a recognition dictionary but also solve the above problems will be described with reference to FIGS. 17 and 18. Note that the third embodiment is a further development of the first embodiment, and the same reference numerals are given to portions common to the first embodiment, and detailed description thereof is omitted.

  FIG. 17 is a block diagram illustrating a functional configuration realized by the CPU 101 of the product reading device 1 when the recognition dictionary creation mode is selected in the product reading device 1 in the third embodiment. As is apparent from comparison with FIG. 4 which is the same block diagram of the first embodiment, in the third embodiment, the CPU 101 has the angle receiving means 41, the storage means 42, the processing means 43, the extracting means 44, and the In addition to the data registration unit 45, a determination unit 47 is realized.

  The determination unit 47 has a function of determining whether or not the feature amount data extracted by the extraction unit 44 is similar to other feature amount data. This function operates in the processed image processing which is step ST108 in the information processing procedure (see FIG. 7) executed by the CPU 101 when the recognition dictionary creation mode is selected.

  The process image processing procedure in the third embodiment is shown in the flowchart of FIG. In FIG. 18, the procedure from step ST301 to step ST306 is the same as that of the first embodiment. That is, the CPU 101 processes an image obtained by rotating the image in the reference image memory 321 rightward by an angle jθ (ST305: processing means 43), and uses the processed image to obtain feature data of products included in the image. Extract (ST306: extraction means 44).

  Thereafter, the CPU 101 calculates the similarity S of the feature amount data by individually comparing the extracted feature amount data with the feature amount data already stored in the buffer memory 323 (ST401). For example, the CPU 101 collates the parameters constituting the feature amount data, and calculates the similarity S from the number of parameters that match or approximate. Then, the CPU 101 determines whether or not feature amount data having a similarity degree exceeding 70% is already stored in the buffer memory 323 (ST402: determination unit 47).

  When no feature quantity data with a similarity degree exceeding 70% is stored in buffer memory 323 (NO in ST402), CPU 101 uses the extracted feature quantity data as feature quantity data i (i is the first number) in buffer memory 323. 1 (the value of 1 counter i) is stored in the area (ST307).

  On the other hand, when at least one feature amount data having a similarity degree exceeding 70% is stored in buffer memory 323 (YES in ST402), CPU 101 stores the extracted feature amount data in buffer memory 323. Discard without.

  As described above, according to the third embodiment including the determination unit 47, two or more feature amount data having a similarity degree exceeding 70% are registered in the recognition dictionary file 8 as the feature amount data of the same product. There is no. Therefore, it is possible to prevent an increase in the amount of data due to the presence of a plurality of feature amount data similar to the recognition dictionary file 8. In addition, there is an effect that variation in recognition rate can be reduced.

  In the third embodiment, the recognition level which is a threshold value for storing or discarding the feature data is 70%, but this value is not particularly limited. If it is in the range of approximately 60% to 80%, the above-described effects can be obtained.

  In addition, it goes without saying that the same effect can be obtained by providing the determination means 47 in the second embodiment.

(Fourth embodiment)
In the second embodiment, the reference image 51 is processed by rotating the reference image 51 at a predetermined angle in the right direction r1 and the left direction r2 with the vertical line 52 passing through the center of the image as the rotation axis. An image rotated at a predetermined angle in the forward direction r3 and the backward direction r4 with the horizontal line 53 passing through the center of the image as a rotation axis is processed, and feature value data of a product included in the image is extracted for each processed image doing. However, for example, in the case of a product for which feature amount data has already been registered in the recognition dictionary file 8, not all rotation direction data is required. Depending on the product, feature value data obtained from an image processed from this reference image is required, while the front image is a reference image. However, feature value data obtained from an image processed from this reference image is the reference image. May not be necessary.

  Therefore, as a fourth embodiment, a recognition dictionary creation device that can not only reduce the time and effort required to create a recognition dictionary but also solve the above problems will be described with reference to FIGS. 19 to 22.

In addition, the same code | symbol is attached | subjected to the part which is common in 2nd Embodiment, and detailed description is abbreviate | omitted.

  FIG. 19 is a block diagram illustrating a functional configuration realized by the CPU 101 of the product reading device 1 when the recognition dictionary creation mode is selected in the product reading device 1 in the fourth embodiment. In the fourth embodiment, the CPU 101 implements a direction receiving unit 48 in addition to the angle receiving unit 41, the storage unit 42, the processing unit 43, the extracting unit 44, and the data registration unit 45.

  In the fourth embodiment, the flag table 70 having the data structure shown in FIG. The flag table 70 stores execution flags f1 to f4 for each rotation direction of the right direction r1, the left direction r2, the front direction r3, and the rear direction r4. The execution flags f1 to f4 identify whether or not the processing in the corresponding rotation direction is to be executed. In this embodiment, the execution flags f1 to f4 are executed when “1” and not executed when “0”.

  The direction receiving means 48 is a function that receives selection of the rotation direction. This function works when the recognition dictionary creation program is started. That is, when the recognition dictionary creation program is activated, the CPU 101 displays a direction acceptance screen on the touch panel 12 as shown in FIG. 21 (ST71).

  On the direction acceptance screen, check boxes are displayed for each of the right direction r1, the left direction r2, the front direction r3, and the rear direction r4, and a check mark indicating an execution state is entered as a default. When the feature amount data in all directions is necessary, the operator inputs an execution key without removing the check mark from the check box. If there is a direction that does not require feature data, the execution key is input after removing the check mark from the check box corresponding to the direction. The execution key is provided on the keyboard 11 or the touch panel 12.

  CPU 101 waits for input of an execution key (ST72). If the execution key is input, the CPU 101 checks the check box on the direction reception screen. The CPU 101 sets the execution flags f1 to f4 corresponding to the direction in which the check mark is set to “1”, and sets the execution flags f1 to f4 corresponding to the direction in which the check mark is not set to “0”. Thereafter, the CPU 101 proceeds to the process of step ST1 in FIG.

  The execution flags f1 to f4 are referred to in the processed image processing. As shown in FIG. 22, when processing image processing is started, the CPU 101 first checks the execution flag fr (r is the value of the fourth counter r). When the execution flag fr is set to “1”, the CPU 101 proceeds to the process of step ST41 in FIG.

  However, if the execution flag fr has been reset to “0”, the processed image processing ends. That is, the processed image processing is not executed in the direction in which the execution flag fr is reset to “0”.

  As described above, according to the fourth embodiment including the direction receiving unit 48, the processed image processing in the direction that the operator has determined to be unnecessary in the information processing procedure of the CPU 101 when the recognition dictionary creation mode is selected. Can be omitted. Therefore, the information processing time can be shortened. Further, since unnecessary feature amount data is not registered in the recognition dictionary file 8, an effect of preventing an increase in the amount of data can be achieved.

  In the fourth embodiment, the direction reception screen is not limited to the above configuration. In short, any screen configuration that allows the operator to select a desired direction may be used.

  In the fourth embodiment, the direction receiving unit 48 is added to the product reading apparatus 1 of the second embodiment. However, the direction receiving unit 48 is added to the product reading apparatus 1 of the first or third embodiment. However, it goes without saying that the same effect can be achieved.

In addition, this invention is not limited to each embodiment mentioned above.
For example, in each of the above embodiments, the product reading device 1 separate from the POS terminal 2 is provided with the function of the recognition dictionary creation device. However, the product reading device 1 is incorporated into the POS terminal 2 and integrated with the device. The function of a recognition dictionary creation device may be provided. Alternatively, the function of the recognition dictionary creation device can be provided by installing the recognition dictionary creation program in a computer device to which an imaging means such as a digital camera is connected. In this case, the recognition dictionary file 8 may be stored in an internal storage device of the computer device, or may be stored in a storage device of an external device connected online to the computer device.

  In the above embodiments, the processing unit 43 rotates the image in the left, right, front, and rear directions, but the rotation direction is not limited to this. For example, an image rotated in an oblique direction may be processed. In addition to processing a rotating image, color correction may be performed.

  In each of the above embodiments, a recognition dictionary creating program for realizing the functions of the invention is recorded in advance in the ROM 103 which is a program storage unit in the apparatus. However, the present invention is not limited to this, and a similar program may be downloaded from the network to the apparatus. Alternatively, a similar program recorded on the recording medium may be installed in the apparatus. The recording medium may be in any form as long as it can store a program such as a CD-ROM or a memory card and can be read by the apparatus. Further, the function obtained by installing or downloading the program may be realized in cooperation with an OS (operating system) in the apparatus.

  In addition, although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Merchandise reading apparatus, 2 ... POS terminal, 8 ... Recognition dictionary file, 14 ... Imaging part, 32 ... Memory | storage part, 41 ... Angle reception means, 42 ... Storage means, 43 ... Processing means, 44 ... Extraction means, 45 ... Data registration means, 46 ... file access means, 47 ... determination means, 48 ... direction reception means.

Claims (6)

File access means for accessing a recognition dictionary file in which feature quantity data representing surface information of the article is stored for each article to be recognized;
Storage means for storing an image of the article imaged by the imaging means;
Processing means for processing the image stored in the storage means into an image obtained by imaging an article included in the image from a different direction by the imaging means;
Extraction means for extracting the feature amount data of the article included in the image from the image processed by the processing means;
Data registration means for registering the feature data extracted by the extraction means in the recognition dictionary file via the file access means;
A recognition dictionary creation device characterized by comprising: Determination means for determining whether or not the feature quantity data extracted by the extraction means is similar to other feature quantity data;
2. The data registration unit registers, in the recognition dictionary file, feature amount data determined not to be similar by the determination unit among the feature amount data extracted by the extraction unit. Recognition dictionary creation device. Angle receiving means for receiving an input of the rotation angle,
The processing unit processes the image stored in the storage unit into an image obtained by rotating an article included in the image in a predetermined direction by an input angle received by the angle receiving unit. Item 3. The recognition dictionary creation device according to item 1 or 2. Direction receiving means for receiving a selection of the rotation direction,
The processing means processes the article contained in the image from the image stored in the storage means into an image rotated by the input angle received by the angle receiving means in the selection direction received by the rotation receiving means. The recognition dictionary creation device according to claim 3. A computer capable of accessing a recognition dictionary file in which feature amount data representing surface information of an article is stored for each article to be recognized.
Storage means for storing an image of the article imaged by the imaging means in a storage unit;
Processing means for processing the image stored in the storage unit into an image obtained by imaging an article included in the image from different directions by the imaging means;
Extraction means for extracting the feature quantity data of the article included in the processed image from the processed image; and data registration means for registering the extracted feature quantity data in the recognition dictionary file;
Recognition dictionary creation program to function as. The computer,
A function for determining whether the extracted feature data is similar to any feature data already stored in the recognition dictionary file;
The said data registration means is a means to register the feature-value data determined not to be similar in the said extracted feature-value data to the said recognition dictionary file via the said file access function. Recognition dictionary creation program.

Images