JP2019096162A - Inventory detection program, inventory detection method and inventory detection apparatus - Google Patents

Abstract

To provide an inventory detection program, an inventory detection method and an inventory detection apparatus that accurately detect a commodity inventory.SOLUTION: An inventory detection program causes a computer to perform: processing to detect a shelf board of a shelf, in which commodities are displayed, using information associated with a distance from a reference point; processing to calculate a depth from the shelf board at a measurement point; and processing to determine an inventory status of the commodity at a position corresponding to the measurement point using the depth.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an inventory detection program, an inventory detection method, and an inventory detection device.

  When managing shelf allocation information and inventory information of a product displayed on a shelf in a store such as a supermarket, there is known a technology for capturing an image of the shelf and detecting the product from the image. For example, techniques are known that receive a rearogram image that includes a plurality of organized objects, and detect and identify the objects in the rearogram image of one or more items on the retail shelf. The technology identifies and labels the foremost part of the shelf, identifies the empty space under the shelf, identifies areas where there may be unidentified products, and "out of stock" products. Identify the area that is

JP, 2016-115348, A

  However, in the above technology, there is a high possibility that false detection of inventory will occur. For example, in the above-described technology, inventory is detected using the brightness of an image, a gray scale value, and the like. However, since luminance and the like are easily influenced by external factors such as external brightness and the like, in the above technology, there is a high possibility that false detection of a product and a shelf board, a product and a back surface of a shelf may occur.

  In one aspect, it is an object of the present invention to provide an inventory detection program, an inventory detection method, and an inventory detection device capable of accurately detecting commodity inventory.

  In one aspect, the inventory detection program causes the computer to execute a process of detecting a shelf board of a shelf on which a product is displayed, using information on the distance from the reference point. The stock detection program causes the computer to execute a process of calculating the depth from the shelf at the measurement point using the information on the distance from the reference point. The stock detection program causes the computer to execute processing for determining the stock status of the product at the position corresponding to the measurement point using the depth.

  According to one aspect, the inventory of goods can be detected accurately.

FIG. 1 is a diagram illustrating an example of distance data acquisition processing according to the first embodiment. FIG. 2 is a diagram showing an example of the stock status. FIG. 3 is a view showing an example of distance measurement data corresponding to the stock situation in the first embodiment. FIG. 4 is a diagram showing an example of distance measurement data converted into point group data. FIG. 5 is a diagram illustrating an example of the detection device in the first embodiment. FIG. 6 is a diagram showing an example of the distance measurement data DB in the first embodiment. FIG. 7 is a diagram illustrating an example of the depth DB in the first embodiment. FIG. 8 is a diagram illustrating an example of the shelf assignment DB according to the first embodiment. FIG. 9A is a diagram illustrating an example of a shelf board detection process according to the first embodiment. FIG. 9B is a diagram illustrating an example of a shelf board detection process according to the first embodiment. FIG. 9C is a diagram illustrating an example of a shelf board detection process according to the first embodiment. FIG. 10 is a diagram illustrating an example of the depth calculation process according to the first embodiment. FIG. 11 is a diagram illustrating an example of an output result in the first embodiment. FIG. 12 is a flowchart illustrating an example of the stock detection process according to the first embodiment. FIG. 13 is a flowchart illustrating an example of a shelf board detection process according to the first embodiment. FIG. 14 is a flowchart illustrating an example of the stock determination process according to the first embodiment. FIG. 15 is a diagram illustrating an example of depth calculation processing in the second embodiment. FIG. 16 is a diagram of an example of distance measurement data corresponding to the stock status in the second embodiment. FIG. 17 is a diagram illustrating an example of a detection device in the second embodiment. FIG. 18 is a view showing an example of distance measurement data DB in the second embodiment. FIG. 19 is a diagram illustrating an example of the depth DB in the second embodiment. FIG. 20 is a flowchart of an example of the stock detection process according to the second embodiment. FIG. 21 is a flowchart of an example of the depth identification process according to the second embodiment. FIG. 22 is a diagram of an example of a computer that executes a detection program.

  Hereinafter, embodiments of an inventory detection program, an inventory detection method, and an inventory detection device disclosed in the present application will be described in detail based on the drawings. The present invention is not limited by this embodiment. In addition, the embodiments described below may be combined appropriately as long as no contradiction occurs.

  The detection apparatus 100 in the present embodiment detects inventory of goods using distance measurement data obtained by measuring the distance from the reference point to the measurement point. FIG. 1 is a diagram illustrating an example of distance data acquisition processing according to the first embodiment. As shown in FIG. 1, the detection device 100 in the present embodiment is, for example, a self-propelled robot. The detection apparatus 100 acquires distance measurement data from the reference point 920 to the measurement point using, for example, a distance measurement sensor that measures the distance to the object by irradiating infrared light, sound waves, or the like. The distance measurement data is an example of information on the distance.

  The detection apparatus 100 specifies the position of the shelf plate 901 of the shelf 900 illustrated in FIG. 1 using, for example, data with the smallest distance among the acquired distance measurement data. Similarly, the detection apparatus 100 measures from the shelf board 901 using distance measurement data at the measurement point 910 corresponding to the display position of the product, which has moved to any coordinate from the position of the shelf board 901 shown in FIG. Measure the depth indicating the depth to point 910. Then, if the depth is 0 or small, the detection apparatus 100 determines that the inventory of the product is sufficient, and if the depth is large, determines that the inventory of the product is small or not at all.

  The relationship between the depth and the stock of goods will be described with reference to FIGS. 2 and 3. FIG. 2 is a diagram showing an example of the stock status. In FIG. 2, the upper direction of the figure corresponds to the depth direction of the shelf, and the lower direction of the figure corresponds to the near direction of the shelf. As shown in FIG. 2, for example, the store clerk 700 grasps the stock of the products 810 to 850 by visual observation from the front side of the shelf.

  In FIG. 2, for example, a sufficient number of products 810 and 850 are on the shelf, but the product 820 has a low product inventory. On the other hand, the product 840 is out of stock.

  The measurement result of the depth which shows the depth from the shelf board 520 in the shelf of such goods is shown in FIG. FIG. 3 is a view showing an example of distance measurement data corresponding to the stock situation in the first embodiment. In FIG. 3, the x-axis indicates the left-right direction of the shelf, and the z-axis indicates the depth direction of the shelf. The z-axis indicates that the greater the numerical value, the greater the depth.

  As shown in FIG. 3, for items 810 with sufficient inventory, the corresponding depth 811 is less than the first threshold 801. The same applies to the depth 851 corresponding to the product 850. On the other hand, the depth 821 corresponding to the low-stock item 820 is included in the range 891 which is equal to or greater than the first threshold 801 but smaller than the second threshold 802. Further, the depth 841 corresponding to the out-of-stock product 840 is included in the range 892 of the second threshold 802 or more.

  The point cloud data acquired as distance measurement data is illustrated in FIG. 4 as an image. FIG. 4 is a diagram showing an example of distance measurement data converted into point group data. In the image 500 shown in FIG. 4, distance measurement data in a small distance portion is shown in a color close to white, and is shown in a color close to black as the distance increases. In addition, for example, a transparent commodity or a portion where the distance can not be measured, such as a commodity including a liquid, is shown in black.

  As described above, the detection device 100 according to the present embodiment detects the shelf board of the display shelf using the distance measurement data indicating the distance from the measurement point, and uses the distance measurement data above a predetermined coordinate from the shelf board Since the stock status of the product is determined by calculating the depth from the board, the stock can be easily detected.

[Function block]
Next, the function of the detection apparatus 100 in the present embodiment will be described using FIG. FIG. 5 is a diagram illustrating an example of the detection device in the first embodiment. As shown in FIG. 5, the detection device 100 in the present embodiment has an external I / F 110, a storage unit 120, and a control unit 130.

  The external I / F 110 controls communication with another computer such as a clerk terminal (not shown) or another database server (not shown), whether wired or wireless. The external I / F 110 is, for example, a communication interface such as a network interface card (NIC). The external I / F 110 acquires distance measurement data from, for example, an external device having a distance measurement sensor. In addition, as shown in FIG. 1, when the detection apparatus 100 is mounted by the robot etc. which have a ranging sensor, you may have a ranging sensor instead of external I / F110.

  The storage unit 120 stores, for example, various data such as a program executed by the control unit 130. The storage unit 120 also includes a distance measurement data DB 121, a depth DB 122, and a shelf allocation DB 123. The storage unit 120 corresponds to a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), a flash memory, or a storage device such as a hard disk drive (HDD). In the following description, a database may be described as "DB".

  The distance measurement data DB 121 stores distance measurement data in association with the coordinates of the measurement point. FIG. 6 is a diagram showing an example of the distance measurement data DB in the first embodiment. As shown in FIG. 6, the distance measurement data DB 121 stores, for example, “shelf number”, “x coordinate” and “y coordinate”, and “distance” in association with each other. The information stored in the distance measurement data DB 121 is input by, for example, a distance acquisition unit 131 described later. The distance measurement data DB 121 stores, for example, one record for each measurement point.

  In FIG. 6, "storage bin" stores an identifier (Identifier) uniquely identifying a product display shelf. The “x coordinate” and the “y coordinate” store the coordinates of the measurement point. The y coordinate indicates the vertical direction of the shelf. "Distance" stores the distance at the measurement point.

  The depth DB 122 stores the depth calculated from the distance measurement data in association with the position on the shelf. FIG. 7 is a diagram illustrating an example of the depth DB in the first embodiment. As illustrated in FIG. 7, the depth DB 122 stores “shelf number”, “shelf”, “x coordinate”, and “depth” in association with one another. The information stored in the depth DB 122 is input by, for example, the depth calculation unit 133 described later.

  In FIG. 7, “shelf” stores the position of the shelf in the shelf of the commodity display shelf. For example, for the "shelf", the top is "1". “Depth” stores the depth indicating the depth from the shelf board to the measurement point, which is calculated from the distance measurement data.

  The shelf allocation DB 123 stores the arrangement of goods on the goods display rack. FIG. 8 is a diagram illustrating an example of the shelf assignment DB according to the first embodiment. As illustrated in FIG. 8, the shelf allocation DB 123 stores “ID”, “shelf number”, “shelf stage”, “product name”, and FACE number in association with one another. The information stored in the shelf allocation DB 123 is input in advance by, for example, a manager (not shown) of the detection device 100.

  In FIG. 8, “ID” is an identifier that uniquely identifies a product displayed on the product display rack. The “left end” and the “right end” store the range in the left-right direction of the position at which the product is displayed, for example, in the x-axis direction. The "product name" stores the name of the displayed product.

  Returning to FIG. 5, the control unit 130 is a processing unit that manages the entire processing of the detection device 100. The control unit 130 is realized, for example, by a program stored in an internal storage device being executed by using a RAM as a work area by a central processing unit (CPU), a micro processing unit (MPU) or the like. Further, the control unit 130 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  The control unit 130 includes a distance acquisition unit 131, a shelf detection unit 132, a depth calculation unit 133, an inventory determination unit 134, and an output unit 135. The distance acquisition unit 131, the shelf board detection unit 132, the depth calculation unit 133, the stock determination unit 134, and the output unit 135 are an example of an electronic circuit included in the processor and an example of a process executed by the processor.

  The distance acquisition unit 131 acquires distance measurement data. For example, the distance acquisition unit 131 acquires distance measurement data from the distance measurement sensor through the external I / F 110, associates the coordinates of the measurement point with the measured distance, and stores the distance measurement data in the distance measurement data DB 121.

  The shelf detection unit 132 detects the shelf using the distance measurement data. The processing by the shelf detection unit 132 will be described with reference to FIGS. 9A to 9C. The shelf detection unit 132 is an example of a detection unit.

  9A to 9C are diagrams showing an example of the shelf board detection process in the first embodiment. First, as illustrated in FIG. 9A, the shelf detection unit 132 specifies a center portion in the left-right direction, for example, a middle line 511 on an x-coordinate straight line, in the distance measurement data acquired as point cloud data. Next, the shelf detection unit 132 specifies a point closest to the distance among the points on the center line 511, for example, a point 521 with the smallest distance measurement data as a first point.

  Next, the shelf board detection unit 132 specifies points moved by predetermined coordinates in the left-right direction of the first point as the second point and the third point. For example, when the shelf detection unit 132 identifies the point 521 as the first point, the shelf detection unit 132 identifies the points 522 and 523 illustrated in FIG. 9B as the second point and the third point.

  Then, the shelf detection unit 132 determines whether the difference between the distance measurement data at each of the second point and the third point and the distance measurement data at the first point is less than a predetermined threshold value. Do. When the shelf detection unit 132 determines that the difference in distance is less than the predetermined threshold value, the point further moved by the predetermined coordinates in the horizontal direction from each of the second point and the third point is newly added As the second and third points. The shelf detection unit 132 repeats the process of identifying the second point and the third point until the new second point and the third point reach the left end 532 and the right end 533 of the point cloud data.

  When the new second and third points reach the left and right ends of the point cloud data, the shelf board detection unit 132 shelves the straight line connecting the second point and the third point in the horizontal direction. Detected as the position of the board. Then, the shelf detection unit 132 specifies a point on the midline 511 that has the smallest distance measurement data next to the first point as a new first point. Further, also in the case where the shelf detection unit 132 determines that the difference between the distance measurement data at each of the second point and the third point and the distance measurement data at the first point is equal to or greater than a predetermined threshold value. , Identify a new first point as well.

  The shelf detection unit 132 starts processing for identifying the first point from the lower end of the point cloud data, for example, the lower end in the Y-axis direction, and reaches a point moved downward by a predetermined coordinate from the upper end in the Y-axis direction. By repeating the process until it is detected, a plurality of shelf boards included in the commodity display shelf are detected. For example, in FIG. 9C, the shelf detection unit 132 starts processing from the lower end Ymin in the Y-axis direction, and performs processing from the upper end Ymax in the Y-axis direction to a point Ymax-h1 moved downward by the predetermined coordinates h1. repeat. Thus, the shelf board detection unit 132 detects the shelf boards 540 to 580 in addition to the shelf board 520. The process is not repeated until the upper end Ymax in the Y-axis direction, and the process is ended when the coordinate is moved down by a predetermined coordinate if there is no space between the shelf board and the upper end of the shelf, on the shelf board. It is because goods can not be displayed.

  The depth calculation unit 133 calculates the depth at the measurement point using the distance measurement data. The processing by the depth calculation unit 133 will be described with reference to FIG. The depth calculation unit 133 is an example of a calculation unit.

  FIG. 10 is a diagram illustrating an example of the depth calculation process according to the first embodiment. First, the depth calculation unit 133 specifies, in the point cloud data, a straight line 611 at a position moved upward by a predetermined coordinate from the position at which the shelf 520 is detected. Next, the depth calculation unit 133 acquires distance measurement data at a point 612 of a specific x coordinate on the straight line 611.

  Then, the depth calculation unit 133 calculates the difference between the distance measurement data at the point 612 and the distance measurement data at the shelf 520 as the depth at the point 612. The depth calculation unit 133 stores the calculated depth in the depth DB 122 in association with the x coordinate of the point 612 and the tray corresponding to the shelf 520.

  For example, the depth calculation unit 133 starts the process from the left end of the shelf in the x coordinate direction, and when the depth is calculated, the depth calculation unit 133 moves to the right by a predetermined coordinate and repeats the process.

  The stock determination unit 134 determines the stock of the product using the depth. The stock determination unit 134 is an example of a determination unit.

  The stock determination unit 134 acquires the depth from the depth DB 122. In addition, the inventory determination unit 134 refers to the shelf allocation DB 123, and identifies a product to be displayed at a position corresponding to the acquired depth. Then, the inventory determination unit 134 compares the acquired depth with the first threshold and the second threshold.

  When the depth is less than the first threshold, the stock determining unit 134 determines that the product displayed at the corresponding position is “stocked”. When the depth is greater than or equal to the first threshold and less than the second threshold, the stock determining unit 134 determines that the product displayed at the corresponding position is “shortage”. When the depth is equal to or greater than the second threshold, the stock determining unit 134 determines that the product displayed at the corresponding position is “out of stock”.

  The output unit 135 outputs the determination result of the stock at each measurement point in association with the information on the shelving. The output unit 135 acquires “shelf number”, “shelf”, and “x coordinate” of each depth from the depth DB 122. Next, the output unit 135 refers to the shelf allocation DB 123, and specifies a product to be displayed at a position corresponding to each “shelf number”, “shelf”, and “x coordinate”. Then, the output unit 135 outputs the stock status of the product displayed at each position.

  For example, the output unit 135 specifies that the shelf number “1”, the tray level “1”, and the x coordinate “80” in the depth DB 122 correspond to the ID “A001” in the shelf allocation DB 123. Then, the output unit 135 outputs information indicating that the product is "in stock".

  FIG. 11 is a diagram illustrating an example of an output result in the first embodiment. As illustrated in FIG. 11, the output unit 135 generates an output result including information indicating that the product “●● ramen” of the ID “A001” is “in stock”. Similarly, the output unit 135 associates each position in the depth DB 122 with each product in the shelf allocation DB 123, and outputs the stock status of each product. For example, if the first threshold is "30" and the second threshold is "50", it is indicated that a product whose depth at the corresponding display position is "40" is "poor" It is shown that a product whose depth at the display position where the display is displayed is "50" is "out of stock".

[Flow of processing]
Next, processing in the present embodiment will be described using FIG. 12 to FIG. FIG. 12 is a flowchart illustrating an example of the stock detection process according to the first embodiment. As shown in FIG. 12, the distance acquisition unit 131 of the detection apparatus 100 stands by until, for example, distance measurement data is acquired through the external I / F 110 (S10: No).

  When the shelf detection unit 132 determines that the distance measurement data has been acquired (S10: Yes), the shelf detection process shown in FIG. 13 is performed (S11). The shelf detection process will be described later. Next, the depth calculation unit 133 calculates the depth (S12). Next, the stock determination unit 134 executes stock determination processing using the calculated depth (S13). The stock determination process will be described later.

  Next, the inventory determination unit 134 specifies the measurement point moved by w coordinates in the x direction (S14), and determines whether the x coordinate of the measurement point is equal to or more than a predetermined threshold (S15). If the depth calculation unit 133 determines that the x coordinate is less than the predetermined threshold (S15: No), the process returns to S12 to repeat the process.

  On the other hand, when the stock determining unit 134 determines that the x coordinate is equal to or more than the predetermined threshold (S15: Yes), the stock determining unit 134 determines whether the processing has been completed for all the shelf boards (S16). When it is determined that the process has not been completed for all the shelf boards (S16: No), the depth calculation unit 133 identifies the next shelf board detected (S17), and returns to S12 to repeat the process.

On the other hand, if the output unit 135 determines that the processing has been completed for all the shelf boards (S16)
Yes: The shelf allocation DB 123 is referred to, and the coordinates stored in the depth DB 122 are matched with the shelf allocation information (S18). Then, the output unit 135 outputs the matching result (S19), and ends the process.

  Next, the shelf detection process will be described. FIG. 13 is a flowchart illustrating an example of a shelf board detection process according to the first embodiment. As illustrated in FIG. 13, the shelf board detection unit 132 specifies a middle line on the x-coordinate straight line (S110). Next, the shelf detection unit 132 identifies a point closest to the center line as a first point (S111).

  Then, the shelf board detection unit 132 specifies the second point and the third point (S113). The shelf detection unit 132 compares the distance measurement data of the first point with the distance measurement data of each of the second point and the third point, and determines whether the difference in distance is less than a predetermined threshold value. It determines (S114). When the shelf detection unit 132 determines that the difference in distance is equal to or greater than the predetermined threshold (S114: No), the processing proceeds to S117.

  If the shelf detection unit 132 determines that the difference in distance is less than the predetermined threshold (S114: Yes), the second point and the third point reach both left and right ends in the point cloud data, respectively. , S113 and the process is repeated (S115: No).

  Then, when the shelf detection unit 132 determines that the second point and the third point have reached both left and right ends in the point cloud data (S115: Yes), the second point and the third point Is detected as a shelf board (S116).

  Then, the shelf board detection unit 132 identifies a point whose distance is next closest on the middle line (S117), and determines whether the y-coordinate of the point is Ymax-h1 or more (S118). If the shelf detection unit 132 determines that the y coordinate is less than Ymax−h1 (S118: No), the process returns to S113 and the process is repeated. On the other hand, when the shelf detection unit 132 determines that the y-coordinate is Ymax-h1 or more (S118: Yes), the processing returns to the inventory detection process.

  FIG. 14 is a flowchart illustrating an example of the stock determination process according to the first embodiment. First, the stock determining unit 134 determines whether the acquired depth is equal to or more than the first threshold (S130). If the stock determination unit 134 determines that the depth is less than the first threshold (S130: No), it determines that the product displayed at the corresponding position is "stock" (S132), and the stock detection process is performed. Return to

  On the other hand, when the stock determining unit 134 determines that the depth is greater than or equal to the first threshold (S130: Yes), the stock determining unit 134 determines whether the depth is greater than or equal to the second threshold (S131). If the stock determination unit 134 determines that the depth is less than the second threshold (S131: No), it determines that the product displayed at the corresponding position is "poor" (S133), and the stock detection process is performed. Return.

  If the stock determining unit 134 determines that the depth is equal to or greater than the second threshold (S131: Yes), the stock displayed at the corresponding position is determined to be "out of stock" (S134). Return.

[effect]
As described above, the inventory detection device in the present embodiment detects the shelf board of the shelf on which the product is displayed, using the information on the distance from the reference point. The inventory detection device calculates the depth from the shelf at the measurement point using the information on the distance from the reference point. The stock detection device uses the depth to determine the stock status of the product at the position corresponding to the measurement point. As a result, the product inventory can be detected accurately.

  Further, the stock detection device in the present embodiment determines that the product is short, for example, when the depth is equal to or greater than the first threshold and less than the second threshold, and indicates that the depth is equal to or greater than the second threshold. , It is determined that the product is out of stock. Furthermore, the inventory detection device may output the result of the process of determination. Thus, the status of the product inventory can be further subdivided and identified.

  Further, the inventory detection apparatus according to the present embodiment determines the first point at which the distance from the reference point is determined to be closest to the center point of the point cloud data acquired as information on the distance from the reference point. Identify. The stock detection device is a second point and a third point moved by predetermined coordinates in the left-right direction of the first point, and the distance from the reference point is the distance from the reference point to the first point Detect points that are determined to be close. Then, when the second point and the third point are respectively detected at both ends in the left-right direction of the point cloud data, the inventory detection device specifies a straight line connecting the second point and the third point as the shelf board Do. Thereby, the shelf board can be identified more accurately.

  Further, when the shelf board is specified or the second point or the third point is not specified, the stock detection device in the present embodiment is next to the reference point at the center in the horizontal direction of the point cloud data. The point determined to be close to the point from may be set as a new first point. Furthermore, the inventory detection apparatus may repeat the process of detecting from the point at the lower end in the vertical direction of the point cloud data until it reaches a point moved from the upper end in the vertical direction of the point cloud data by a predetermined coordinate . Thereby, a plurality of shelf boards can be specified with sufficient accuracy.

  In addition, the inventory detection device in the present embodiment specifies the first measurement point moved upward by a predetermined coordinate from the shelf board, and information on the distance from the reference point to the first measurement point and the shelf board The difference between this and the information on the distance is calculated as the depth. This makes it possible to specify the depth of the product relative to the position of the shelf board.

  By the way, as shown in FIG. 4, there are cases where the distance can not be measured due to the characteristics of the distance measuring sensor, the characteristics of the product at the measurement point, and the like. If the distance measurement data can not be obtained, it is difficult to appropriately detect the stock of goods.

  So, in a present Example, when ranging data can not be acquired, a structure which changes a measurement point and tries acquisition of distance again is demonstrated. FIG. 15 is a diagram illustrating an example of depth calculation processing in the second embodiment. Point cloud data 699 shown in FIG. 15 indicates, for example, distance measurement data of a position where the plastic bottle 690 is displayed. Also, the plastic bottle shown in FIG. 15 includes a part covered by the label and a transparent part not covered by the label.

  As shown in FIG. 15, when the measurement point 691 corresponding to the straight line 611 is a transparent portion not covered with the label of the plastic bottle, the distance can not be correctly acquired for the measurement point. Therefore, in FIG. 15, distance measurement data 613 corresponding to the measurement point 691 in the point cloud data indicates that the distance could not be measured.

  In this case, the detection device 300 in this embodiment, which will be described later, further specifies a straight line 621 at a position moved upward by a predetermined coordinate from the straight line 611. Then, the detection apparatus 300 acquires ranging data at the measurement point 623 of the specific x coordinate on the straight line 621. In this case, since the portion 692 of the plastic bottle 690 corresponding to the measurement point 623 is a portion covered with a label, distance measurement data at the measurement point 623 can be acquired.

  For example, when the distance can not be measured also in the portion 692, the detection device 300 specifies a straight line 631 at a position moved upward by a predetermined coordinate from the straight line 621. Then, the detection apparatus 300 acquires distance measurement data at the measurement point 633 of the specific x coordinate on the straight line 631. In this case, since the portion 693 of the plastic bottle 690 corresponding to the measurement point 633 is a transparent portion not covered by the label, the distance can not be obtained even at the measurement point 633.

  The measurement result of the depth in a present Example is shown in FIG. FIG. 16 is a diagram of an example of distance measurement data corresponding to the stock status in the second embodiment. In the following embodiments, the same parts as the parts shown in the above-described drawings are denoted by the same reference numerals, and redundant description will be omitted.

  In FIG. 16, for example, “●” marks 8021, 8041 and 8051 indicate that the distance is acquired at the measurement point on the straight line 611. A "*" mark 8011 indicates that the distance is acquired at the measurement point on the straight line 621 at a position moved upward by a predetermined coordinate from the straight line 611. Also, the “x” mark 8031 indicates that the distance could not be obtained at any of the straight measurement points.

[Function block]
Next, with regard to the function of the detection device 300 in the present embodiment, FIG. 17 is a diagram showing an example of the detection device in the second embodiment. As illustrated in FIG. 17, the detection device 300 in the present embodiment includes an external I / F 110, a storage unit 320, and a control unit 330. In the following embodiments, the same parts as the parts shown in the above-described drawings are denoted by the same reference numerals, and redundant description will be omitted.

  The storage unit 320 stores, for example, various data such as a program executed by the control unit 330. In addition, the storage unit 320 includes a distance measurement data DB 321, a depth DB 322, and a shelf allocation DB 123. The storage unit 320 corresponds to a semiconductor memory device such as a RAM, a ROM, or a flash memory, or a storage device such as an HDD.

  Similar to the data shown in FIG. 6, the distance measurement data DB 321 stores distance measurement data. In addition, distance measurement data DB 321 further stores information (for example, “-” symbol) indicating that distance measurement data is not acquired when distance measurement data is not acquired by distance acquisition unit 331 described later. Do. FIG. 18 is a view showing an example of distance measurement data DB in the second embodiment. As shown in FIG. 18, the distance measurement data DB 321 stores information indicating that distance measurement data was not obtained at the measurement point at the x coordinate “200” and y coordinate “0” of the bin number “1”. .

  The depth DB 322 stores the depth calculated from the distance measurement data as shown in FIG. 7 in association with the position on the shelf and the height at which the distance is measured. FIG. 19 is a diagram illustrating an example of the depth DB in the second embodiment. As illustrated in FIG. 19, the depth DB 322 stores “measurement height” in association with the “shelf number”, the “shelf”, the “x coordinate”, and the “depth” in addition to the “measurement height”.

  In FIG. 19, “measurement height” stores the height in the Y-axis direction of the measurement point at which the distance has been measured. In the example illustrated in FIG. 19, for example, the measurement height “1” indicates that the distance is acquired at the measurement point on the straight line 611 illustrated in FIG. 15, and the measurement height “2” indicates the measurement on the straight line 621 Indicates that a distance was obtained at a point. In addition, "-" shows that distance was not able to be acquired also in any measurement point. In the example shown in FIG. 19, in the depth DB 322, at the x-coordinate "80" of the shelf number "1" and the tray "1", the distance is acquired at the measurement point on the straight line 611, and the x-coordinate "160" and "x" In 240 ", it memorize | stores that the distance was acquired at the measurement point on the straight line 621.

  The control unit 330 is a processing unit that controls the entire processing of the detection device 300. The control unit 330 is realized by, for example, a program stored in an internal storage device being executed by a CPU, an MPU, or the like, using a RAM as a work area. Also, the control unit 330 may be realized by an integrated circuit such as an ASIC or an FPGA, for example.

  The control unit 330 includes a distance acquisition unit 331, a shelf detection unit 132, a depth calculation unit 333, an inventory determination unit 334, and an output unit 135. The distance acquisition unit 331, the shelf board detection unit 132, the depth calculation unit 333, the stock determination unit 334, and the output unit 135 are an example of an electronic circuit included in the processor and an example of a process executed by the processor.

  When the distance measurement data can not be acquired, the distance acquisition unit 331 stores information indicating that the distance measurement data is not acquired in the distance measurement data DB 321 in association with the target coordinates.

  The depth calculation unit 333 calculates the depth at the measurement point using the distance measurement data, and also stores the measurement height at which the used distance measurement data is acquired when storing the depth in the depth DB 322. When the distance measurement data at the measurement point can not be obtained, the depth calculation unit 333 tries to calculate the depth again using the distance measurement data at different measurement points.

  For example, when the distance measurement data can not be acquired at the measurement point on the straight line 611 shown in FIG. 15, the depth calculation unit 333 acquires the distance measurement data at the measurement point on the straight line 621 at the position moved upward by the predetermined coordinates. . For example, when the distance measurement data can be acquired at the measurement point on the straight line 621, the depth calculation unit 333 calculates the depth using the distance measurement data, and associates the depth with the measurement height "2" corresponding to the measurement point. It memorize | stores in depth DB322.

  In addition, when distance measurement data can not be acquired at any of the candidate measurement points, the depth calculation unit 333 stores information indicating that the depth can not be identified in the depth DB 322. For example, when the depth calculation unit 333 can not acquire distance measurement data even at the measurement point on the straight line 631 illustrated in FIG. 15, information indicating that “depth” and “measurement height” can not be acquired in the depth DB 322 "-" Is stored.

[Flow of processing]
Next, processing in the present embodiment will be described using FIGS. 20 and 21. FIG. In the following description, the same reference numerals as in the steps shown in FIG. 12 to FIG. 14 denote the same steps, so detailed description will be omitted.

  FIG. 20 is a flowchart of an example of the stock detection process according to the second embodiment. As shown in FIG. 20, in the stock detection process in the second embodiment, a depth identification process is executed instead of S12 and S13 shown in FIG. 12 (S22).

  FIG. 21 is a flowchart of an example of the depth identification process according to the second embodiment. First, the depth calculation unit 333 specifies a straight line at a position moved upward by a predetermined coordinate from the detected shelf board (S221). Next, the depth calculation unit 333 refers to the distance measurement data DB 321, and determines whether or not the distance can be acquired at the measurement point on the specified straight line (S222).

  When it is determined that the distance can be acquired (S222: Yes), the depth calculation unit 333 calculates the depth using the acquired distance measurement data (S223). Then, the depth calculation unit 333 performs the stock determination process shown in FIG. 14 using the calculated depth (S13), and returns to the stock detection process.

  On the other hand, when it is determined that the distance can not be acquired (S222: No), the depth calculation unit 333 specifies a straight line at a position moved upward by a predetermined coordinate from the specified measurement point (S224). Then, the depth calculation unit 333 determines whether the y coordinate of the specified straight line is equal to or greater than a predetermined threshold (S225). When the depth calculation unit 333 determines that the y coordinate is less than the predetermined threshold (S225: No), the process returns to S222 and repeats the processing.

  On the other hand, when the depth calculation unit 333 determines that the y coordinate is equal to or more than the predetermined threshold (S225: Yes), it determines that "depth information is not present" and stores the determination result in the depth DB 322 (S226) Return to the inventory detection process.

[effect]
As described above, the inventory detection device according to the present embodiment can check the shelf allocation data including the number of displayed commodities displayed on the shelf, the inventory status, and the information on the size of the commodity. Identify the type and number of items in stock. This makes it possible to identify the stock status in line with the shelf allocation.

  In addition, when the inventory detection device in the present embodiment can not specify the depth at the first measurement point, the inventory detection device may specify the second measurement point moved by a predetermined coordinate further upward from the first measurement point. . Then, the inventory detection device may calculate, as the depth, the difference between the information on the distance from the reference point to the second measurement point and the information on the distance to the shelf board. As a result, for example, even when distance measurement data can not be acquired at a specific position, the possibility of detecting inventory can be improved.

  Although the embodiments of the present invention have been described above, the present invention may be implemented in various different modes other than the above-described embodiments. For example, although the detection apparatus 100 in the first embodiment is mounted by a self-propelled robot as shown in FIG. 1, the present invention is not limited to this, and may be realized by a computer such as a server that acquires an image captured by the robot. You can also. Further, the server may acquire an image captured by a store clerk or the like (not shown) instead of the image captured by the robot or the like.

  Further, in FIG. 7, although the depth calculation unit 133 stores, for example, the calculation result in the case of moving “80” in the right direction after calculating the depth, the moving coordinates are an example, for example, the width of the shelf or You may change suitably according to the width | variety of goods.

  In FIG. 9C, although the configuration is described in which the processing is ended at point Ymax-h1 which is moved downward from the upper end Ymax in the Y-axis direction by the predetermined coordinate h1, the embodiment is not limited thereto. For example, when a product is suspended from the upper end of the shelf board, the product may be displayed on the upper end of the shelf. Therefore, the shelf board detection process may be repeated until the upper end Ymax in the Y axis direction.

  In the shelf detection process shown in FIG. 13, when the second and third points are not detected, the position of the shelf may be erroneously detected because, for example, the product protrudes before the shelf. There is sex. Therefore, when the second point and the third point are not detected, the shelf board detection unit 132 may output information indicating a display abnormality of the product. Moreover, in the stock determination process shown in FIG. 14, the same applies to the case where it is determined that the depth is a negative value.

[system]
Further, each component of each unit shown in the drawings does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the dispersion and integration of each part is not limited to the illustrated one, and all or a part thereof is functionally or physically dispersed or integrated in any unit according to various loads, usage conditions, etc. Can be configured. For example, the depth calculation unit 133 and the stock determination unit 134 may be integrated. Further, the process of collating the stock status of the output unit 135 with the shelving information and the process of outputting the collated result may be divided and provided as separate processing units.

  Furthermore, all or any part of various processing functions performed by each device may be executed on a CPU (or a microcomputer such as an MPU or an MCU (Micro Controller Unit)). In addition, various processing functions may be executed in whole or any part on a program analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware by wired logic. It goes without saying that it is good.

  The various processes described in the above embodiments can be realized by executing a prepared program on a computer. So, below, an example of a computer which runs a program which has the same function as the above-mentioned example is explained. FIG. 22 is a diagram of an example of a computer that executes a detection program. In addition, although the detection apparatus 100 in Example 1 is demonstrated to an example below, it can implement | achieve also about the detection apparatus 300 in Example 3 by the same computer.

  As illustrated in FIG. 22, the computer 200 includes a CPU 201 that executes various types of arithmetic processing, an input device 202 that receives data input, and a monitor 203. The computer 200 also includes a medium reading device 204 reading programs and the like from a storage medium, an interface device 205 for connecting with various devices, and a communication device 206 for connecting with other information processing devices and the like by wire or wirelessly. Have. The computer 200 also has a RAM 207 for temporarily storing various information, and a hard disk drive 208. Each of the devices 201 to 208 is connected to the bus 209.

  The hard disk drive 208 stores a detection program having the same function as each processing unit of the distance acquisition unit 131, the shelf board detection unit 132, the depth calculation unit 133, the stock determination unit 134, and the output unit 135 shown in FIG. Ru. Further, the hard disk drive 208 stores a distance measurement data DB 121, a depth DB 122, a shelf allocation DB 123, and various data for realizing a detection program. The input device 202 receives, for example, an input of various information such as management information from the administrator of the computer 200. The monitor 203 displays, for example, a screen of management information and various screens for the administrator of the computer 200. The interface device 205 has, for example, the same function as the external I / F 110 shown in FIG. 5, and is connected to an external device having a sensor such as a camera or a distance measurement sensor, for example, to exchange various information. The communication device 206 has, for example, the same function as the external I / F 110 illustrated in FIG. 5 and is connected to, for example, an external database to exchange various information.

  The CPU 201 reads out each program stored in the hard disk device 208, develops the program in the RAM 207, and executes the program to perform various processes. In addition, these programs can cause the computer 200 to function as the distance acquisition unit 131, the shelf detection unit 132, the depth calculation unit 133, the stock determination unit 134, and the output unit 135 illustrated in FIG.

  The above detection program does not necessarily have to be stored in the hard disk drive 208. For example, the computer 200 may read out and execute a program stored in a storage medium readable by the computer 200. The storage medium readable by the computer 200 corresponds to, for example, a CD-ROM, a DVD disk, a portable recording medium such as a USB (Universal Serial Bus) memory, a semiconductor memory such as a flash memory, a hard disk drive, or the like. Alternatively, the detection program may be stored in a device connected to a public line, the Internet, a LAN or the like, and the computer 200 may read out the detection program from these and execute it.

100, 300 Detector 110 External I / F
120, 320 storage unit 121, 321 distance measurement data DB
122, 322 depth DB
123 Shelf Allocation DB
130, 330 Control unit 131, 331 Distance acquisition unit 132 Shelf detection unit 133, 333 Depth calculation unit 134, 334 Inventory determination unit 135 Output unit

Claims (10)

Use the information on the distance from the reference point to detect the shelf shelf on which the product is displayed,
Using the information on the distance from the reference point, calculate the depth from the shelf at the measurement point;
The stock detection program characterized by making a computer perform processing which determines a stock situation of goods in a position corresponding to the measurement point using the depth. Further causing the computer to execute a process of outputting the result of the process of determination;
In the determination process, when the depth is equal to or more than a first threshold and less than a second threshold, it is determined that the item is insufficient, and the item is equal to or more than a second threshold. The stock detection program according to claim 1, wherein the stock detection program is determined to be out of stock.   A process of identifying the type of the product and the stock quantity of the product by collating shelf allocation data including the number of displayed products displayed on the shelf, the stock status, and information on the size of the product The stock detection program according to claim 1 or 2, further causing the computer to execute.   In the processing for detecting, a first point determined to be closest to the reference point is specified at a central portion in the horizontal direction of point cloud data acquired as information on the distance from the reference point, The second and third points moved by predetermined coordinates in the left-right direction of the first point, wherein the distance from the reference point is close to the distance from the reference point to the first point When the second point and the third point are respectively detected at both ends in the left-right direction of the point group data, the second point and the third point are detected. The stock detection program according to any one of claims 1 to 3, wherein a straight line connecting is specified as the shelf board.   When the shelf board is identified, or when the second point or the third point is not identified, the detection process is performed next to the reference point at the center in the horizontal direction of the point cloud data. 5. The stock detection program according to claim 4, wherein a point determined to be close to a point from is newly set as the first point.   Starting from the point at the lower end in the vertical direction of the point cloud data, the process of detecting is repeatedly performed by the computer until reaching a point moved by a predetermined coordinate from the upper end in the vertical direction of the point cloud data. The stock detection program according to claim 5, characterized in that:   The process of calculating specifies a first measurement point moved upward by a predetermined coordinate from the shelf board, information on a distance from the reference point to the first measurement point, and the shelf board The stock detection program according to any one of claims 1 to 6, wherein a difference from information on a distance is calculated as the depth.   When the depth can not be specified at the first measurement point, the calculation process specifies a second measurement point moved by a predetermined coordinate further upward from the first measurement point, and from the reference point The stock detection program according to claim 7, wherein a difference between information on a distance to the second measurement point and information on a distance to the shelf is calculated as the depth. The computer is
Use the information on the distance from the reference point to detect the shelf shelf on which the product is displayed,
Using the information on the distance from the reference point, calculate the depth from the shelf at the measurement point;
A stock detection method characterized by performing processing which judges a stock situation of goods in a position corresponding to the measurement point using the depth. A detection unit that detects a shelf board of a shelf on which a product is displayed using information on a distance from a reference point;
A calculation unit that calculates the depth from the shelf board at the measurement point using information on the distance from the reference point;
A determination unit that determines an inventory status of a product at a position corresponding to the measurement point using the depth.