TWI774088B - Food content identifying method and food content identifying system - Google Patents

Abstract

A food identifying system is adapted to perform a food identifying method. The method includes obtaining plural distance values on a detecting area to generate a distance distribution, and capturing an image to be analyzed including the detecting area; mapping the distance distribution to the image to be analyzed; setting a distance threshold and defining a part, in the image to be analyzed, in which each of the distance values is smaller than the distance threshold as a section to be identified; performing image identification to the section to be identified to identify a food in the section; and obtaining the volume of the food according to distance distribution.

Description

Catering Content Recognition Method and Catering Content Recognition System

The invention relates to automatic identification of catering content, in particular to a catering content identification method and a catering content identification system.

In the occasions where it is necessary to quickly record the content of catering, such as the pricing of self-service catering, and the management of personal food content, it is often necessary to manually identify and record the type and weight of food, so as to further aggregate other related information, such as catering prices, catering Nutritional composition.

The prior art proposes several technical solutions for automatically recognizing food and taking its weight. One of the various technical solutions is to store different foods in corresponding containers. Machine-readable labels, such as barcodes and RFID, are installed on the containers, so that the reading machine can quickly interpret them. The types of food can be read through the labels and weighed by electronic scales and other devices. measurement to distinguish food types and food weights. This method requires relatively complicated equipment, and at the same time, the food must be placed in the correct container in advance, which requires a relatively complicated pre-operation process.

Another technical solution is to use image recognition to distinguish food and interpret food volume, and then use food volume and food information to convert food weight. Image recognition needs to exclude parts that do not belong to food in the complete image to be analyzed, and requires high computing resources for image processing and image recognition, and the processing performance is limited by hardware performance. And interpreting food volume also relies on images Recognition, analyzing food volume through image recognition brings additional computational load. At the same time, in order to avoid misinterpretation, the three-dimensional shape of the food is also limited, resulting in the need for additional pre-processing procedures for food processing.

The technical solution of automatically recognizing food and taking weight in the prior art has the problems of setting up relatively complex equipment, requiring high computing resources, or adding additional pre-processing procedures for additional food processing.

Based on the above technical problems, the present invention proposes a method for recognizing food and beverage content, comprising: obtaining a plurality of distance values on a detection area to generate a distance distribution, and capturing an image to be analyzed including the detection area; mapping to the image to be analyzed; setting a threshold distance to define a portion of the image to be analyzed whose distance value is smaller than the threshold distance as at least one block to be identified; performing image recognition on at least one block to be identified, and identifying the part located in the image to be identified A type of food in the block; and according to the distance distribution, obtain the volume of the food.

In at least one embodiment, before obtaining a plurality of distance values, the image to be analyzed is continuously captured, and it is analyzed whether the image to be analyzed includes a detection area.

In at least one embodiment, capturing the image to be analyzed includes capturing the image to be analyzed with an image capturing device, and there is an actual reference distance value between the image capturing device and a surface of the detection area, and the threshold distance is less than or Equal to the actual base distance value.

In at least one embodiment, the number of pixels occupied by the length and width of the detection area in the image to be analyzed are respectively Xn×Yn, and there is a predetermined reference distance value between the image capturing device and the surface of the detection area, an actual length of the detection area under the preset reference distance value, and The number of pixels occupied by an actual width in the image to be analyzed is L×W respectively, and the relationship between the actual reference distance value and the preset reference distance value is: z0'=z0×(L/Xn); or z0'= z0×(W/Yn); wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value.

In at least one embodiment, the method for identifying food and beverage content further includes determining whether each distance value is within a range between an upper distance value and a lower distance value; if any distance value exceeds the upper distance value and the distance The range interval of the lower limit value, re-acquiring multiple distance values and re-acquiring the image to be analyzed.

In at least one embodiment, the method for identifying food and beverage content further includes identifying whether at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and at least one standard container information.

In at least one embodiment, the step of identifying the type of food located in the block to be identified includes: loading at least one piece of food information; wherein the at least one piece of food information at least includes the type of food and a sample image; Whether the image and the sample image are similar, so that the type of food can be identified in the block to be identified.

In at least one embodiment, the food content identification method further includes: loading at least one food information; wherein the at least one food information further includes food density of the food; and multiplying the food density by the volume of the food to obtain the weight of the food.

In at least one embodiment, the method for identifying food and beverage content further includes: loading at least one piece of food information; wherein, the at least one food information further includes nutritional components, price per unit weight; and multiplying the nutritional components by the weight of the food to obtain the food A nutritional composition of ; price per unit weight The grid is multiplied by the weight of the food to obtain the price of the food; and the type, weight, nutritional composition and price of the food are associated with a diner and uploaded to a management platform.

In at least one embodiment, the method for recognizing food and beverage content further includes analyzing whether at least one block to be identified conforms to a correctly placed standard container; when at least one block to be identified does not conform to a correctly placed standard container, an error message is generated .

The present invention also provides a food and beverage content recognition system, which includes a three-dimensional scanner, an image capture device and a data processing module.

The three-dimensional scanner is used to obtain a plurality of distance values on a detection area to generate a distance distribution. The image capturing device is used for capturing an image to be analyzed in the detection area. The data processing module is electrically connected to the 3D scanner and the image capture device; wherein, the data processing module maps the distance distribution to the image to be analyzed; the data processing module sets a threshold distance to be defined in the image to be analyzed The portion of each distance value smaller than the threshold distance is used as at least one block to be identified; and the data processing module performs image recognition on the at least one block to be identified, and identifies a type of food located in the at least one block to be identified, and the distance distribution to obtain the volume of the food.

In at least one embodiment, before obtaining a plurality of distance values, the image capturing device continuously captures the image to be analyzed, and the data processing module analyzes whether the image to be analyzed includes the detection area.

In at least one embodiment, there is an actual reference distance between the image capturing device and a surface of the detection area, and the threshold distance is less than or equal to the actual reference distance.

In at least one embodiment, the number of pixels occupied by the length and width of the detection area in the image to be analyzed are respectively Xn×Yn, and there is a predetermined reference distance value between the image capturing device and the surface of the detection area, an actual length of the detection area under the preset reference distance value, and The number of pixels occupied by an actual width in the image to be analyzed is L×W respectively, and the relationship between the actual reference distance value and the preset reference distance value is: z0'=z0×(L/Xn); or z0'= z0×(W/Yn); wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value.

In at least one embodiment, the data processing module determines whether each distance value is within a range interval between a distance upper limit value and a distance lower limit value; if any distance value exceeds the distance upper limit value and the distance lower limit The range interval of the value is obtained by using the 3D scanner and the image capturing device to re-acquire a plurality of distance values and re-capture the image to be analyzed.

In at least one embodiment, the data processing module loads at least one standard container information, and identifies whether at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and the at least one standard container information.

In at least one embodiment, the data processing module loads the type of food and a sample image, and compares the image of the block to be identified and the sample image to determine whether the image of the block to be identified is similar, so as to identify the type of food in the block to be identified.

In at least one embodiment, the data processing module loads the food density of the food, and multiplies the food density by the volume of the food to obtain the weight of the food.

In at least one embodiment, the data processing module loads the nutrients and the price per unit weight, multiplies the nutrients by the weight of the food to obtain a nutritional composition of the food, and multiplies the price per unit weight by the weight of the food to obtain the weight of the food. price; and the data processing module relates the type, weight, nutritional composition and price of food to a diner, and uploads it to a management platform.

In at least one embodiment, the data processing module analyzes whether the at least one block to be identified conforms to a correctly placed standard container; when the at least one block to be identified does not conform to the correctly placed standard container, an error message is generated.

Through the combination of 3D scanning and image capture, the present invention marks the area occupied by the standard container in the detection area in the image to be analyzed based on the corresponding relationship between the captured image and the 3D scanning information, while only the standard container is detected. Image recognition is performed on the corresponding block to be recognized, instead of performing image recognition on the entire image to be analyzed, so as to determine the type of food. Therefore, the present invention can reduce the computing load required for image recognition, and can speed up the food recognition process. At the same time, through the application of standard containers, the distance distribution obtained by 3D scanning can be used to effectively and relatively accurately calculate the volume of food. With the correlation of food information, the information of the complete meal can be quickly analyzed, so as to combine the diners. Identify information for back-end management.

100: Catering Content Recognition System

110: 3D Scanner

120: Image capture device

130:Data processing module

140: Bracket

150: Container Repository

160: Food Database

200: Food

301, 302, 303: Standard Containers

304: Non-standard container

400: Management Platform

A,A': detection area

B: block to be identified

C: Image to be analyzed

D: separation distance

L: Actual length of detection area A

Xn: length of detection area A'

W: Actual width of detection area A

Yn: Width of detection area A'

P: distance distribution

S: Placement table

X: length direction

Y: width direction

Z: height direction

△a: area of grid area

h: grid height

z(x,y): distance value

zmax: threshold distance

z0: preset reference distance value

z0’: Actual reference distance value

Step 112~Step 170: Steps

FIG. 1 is a schematic side view of a food and beverage content recognition system according to an embodiment of the present invention.

FIG. 2 is a schematic top view of a food and beverage content recognition system according to an embodiment of the present invention.

3 is a system block diagram of a food and beverage content identification system according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of mapping distances to images to be analyzed according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of correcting a plurality of distance values in an embodiment of the present invention.

FIG. 6 is a flowchart (1) of a method for recognizing food and beverage content according to an embodiment of the present invention.

7 is a schematic diagram of defining at least one block to be identified according to a distance threshold in the method for identifying food and beverage content according to an embodiment of the present invention.

FIG. 8 is an information comparison table in a food database in the method for identifying food and beverage content according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of obtaining the volume of food according to the distance distribution in the method for recognizing food and beverage content according to an embodiment of the present invention.

FIG. 10 is a flowchart (2) of a method for recognizing food and beverage content according to an embodiment of the present invention.

FIG. 11 is a flowchart (3) of a method for recognizing food and beverage content according to an embodiment of the present invention.

FIG. 12 is a flowchart (4) of a method for recognizing food and beverage content according to an embodiment of the present invention.

FIG. 13 is a schematic diagram of a recognition detection area in the method for recognizing food and beverage content according to an embodiment of the present invention.

14 is a schematic diagram of defining at least one block to be identified in the method for identifying food and beverage content according to an embodiment of the present invention.

15 is a schematic diagram of analyzing whether at least one block to be identified conforms to a correctly placed standard container in the method for identifying food and beverage content according to an embodiment of the present invention.

The term "module" used in the following description may refer to an application specific integrated circuit (ASIC), electronic circuit, microprocessor, chip or circuit design that executes one or more software or firmware programs. The modules are configured to perform various algorithms, transformations and/or logical processing to generate one or more signals. When the module is implemented in software, the module can be implemented in a memory as a code readable by a chip or circuit design through code execution.

As shown in FIG. 1 , FIG. 2 , and FIG. 3 , a food and beverage content recognition system 100 disclosed in an embodiment of the present invention is used to execute a food and beverage content recognition method, and to recognize at least one food 200 located in a detection area A. type. The food content recognition system 100 includes a 3D scanner 110 , an image capture device 120 and a data processing module 130 .

As shown in FIG. 1 , FIG. 2 and FIG. 3 , the 3D scanner 110 and the image capture device 120 are disposed on a bracket 140 , and the 3D scanner 110 and the image capture device 120 face the placing table S to detect The area A is located on the placement table S.

As shown in FIG. 1 , FIG. 2 , FIG. 3 and FIG. 4 , the three-dimensional scanner 110 is configured to perform three-dimensional scanning on the detection area A, and obtain multiple distance values z of multiple detection points on the detection area A (x,y) to produce a distance distribution. The distance distribution, that is, the distance values z(x, y) and the position corresponding to the detection area A, form a point cloud, which can show the fluctuation state of the detection area A, so as to be judged by the data processing module 130 Detects whether there is a decoration object in the area A, and can be used to analyze the three-dimensional shape of the object. The 3D scanner 110 can be, but is not limited to, a Stereo Camera, a Time of Flight (TOF) camera, a LiDAR, or other optical and acoustic ranging devices, as long as it can detect multiple depth value.

As shown in FIG. 1 , FIG. 2 , FIG. 3 and FIG. 4 , the image capturing device 120 is used for capturing an image C to be analyzed including the detection area A. As shown in FIG. In a specific embodiment, the image capture device 120 is an RGB color camera, and can capture a color image C to be analyzed for performing image analysis. The image capturing device 120 is not limited to a color camera, and a monochrome camera is not excluded, so that the image C to be analyzed is a monochrome (black and white) image.

As shown in FIG. 1 , FIG. 2 , FIG. 3 and FIG. 4 , the data processing module 130 is electrically connected to the 3D scanner 110 and the image capturing device 120 for receiving the distance distribution and the image C to be analyzed. The 3D scanner 110 and the image capturing device 120 mounted on the bracket 140 are located at approximately the same height (relative to the detection area A), but are separated by a distance D in the horizontal direction.

As shown in FIG. 4 , the data processing module 130 can shift (Shift) the distance distribution and the plane coordinate relationship of the image C to be analyzed according to the separation distance D, and map the distance distribution P to the image C to be analyzed.

As shown in FIG. 5 , in addition, the data processing module 130 also uses the pixel information of the detection area A in the image C to be analyzed and the actual size of the detection area A to correct the reference plane of the distance value z(x, y), and also That is, the actual reference distance value z0' from the image capturing device 120 to the surface of the detection area A is obtained.

Generally speaking, the detection area A is usually rectangular and has a fixed length and width. For example, in the embodiment of the present invention, the surface of the dinner plate is used as the detection area A. Standardized plates have fixed lengths and widths.

As shown in FIG. 5 , in a standard operating state, the dinner plate is stably placed on a platform so that there is a predetermined reference distance z0 between the surface of the detection area A and the image capturing device 120 . At this time, in the length direction X and the width direction Y, the actual length and actual width of the detection area A occupy the number of pixels in the image C to be analyzed, which are L×W respectively.

When actually operating the food and beverage content recognition system 100 of the present invention, the user may not place the food plate completely flat on the platform, but may drag the food plate to a certain height by hand. At this time, the distance between the plane of the detection area A and the image capture device 120 will be reduced, so that the preset reference distance value z0 is not applicable, resulting in an incorrect setting of the subsequent threshold distance zmax, which affects subsequent judgment and calculation operations. .

As shown in FIG. 5 , when the dinner plate is raised to a higher height, the distance between the plane serving as the detection area A' and the image capturing device 120 is reduced to the actual reference distance value z0'. At this time, in the image C to be analyzed captured by the data processing module 130, the detection area A' will be Zoom in, so that the length and width of the detection area A' occupy the number of pixels in the image C to be analyzed to be changed to Xn × Yn respectively.

In this case, the preset reference distance value z0 originally stored in the data processing module 130 is not applicable, and the actual reference distance value z0' needs to be obtained. At this time, the relationship between the actual reference distance value z0' and the preset reference distance value z0 is represented by the ratio of the number of available pixels as follows: z0'=z0×(L/Xn); or z0'=z0×(W/ Yn).

As shown in FIG. 2 and FIG. 3 , different foods 200 are filled in different standard containers 301 , 302 , and 303 respectively, and the openings of the standard containers 301 , 302 , and 303 face the top of the detection area A, so that the three-dimensional scanner 110 and the image capture device 120 can detect The condition inside the standard containers 301, 302, 303 is measured. In addition, each standard container 301, 302, 303 has a side wall surrounding the inner space, so that the standard container 301, 302, 303 forms an annular area in the detection area A higher than the detection area A. Specifically, the data processing module 130 can be built in or connected to a container database 150, and the container database 150 stores the information of the standard containers 301, 302, 303, such as the area and shape of the opening, the thickness of the side wall, and the cross-sectional area of the accommodating space size. In ideal standard containers 301 , 302 , and 303 , the cross-sectional area of the accommodating space is preferably kept consistent in the height direction Z, such as a column shape, which is beneficial for subsequent analysis and calculation. However, it does not rule out that the accommodating space presents an inverted cone shape, that is, the cross-sectional area from the opening to the bottom is tapered. Table or formula operation can be obtained.

Generally speaking, for example, in the case of self-service, the diners place the food 200 to be taken together with its standard containers 301, 302, 303 on a plate. Therefore, the detection area A can be set to be located on the surface of a plate. Through the differentiated setting of colors, the surface of the plate (ie, the detection area A) can be contrasted with the environment, which is beneficial to mark the detection area A. At the same time, the standardized plate can also make the detection area A have a fixed length (L) and width (W).

The following describes the catering content identification method disclosed in the embodiments of the present invention.

As shown in FIG. 1 , FIG. 2 , FIG. 4 and FIG. 6 , first, the data processing module 130 starts and controls the 3D scanner 110 and the image capturing device 120 to work. According to the control of the data processing module 130, the three-dimensional scanner 110 performs three-dimensional scanning on the detection area A, and obtains a plurality of distance values z(x, y) on the detection area A, so as to generate the distance distribution P, as shown in the step shown in Step 112. The image capturing device 120 captures an image C to be analyzed including the detection area A, as shown in Step 114 . The aforementioned steps Step 112 and Step 114 may be performed simultaneously, or may be performed sequentially.

As shown in FIG. 4 and FIG. 6 , next, the data processing module 130 maps the distance distribution P to the image C to be analyzed, as shown in Step 120 . This mapping step is used to make each distance value z(x,y) have one or a group of corresponding pixels on the image C to be analyzed, that is, each distance value z(x,y) is in the image C to be analyzed. The analysis image C has a corresponding coordinate value (x, y).

As shown in FIG. 6 and FIG. 7 , a threshold distance zmax is set in the data processing module 130 . According to the threshold distance zmax, the data processing module 130 defines the part of each distance value z(x, y) smaller than the threshold distance zmax in the image C to be analyzed as one or more blocks B to be identified, as shown in Step 130 . Show.

As shown in Fig. 7, the threshold distance zmax can be set to fall on the surface of the detection area A, that is, the actual reference distance value z0' is set as the threshold distance zmax, or set slightly A value smaller than the actual reference distance value z0' is used as the threshold distance zmax. Each standard container 301, 302, 303 has a side wall surrounding its inner space, so that the standard container 301, 302, 303 forms an annular area in the detection area A higher than the surface of the detection area A; even the bottom of the inner space in the empty standard container 301, 302, 303, It is also higher than the surface of the detection area A because of the physical thickness of the standard containers 301, 302, 303; therefore, as long as the height is higher than zero, that is, the pixels whose corresponding distance value z(x,y) is less than the threshold distance zmax can be regarded as standard containers Areas covered by 301, 302, 303. Therefore, the data processing module 130 can separate the portion occupied by the relevant pixels in the image C to be analyzed as the block B to be identified, instead of analyzing the entire image C to be analyzed.

As shown in FIG. 6 and FIG. 7 , the data processing module 130 then performs image recognition on the to-be-identified block B to identify the type of a food 200 located in the to-be-identified block B, as shown in Step 140 .

As shown in FIG. 6 and FIG. 7 , in Step 130 , the data processing module 130 has segmented the to-be-identified block B where food and beverages may be placed, and excludes the pixel areas not related to the food 200 and the standard containers 301 , 302 , and 303 . Therefore, in step Step 140, the data processing module 130 only needs to perform image recognition on the block B to be identified, but does not need to perform image recognition on the entire image C to be analyzed, and the data processing module 130 needs to consume less computing resources. The efficiency of image recognition can be accelerated, and the probability of recognition errors can be effectively reduced.

As shown in FIG. 3 and FIG. 8 , the data processing module 130 has a built-in or externally connected food database 160 . The food database 160 stores multiple pieces of food information. The food information includes information such as the type of food 200 , sample images, food density, nutritional content, price per unit weight (eg, RMB/100 grams). The data processing module 130 can identify the type of the food 200 in the to-be-identified block B by comparing the image of the to-be-identified block B and the sample image.

As shown in FIG. 6 , according to the distance distribution P in the block B to be identified, the data processing module 130 obtains the volume of the food 200 , as shown in Step 150 .

As shown in FIG. 9 , a specific embodiment of obtaining the volume of the food 200 is described as follows. The block B to be identified can be divided into a plurality of grid regions, and each grid region corresponds to a distance value z(x, y). According to the standard container information and the actual reference distance value z0' of the surface of the detection area A (that is, the surface of the plate), the data processing module 130 can convert the distance of the food relative to the bottom of the internal space of the standard container 301, 302, 303 in each grid area. grid height h. The grid height h multiplied by the grid area area Δa is the volume of the local food 200 in the grid area.

Finally, the data processing module 130 adds up the volume of the food 200 in all the grid areas, which is the volume of the food 200 in the standard containers 301 , 302 , and 303 , as shown by the oblique lines in the figure. As mentioned above, the cross-sectional area of the accommodating space is preferably kept consistent in the height direction Z, such as a columnar shape, which is beneficial for subsequent analysis and calculation. However, it is not ruled out that the accommodating space presents an inverted cone shape; at this time, according to the standard container information, the grid height h can be corrected according to the slope condition. As shown in Figure 9, the grid height h at the edge is deducted from the average height of the slope. The standard container information can include the standard containers 301 , 302 , 303 and the three-dimensional shape of the accommodating space. Therefore, the grid height h can be corrected according to the three-dimensional shape, so as to obtain the correct volume of the food 200 . The obtaining of the actual reference distance value z0' described in the aforementioned FIG. 5 and the corresponding description is to avoid wrongly adopting the preset reference distance value z0, resulting in an erroneous increase of the grid height h and an error in the calculation of the volume of the food 200.

As shown in FIG. 6 and FIG. 8 , finally, the weight of the food 200 is calculated according to the food information loaded in the food database 160 and the volume of the food 200 , as shown in Step 160 . As mentioned above, the food information also includes information such as food density, nutritional composition, price per unit weight, etc. Therefore, the data processing module 130 multiplies the food density by the volume of the food 200 to obtain the food 200 the weight of. In addition, the nutritional composition of the food 200 can be obtained by multiplying the nutritional components in the composition by the weight, respectively. Likewise, multiply the price per unit weight by the weight to get the price of the food 200.

The aforementioned meal information, including the type, weight, nutritional composition, and price of the food 200 , can be associated with a specific diner. The data processing module 130 associates the diner identification information and the catering content, and uploads it to a management platform 400 for catering management or catering expense management of the diners, as shown in Step 170 . The identification method of the specific diners may be biometric identification, reading RFID, inputting account passwords, etc. The identification of the diners may be performed before the execution of the method for identifying the meal content, and may be performed after Step 160 .

The above-mentioned method for recognizing food and beverage content is performed under ideal conditions. In practical applications, different problems may be encountered that need to be solved, and the following further describes the variation of the foregoing method for recognizing food and beverage content.

As shown in FIG. 10 , FIG. 11 , and FIG. 12 , first, the start of the method for recognizing food and beverage content may be started manually or automatically. In the case of automatic activation, the data processing module 130 activates and controls the 3D scanner 110 and the image capturing device 120 to work, and continuously detects the placement table S. The image capturing device 120 continuously captures the image C to be analyzed, and the data processing module 130 continues to analyze whether the image C to be analyzed includes the detection area A, as shown in Step 116 . When it is detected that the image C to be analyzed includes the detection area A, the data processing module 130 executes the mapping of the distance distribution P to the image C to be analyzed, as shown in step Step 120 , and proceeds to step 130 .

As shown in FIG. 13 , the detection area A is the surface of the dinner plate, and has a fixed shape and a specific color in a plan view. Therefore, the data processing module 130 can determine whether the detection area A (ie the dinner plate) is present on the placing table S according to the shape and color, and automatically start the subsequent identification know the program.

As shown in FIG. 5 and FIG. 10 , then, the data processing module 130 obtains the actual reference distance value z0 ′ by analyzing the pixel information in the image of the detection area A and the actual size of the detection area A, as in Step 122 shown. The relationship between the actual reference distance value z0' and the preset reference distance value z0 is expressed by the ratio of the number of available pixels as follows: z0'=z0×(L/Xn); or z0'=z0×(W/Yn).

As shown in FIG. 10, the data processing module 130 analyzes whether the distribution of the distance value z(x, y) is reasonable according to the actual reference distance value z0', as shown in Step 124. For example, according to the thickness of the plate, the height of the standard container 301, 302, 303, and the possible filling amount of food, each distance value z(x, y) should be within a range between an upper distance value and a lower distance value. If If one of the distance values z(x, y) exceeds the range of the distance upper limit value and the distance lower limit value, the distance value z(x, y) is unreasonable, and steps 112 and 114 need to be performed again.

The one or more to-be-identified blocks B found in Step 130 may be standard containers 301 , 302 and 303 . The openings of the standard containers 301 , 302 and 303 face the upper part of the detection area A, and have a fixed top view shape.

As shown in FIG. 14 , at this time, the data processing module 130 can remove the parts other than the one or more to-be-identified blocks B in the image C to be analyzed according to the detection area A identified in Step 116 , so as to as a follow-up image.

As shown in FIG. 11 , the data processing module 130 loads the information of the standard containers 301 , 302 , 303 from the container database 150 , such as the area and shape of the opening, and analyzes whether the block B to be identified conforms to the area and shape of the opening of the standard container 301 , 302 , and 303 , so as to Determine each to-be-identified area Whether block B corresponds to correctly placed standard containers 301, 302, 303, as shown in Step 132.

As shown in FIG. 15 , the wrong arrangement includes: the standard containers 301 , 302 , and 303 interfere and overlap each other, so that the standard containers 301 , 302 , and 303 are jointly defined as a single block B to be identified, resulting in that the top view shape does not conform to any of the standard containers 301 , 302 , and 303 . the opening shape of the standard container 301, 302, 303 is overturned, resulting in a top view that does not conform to the opening shape of any of the standard containers 301, 302, 303; the standard container 301, 302, 303 is placed beyond the detection area A, so that the shape of the part located in the detection area A does not meet the standard The shape of the opening of the containers 301, 302, 303; or, the wrong non-standard container 304 placed in the detection area A.

As shown in FIG. 11 and FIG. 15 , when one or more blocks B to be identified do not conform to the opening shape of any standard container 301 , 302 , 303 , the data processing module 130 determines that at least one block B to be identified corresponds to the standard container 301 , 302 , 303 is not If it is placed correctly, the data processing module 130 generates an error prompt, and drives a display device or a warning device to issue an error prompt, as shown in Step 134 .

As shown in FIG. 11 , if each block B to be identified corresponds to correctly placed standard containers 301 , 302 , 303 , the data processing module 130 executes Step 140 to identify the type of a food 200 located in the block B to be identified. When the type of the food 200 is successfully identified, the volume and weight of the food 200 are continuously calculated, such as Step 142, Step 150, Step 160, and based on the identification of the diners, the identification information of the diners and the content of the meal are associated, and the data is uploaded to The management platform 400 is shown in step Step 170 .

As shown in FIG. 11 and FIG. 12 , some of the food 200 may be contained in specific standard containers 301 , 302 and 303 . Therefore, if it is determined in Step 142 that the type of the food 200 cannot be identified, the data processing module 130 analyzes whether it can be based on Standard Container Information, by Standard Container The types of 301, 302, and 303 determine the type of the food 200, as shown in Step 144. If the type of the food 200 can be determined by the types of the standard containers 301 , 302 and 303 , the data processing module 130 continues to execute Step 150 and Step 160 . If the type of the food 200 cannot be determined from the types of the standard containers 301 , 302 and 303 , the data processing module 130 executes Step 134 to generate an error message. Step Step 134 can also be omitted. When it is determined in Step 142 that the type of the food 200 cannot be identified, the data processing module 130 directly executes Step 134.

Through the combination of 3D scanning and image capture, the present invention marks the area occupied by the standard container in the detection area in the image to be analyzed based on the corresponding relationship between the captured image and the 3D scanning information, while only the standard container is detected. Image recognition is performed on the corresponding block to be recognized, instead of performing image recognition on the entire image to be analyzed, so as to determine the type of food. Therefore, the present invention can reduce the computing load required for image recognition, and can speed up the food recognition process. At the same time, through the application of standard containers, the distance distribution obtained by 3D scanning can be used to effectively and relatively accurately calculate the volume of food. With the correlation of food information, the information of the complete meal can be quickly analyzed, so as to combine the diners. Identify information for back-end management.

Step 112~160: Steps

Claims (16)

A method for recognizing food and beverage content, comprising: obtaining a plurality of distance values on a detection area through a three-dimensional scanner to generate a distance distribution, and using an image capture device to capture a to-be-analyzed area including the detection area image; a data processing module maps the distance distribution to the image to be analyzed; the data processing module sets a threshold distance to define each part of the image to be analyzed whose distance value is smaller than the threshold distance as at least one block to be identified; the data processing module performs image recognition on the at least one block to be identified to identify a type of food located in the block to be identified; and the data processing module obtains the information of the food according to the distance distribution volume; wherein, there is an actual reference distance value between the image capture device and a surface of the detection area, the threshold distance is less than or equal to the actual reference distance value; wherein, a length and a width of the detection area The number of pixels occupied in the image to be analyzed is Xn×Yn respectively, and there is a predetermined reference distance between the image capturing device and the surface of the detection area, and the detection area is at the predetermined reference distance The number of pixels occupied by an actual length and an actual width in the image to be analyzed are L×W respectively, and the relationship between the actual reference distance value and the preset reference distance value is: z0′=z0×( L/Xn); or z0'=z0×(W/Yn); wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value. The method for recognizing food and beverage content according to claim 1, wherein, before obtaining the plurality of distance values, the image capturing device continues to capture the image to be analyzed, and the data processing module Analyze whether the image to be analyzed includes the detection area. The food and beverage content identification method according to claim 1, further comprising the data processing module judging whether each distance value is within a range between a distance upper limit value and a distance lower limit value; if any of the distance values When the value exceeds the upper limit value of the distance and the lower limit value of the distance, the plurality of distance values are retrieved and the image to be analyzed is retrieved again. The food and beverage content identification method according to claim 1, further comprising the data processing module identifying whether the at least one block to be identified conforms to a correctly placed standard container according to the distance distribution and at least one standard container information. The food and beverage content identification method according to claim 1, wherein the step of identifying the type of the food located in the to-be-identified block comprises: the data processing module loads at least one piece of food information; wherein the at least one piece of food information at least Including the type of the food and a sample image; and the data processing module compares the image of the block to be identified and the sample image whether the image is similar, so as to identify the type of the food in the block to be identified. The method for identifying food and beverage content according to claim 5, further comprising: loading the at least one food information by the data processing module; wherein the at least one food information further includes the food density of the food; and the data processing module will The density of the food is multiplied by the volume of the food to obtain the weight of the food. The food and beverage content identification method according to claim 6, further comprising: loading the at least one food information by the data processing module; wherein, the at least one food information further includes a nutritional ingredient and a unit weight price; The data processing module multiplies the nutrient composition by the weight of the food to obtain a nutritional composition of the food; the data processing module multiplies the unit weight price by the weight of the food to obtain the price of the food; and The data processing module relates the type, weight, nutritional composition and price of the food to a diner, and uploads it to a management platform. The method for recognizing food and beverage content according to claim 1, further comprising the data processing module analyzing whether the at least one block to be identified conforms to a standard container for correct placement; when the at least one block to be identified does not conform to the correct placement of the standard container, an error message is generated. A food and beverage content recognition system, comprising: a three-dimensional scanner for acquiring a plurality of distance values on a detection area to generate a distance distribution; an image capture device for capturing one of the detection areas an image to be analyzed; and a data processing module electrically connected to the 3D scanner and the image capture device; wherein the data processing module maps the distance distribution to the image to be analyzed; the data processing module and set a threshold distance, so as to define each part of the image to be analyzed whose distance value is smaller than the threshold distance as at least one block to be recognized; and the data processing module performs image recognition on the at least one block to be recognized , identify the type of a food located in the at least one block to be identified, and obtain the volume of the food according to the distance distribution; wherein, there is an actual reference distance between the image capturing device and a surface of the detection area value, the threshold distance is less than or equal to the actual reference distance value; Wherein, the number of pixels occupied by a length and a width of the detection area in the image to be analyzed are Xn×Yn respectively, and there is a predetermined reference distance between the image capture device and the surface of the detection area value, the number of pixels occupied by an actual length and an actual width of the detection area under the preset reference distance value in the image to be analyzed are L×W respectively, the actual reference distance value and the preset reference distance value The relationship between them is: z0'=z0×(L/Xn); or z0'=z0×(W/Yn); wherein, z0 is the preset reference distance value, and z0' is the actual reference distance value. The food and beverage content recognition system according to claim 9, wherein, before obtaining the plurality of distance values, the image capture device continues to capture the image to be analyzed, and the data processing module analyzes whether the image to be analyzed includes the image to be analyzed detection area. The food and beverage content identification system according to claim 9, wherein the data processing module determines whether each distance value is within a range between a distance upper limit value and a distance lower limit value; In the range interval where the value exceeds the upper limit value of the distance and the lower limit value of the distance, the 3D scanner and the image capturing device re-acquire the plurality of distance values and re-capture the to-be-analyzed image. The food and beverage content identification system according to claim 9, wherein the data processing module loads at least one standard container information, and identifies whether the at least one block to be identified is based on the distance distribution and the at least one standard container information Complies with a properly placed standard container. The food and beverage content identification system according to claim 9, wherein the data processing module loads the type of the food and a sample image, and compares the image of the block to be identified and the sample image to determine whether they are similar to The to-be-identified block identifies the type of the food. The food and beverage content identification system of claim 13, wherein the data processing module loads the food density of the food, and multiplies the food density by the volume of the food to obtain the weight of the food. The food and beverage content identification system as claimed in claim 14, wherein the data processing module loads nutritional ingredients and price per unit weight, multiplies the nutritional ingredients by the weight of the food to obtain a nutritional composition of the food, and uses the The price per unit weight is multiplied by the weight of the food to obtain the price of the food; and the data processing module relates the type, weight, nutritional composition and price of the food to a diner, and uploads it to a management platform. The food and beverage content identification system according to claim 9, wherein the data processing module analyzes whether the at least one block to be identified conforms to a standard container that is correctly placed; when the at least one block to be identified does not conform to the correct placement of the standard container, an error message is generated.

Images