JP2018163615A - Information providing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information providing device and a program capable of providing information on eating and drinking behavior when a user eats and drinks. SOLUTION: A scene in which a user eats or drinks at least one served food or drink is imaged, the served food or drink is identified, and each of the identified foods and drinks of the user is based on the captured image. It is an information providing device that generates eating and drinking mode information related to eating and drinking modes of an object, and executes a predetermined information providing process based on the generated eating and drinking mode information during imaging of the eating and drinking scene. [Selection diagram] Fig. 1

Description

  The present invention relates to an information providing apparatus and a program for providing information in a eating and drinking scene.

  In recent years, research on lifestyle-related diseases due to dietary habits that lack nutritional balance has been widely conducted, and there is a need for an opportunity to review dietary lifestyles to prevent lifestyle-related diseases. As one of the methods for reviewing such eating habits, there is a service that records the contents of daily meals and provides advice from a nutrition manager or the like based on the records.

JP 2008-299803 A

  However, in the above conventional service, the user of the service will receive nutritional guidance based on the contents already eaten and eaten until the user points out a problem in the eating and drinking behavior. The time lag is large. In addition, for users, it will be nutritional guidance based on past eating and drinking behavior, so it is necessary to give repeated guidance to understand which foods and how much you can eat and drink at the time of current and future eating and drinking, Currently, it takes a considerable amount of time to change the user's eating and drinking behavior.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an information providing apparatus and a program that can provide information on eating and drinking behavior when the user eats and drinks.

  Note that Patent Document 1 discloses an example in which a non-contact IC tag attached to a shopping cart is used to analyze the purchase behavior of a customer in a store.

  One aspect of the present invention that solves the problems of the above-described conventional example is an information providing device, in which an image capturing unit that captures a scene in which a user eats or drinks at least one food or drink served is arranged. Identification means for identifying food and drink, and generation means for generating food and drink aspect information relating to the food and drink aspect of each identified food and drink based on the video imaged by the imaging means, A predetermined information providing process based on the generated eating and drinking mode information is executed during imaging of the eating and drinking scene by the imaging means.

  ADVANTAGE OF THE INVENTION According to this invention, the information regarding eating and drinking action can be provided in the case of a user's eating and drinking action.

It is a block diagram showing the example of a structure of the information provision apparatus which concerns on embodiment of this invention. It is a functional block diagram showing the example of the information provision apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the recognition result which the information provision apparatus which concerns on embodiment of this invention records. It is explanatory drawing showing the detection example of the eating and drinking action by the information provision apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the food / beverage record which the information provision apparatus which concerns on embodiment of this invention produces | generates. It is explanatory drawing showing the example of the information provision conditions set in the information provision apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of information provision by the information provision apparatus which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the drawings. The information providing apparatus 1 according to the embodiment of the present invention is provided in a dining room if it is in the home, a canteen or a restaurant if it is an organization or the like. As illustrated in FIG. 1, the information providing apparatus 1 includes a control unit 11, a storage unit 12, an imaging unit 13, an information providing unit 14, and a communication unit 15.

  The control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 accepts input of a video (information including a series of image data) of a user's eating and drinking scene imaged by the imaging unit 13 described later, and is arranged in the eating and drinking scene. Identify food and drink. Moreover, this control part 11 produces | generates the food / beverage aspect information which concerns on the food / beverage aspect of each said food / beverage product of a user, and provides the predetermined information based on the produced | generated food / beverage aspect information during the imaging of a food / beverage scene Execute the process. Details of the processing of the control unit 11 will be described later.

  The storage unit 12 is a memory device, a disk device, or the like, and holds a program executed by the control unit 11. The storage unit 12 also operates as a work memory for the control unit 11.

  The imaging unit 13 is a camera or the like, and is disposed above a dining table, a dining room, or other dining table, and a user serving a food or drink on the dining table or performing a eating or drinking operation toward the dining table (there are multiple users) May be taken). The imaging unit 13 outputs an image obtained by imaging to the control unit 11. In the present embodiment, the imaging unit 13 uses a table on which food and drinks are arranged at a predetermined timing (for example, regularly every 1/6 seconds) and uses food and drinks toward the table. Image data obtained by imaging the person is sequentially output.

  The information providing unit 14 is a projector, a speaker, or the like, for example, and provides image and audio information to the user according to an instruction input from the control unit 11. The communication unit 15 is a network interface or the like, and transmits and receives information to and from an external server or the like according to an instruction input from the control unit 11.

  Next, operation | movement of the control part 11 of this Embodiment is demonstrated. As illustrated in FIG. 2, the control unit 11 according to an example of the present embodiment includes a video reception unit 21, a recognition processing unit 22, a food and beverage mode information generation unit 23, a provision information generation unit 24, and information provision The processing unit 25 is functionally included.

  Here, the video receiving unit 21 accepts input of image data output by the imaging unit 13 and outputs the input image data to the recognition processing unit 22 and the eating and drinking mode information generation unit 23. The recognition process part 22 performs the process which each identifies the food / beverage currently imaged by image data about each image data which the image | video reception part 21 received. Specifically, the recognition processing unit 22 displays the type of food and drink contained in the video (other than fried chicken, vegetable salad, etc.) and the position (the food and drink are included) from the video received by the video receiving unit 21. Recognize the area where the dishes are arranged (hereinafter referred to as the serving position). The process of identifying the type of food or drink or recognizing the serving position can employ a widely known process such as a process of recognizing using the color or the like of the food or drink. Information that can be moved during a meal can be a widely known process such as a process of recognizing the movement of an object (here, food or drink) based on a comparison between image data input continuously in time. Detailed description is omitted here.

  Here, the recognition processing unit 22 recognizes the type of food and drink and the serving position based on the video, but the processing of the recognition processing unit 22 is not limited to this example. For example, the recognition processing unit 22 attaches an IC tag that transmits information on the type of food and drink to a dish or the like on which food or drink is served, receives information transmitted from the IC tag, and is distributed on the table. The type of each food and drink and its serving position may be recognized. In addition, a mat with a pressure sensor that can measure the distribution of weight for each place on the table is laid and placed on top of the table to measure the pressure distribution due to the weight of the tableware and food. Based on the result, the serving position and weight may be recognized. Such a mat can be formed by arranging pressure sensors in a matrix, for example.

  The recognition process part 22 specifies the area | region where the information (food name etc.) which specifies each food / beverage arranged on the table, and the food / beverage distribution position (the dish which arranged the food / beverage is arranged) Information on the center position coordinates (x, y) and radius R) of a circle circumscribing the dish, for example, is recorded as food and drink information (FIG. 3).

  The food / beverage mode information generation unit 23 generates the food / beverage mode information related to the food / beverage mode of each identified food and drink based on the video imaged by the imaging unit 13 and received by the video image reception unit 21. Specifically, the food / drink mode information generating unit 23 identifies the position of the user's hand (or each user when a plurality of users are imaged). Since the hand position recognition process can be performed by a widely known method such as identification of the user's head or skin color area, detailed description thereof will be omitted.

  The food / drink mode information generation unit 23 determines which food / drink each user is going to eat / drink by comparing the position of the hand for each user with the serving position of the food / beverage identified by the recognition processing unit 22.

  As an example, the food / beverage mode information generation unit 23 obtains hand positions (left and right hand positions) for each user at predetermined timings from image data that are sequentially input sequentially. Are sequentially recognized to obtain information on the movement locus of the hand position for each user. This information is, for example, the position of the center of gravity (Px (ti), Py (ti)) (where Px, Py is output by the imaging unit 13) for one hand of a user (referred to as a hand of interest for convenience). Ti (i = 1, 2,...) Represents time (processing timing). In the following description, it is assumed that t1 <t2... <Ti <ti + 1.

  The eating and drinking mode information generation unit 23 calculates, for example, ΔP = (Px (t) −Px (ti−1), Py (t) −Py (ti−1)), and the magnitude of this ΔP (the sum of squares of each component). Is larger than a predetermined threshold value, it is determined that the hand of interest is moving.

  When it is determined that the hand of interest is moving as described above, the eating and drinking mode information generation unit 23 determines the current center of gravity P of the hand of interest (assumed to be time ti) as illustrated in FIG. The food and drink in the region Q within the predetermined angle θ centered on the direction of ΔP is searched. That is, the food / beverage information generated and recorded by the recognition processing unit 22 from the image data at the time ti is referred to, and the food / beverage where the serving position recorded in the food / beverage information is in the region Q is searched. And if there exists such food and drink, the food-and-drink mode information generation part 23 uses the information on the kind of food and drink found by the search (if there are a plurality of kinds of information) as the candidate food / drink information Output together with information identifying the person.

  In addition, when the magnitude of ΔP (which may be the square root of the sum of squares of each component) is smaller than a predetermined threshold value, the eating and drinking mode information generation unit 23 determines that the attention hand is stationary. When it is determined that the hand of interest is moving as described above, the eating and drinking mode information generation unit 23 centers on the center of gravity P of the hand of interest (assumed to be time ti) as illustrated in FIG. And searches for food and drink within a circle R having a predetermined radius r.

  That is, with reference to the food / beverage information generated and recorded by the recognition processing unit 22 from the image data at time ti, the food / beverage position recorded in the food / beverage information is searched for the food / beverage in the region R of this circle. To do. And if there exists such food and drink, the food-and-drink mode information generation part 23 was linked | related with the information on the kind of food and drink found by the said search (if there is more than one, the information on the serving position closest to the gravity center position P) Information on the type of food and drink) is temporarily recorded as information on the food and drink, in association with information specifying the user related to the attention hand.

  In addition, when there is already food / beverage target information that is temporarily recorded (information associated with information that identifies a user related to the attention hand), the food / beverage mode information generation unit 23 overwrites the food / beverage target information. The information on the type of food and drink obtained this time is recorded as the food and drink target information in association with the information specifying the user related to the attention hand. Thereby, the food and drink acquired just before the user's attention hand returns to the vicinity of the user's head is specified.

  Furthermore, when the distance between the center-of-gravity position P of the attention hand and the center-of-gravity X of the corresponding user's head is less than a predetermined distance at the processing timing, the eating and drinking mode information generation unit 23 identifies the user. It is investigated whether there is food / beverage object information temporarily recorded in association with the information to be performed. If there is such food / beverage object information, the food / beverage aspect information generation unit 23 identifies the user related to the attention hand, which is recorded in association with the food / beverage object information. To get. And the food-and-drink mode information generation part 23 linked | related the information which specifies the acquired food and drink which the food-and-drink object information represents, the information which specifies a user, and the present time (information showing the time of recording) Records are accumulated and recorded as food and drink records (FIG. 5).

  In this way, if there is a food or drink that the user is reaching for, the food and drink mode information generation unit 23 outputs the information of the food or drink as the candidate food and drink information, and the user actually acquires it. If there is a food / drink that is recognized as having been eaten / drinked (in this example, when the hand rests in the vicinity of the food / beverage distribution position and then returns to the user's head), Information indicating the type, information for identifying the user, and time are recorded as food and drink records. In the example of the present embodiment, the candidate information for eating and drinking and the eating and drinking record correspond to the eating and drinking mode information.

  The provided information generation unit 24, during imaging of the eating and drinking scene by the imaging unit 13, the current or past eating and drinking mode information generated by the eating and drinking mode information generating unit 23 (in this example, the eating target candidate information and The information to be provided is generated based on the food and drink record. Specifically, in an example of the present embodiment, it is assumed that information on the type of food and drink and nutrition information are associated with the storage unit 12 and stored in advance as a nutrition value database. Here, the nutritional value information includes information such as calories, salt content, sugar content, and lipid per unit amount (which may be an amount that can be divided at once with a chopstick, a spoon, or the like).

  In addition, the provision information generation part 24 may produce | generate the information which should be provided with reference to the content of the information provided in the past as well as eating / drinking mode information so that it may mention later.

  The provided information generation unit 24 refers to the food and beverage record recorded by the food and beverage mode information generation unit 23, and the user k (k is a unique identification value for each user, for example, a natural number k = 1, 2,... The number of times F (k, j) (j is a unique identification value for each food and drink, for example, a natural number such as j = 1, 2,... ). And the provision information generation part 24 links | relates with the food / beverage j for every food / beverage, and the information on the nutrition value per unit quantity stored in the nutritional value database includes the number of times F (k, j) The product multiplied by, the total sum of the calculated products is calculated, and the value of nutritional value consumed by the user k from the start of the food and drink record to the present (calorie, salinity, sugar, lipid, etc. intake) Using the obtained nutritional value as the intake state information, the data record associated with the information specifying the user k and the current time (recording time) is accumulated and recorded.

  The provision information generation unit 24 specifies the user k among the information recorded as the intake state information when the eating / drinking mode information generation unit 23 outputs the candidate food / drink information and the information specifying the user k. The value of nutrition value associated with the information is acquired as current information. In addition, the nutritional value per unit amount stored in the nutritional value database in association with information (information specifying food and drink, each if there are multiple foods) included in the candidate food / drink information output by the food / drink mode information generation unit 23 Is obtained as estimated nutritional value information. The provided information generation unit 24 adds estimated nutritional value information (or estimated nutritional value information regarding each of the information included in the candidate information for eating and drinking) to the acquired current information to obtain the estimated accumulated nutritional value information. obtain.

  The provided information generation unit 24 refers to the obtained estimated accumulated nutritional value information and checks whether any one of predetermined information providing conditions is satisfied. Here, for example, as shown in FIG. 6, the information provision condition is a condition (C) relating to the nutritional value, and information on a template of provision information (T) to be generated when the condition is satisfied. Are associated with each other. In the example of FIG. 6, in response to the condition that the nutritional value is “salt content exceeds 10 grams”, the letters “(Food & Drink Name) Let's stop today” An example in which column information is recorded in association with each other is shown.

  When the estimated accumulated nutritional value information satisfying any of the information provision conditions is found, the provision information generating unit 24 includes information on the template (T) included in the information provision condition satisfying the estimated accumulated nutritional value information. And the information for specifying the food and drink associated with the estimated nutrient value information used for the calculation of the estimated accumulated nutrient value information is acquired. Then, the provided information generation unit 24 uses the acquired information (food name) specifying the food and drink and the information on the template, so that “(food and drink name) should be stopped today” The character string information such as is generated and output to the information provision processing unit 25 as provision information. Here, the part of “(Food and drink name)” is information on the character string replaced with the food and drink name.

  The information provision processing unit 25 presents the provision information output by the provision information generation unit 24 to the user. Specifically, the information provision processing unit 25 performs a known speech synthesis process based on the provision information output from the provision information generation unit 24 to generate a voice signal, controls the speaker of the information provision unit 14, and Sound an audio signal. When providing as image data, the information provision processing unit 25 generates image data based on the character string information output from the provision information generation unit 24 and controls the display, projector, and the like included in the information provision unit 14. The image data is displayed and output.

  Further, when the information provision processing unit 25 presents the provision information to the user, the information provision processing unit 25 associates the provision information with time information indicating the presentation time of the provision information, accumulates the information in the storage unit 12, and records the information provision record. May be stored as

  The present embodiment basically has the above-described configuration and operates as follows. In the following description, it is assumed that the information providing apparatus 1 is used at home. In addition, the imaging unit 13 is arranged above the dining room dining table, and is capable of imaging the entire dining table and a user located around the dining table during eating and drinking.

  The information providing apparatus 1 starts processing at the start of eating and drinking (at the start of serving, or at the start of eating after serving), for example, according to a user instruction. And the food and drink currently imaged by image data are each identified from each of the image data which the imaging part 13 outputs sequentially. And the kind of food / beverage imaged by image data (a distinction, such as fried chicken and vegetable salad), and the information showing the serving position are linked | related, and it records as food / beverage information as illustrated in FIG. .

  In the following explanation, it is assumed that rice, fried chicken, and vegetable salad are served on the table. In this case, about four foods and drinks of rice (2 meals for each of the users α, β), a dish with fried chicken, a dish with vegetable salad, Information to identify each ("rice", "fried chicken", "vegetable salad") and information on the serving position of each tea bowl or dish (coordinate values in the X, Y coordinate system and radius R representing the range) Shall be recorded in association with each other.

In the following description, it is assumed that the nutritional value of each food and drink is stored in advance in the nutritional value database as follows:
Rice (per bite)… Energy: 27 kcal, Protein: 0.4 g, Lipid: 0.05 g, Carbohydrate: 6 g, Salt: 0 g, Vitamin E: 0.4 mg ...
Deep-fried chicken (per piece) ... Energy: 100 kcal, protein: 4 grams, fat: 8 grams, carbohydrate: 1.2 grams, salt: 0.4 grams, vitamin E: 0.02 milligrams ...
Vegetable salad (per one serving average (20 grams)) ... Energy: 25 kcal, protein: 0.2 grams, lipid: 3 grams, carbohydrates: 1 gram, salt: 0.06 grams, vitamin E: 0.0. 6 milligrams ...

  The information providing apparatus 1 also generates information representing the position of the user's hand from each of a series of image data captured by the imaging unit 13. Here, when there are two users α and β who are eating toward the table, information indicating the positions of the hands of the users α and β is generated. Then, the information providing apparatus 1 obtains the center-of-gravity positions (Pαx (ti), Pαy (ti)), (Pβx (ti), Pβy (ti)) of each user's hand. Pαx and Pαy are the X and Y coordinate values of the center of gravity of the hand of the user α, Pβx and Pβy are the X and Y coordinate values of the center of gravity of the hand of the user β, and ti (i = 1, 2,...). Represents time. Here, t1 <t2... <Ti <ti + 1.

  Since the information providing device 1 can obtain ΔPα = (Pαx (t) −Pαx (ti−1), Pαy (t) −Pαy (ti−1)) and ΔPβ (ΔPα), (Repeat description is omitted). The information providing apparatus 1 checks whether the magnitudes of ΔPα and ΔPβ are larger than a predetermined threshold value, and determines that the corresponding user's hand is moving when the magnitude is larger than the threshold value.

  For example, when the information providing apparatus 1 determines that the hand of the user α is moving, the information providing apparatus 1 preliminarily centers the direction of ΔPα from the center of gravity P of the hand of the user α (current time t i). The food and drink in the area Q within the determined angle θ is searched. Here, it is assumed that there is a fried dish and a vegetable salad dish in the region Q. In this case, the information providing apparatus 1 stores “fried chicken” and “vegetable salad” in association with information (user α) that identifies the user as food and drink candidate information.

  At this time, as will be described later, the information providing apparatus 1 obtains the estimated nutrient value information and the estimated accumulated nutrient value information and executes a process of providing the information. For convenience, the information provided at this stage Suppose that there was no.

  Eventually, when the hand of the user α moves to the vicinity of the deep-fried dish and the work of picking up the deep-fried food from this dish is performed, the center of gravity of the hand of the user α becomes stationary at that time, so ΔPα The size is smaller than a predetermined threshold value. At this time, the information providing apparatus 1 searches for food and drink in a circle R having a predetermined radius r centered on the gravity center position Pα of the hand of the user α. Here, fried dishes are searched. The information providing apparatus 1 then records the information of the type of food and drink found by the search as “Fried food” temporarily in association with the information specifying the user α as the food target information.

  Thereafter, when the user α picks up one fried chicken and carries it to the hand, the information providing apparatus 1 determines that the hand of the user α is moving. When the user α carries the fried chicken as it is (without picking up food and drink other than the deep-fried food), the center of gravity position Pα of the hand of the user α and the center of gravity Xα of the corresponding user α's head Since the distance is less than the predetermined distance, the information providing apparatus 1 checks whether or not there is food and drink target information temporarily recorded in association with the information specifying the user α at that time. Here, as described above, since the information of “fried chicken” is recorded as the candidate food / drink information in association with the information (user α) that identifies the user, the information providing apparatus 1 is configured as the target information for food / drink. The information “Karaage” that specifies the food and drink represented by, the information that identifies the user (user α), and the current time (information that indicates the time of recording) are associated and stored as a food and drink record. Record. This record is as illustrated in FIG. The information providing apparatus 1 performs the same process for the user β.

Furthermore, the information provision apparatus 1 is the number of times of eating and drinking F (k, j) (j is for each food and drink) for each user at every predetermined timing during eating and drinking (for example, every time a food and drink record is recorded). It is a unique identification value, for example, a natural number j = 1, 2,. Here, for example, by time t i, the user α picks up and eats 5 times of rice from the rice bowl and 3 times of the deep-fried rice from the dish, and the user β eats from the rice bowl. 3 times, 1 time of deep-fried food from a plate of deep-fried food, and 1 time of vegetable salad. In this case, the number of eating and drinking is
F (α, 1) = 5
F (α, 2) = 3
F (α, 3) = 0
F (β, 4) = 4
F (β, 2) = 1
F (β, 3) = 1
It becomes. In this example, the unique identification value of the user α rice bowl is “1”, the unique identification value of the fried dish is “2”, the unique identification value of the vegetable salad dish is “3”, The unique identification value of the rice cake of user β is “4”.

  The information providing apparatus 1 obtains the intake of nutritional value up to the present time (time ti) of each user. This value is obtained using the contents of the previous nutritional value database and the number of times of eating and drinking here. For example, the energy consumed by the user α is 27 × 5 + 100 × 3 + 25 × 0 = 435 kcal, the lipid is 0.05 × 5 + 8 × 3 + 3 × 0 = 24.25 grams, and the salt is 0 × 5 + 0.4 × 3 + 0.06 × 0. = 1.2 grams, vitamin E is calculated as 0.4 × 5 + 0.02 × 3 + 0.6 × 0 = 2.06 milligrams. The information providing device 1 records information on the intake of nutritional value up to the current time (time ti) of each user as intake state information.

  Thereafter, when the information providing apparatus 1 determines that, for example, the user α is moving again, the information providing apparatus 1 is centered on the direction of ΔPα at that time from the center of gravity P of the hand of the user α at that time. The food and drink in the area Q within the predetermined angle θ are searched. Here, it is assumed that there is a fried dish and a vegetable salad dish in the region Q. In this case, the information providing apparatus 1 stores “fried chicken” and “vegetable salad” in association with information (user α) that identifies the user as food and drink candidate information.

  And the information provision apparatus 1 calculates | requires estimated nutrient value information and estimated accumulated nutrient value information, and performs the process which provides information. Specifically, the information providing apparatus 1 acquires, from the nutritional value database, information on the nutritional value related to the food and drink specified by the candidate food / drink information as estimated nutritional value information. Then, the information on the acquired nutritional value is added to the intake state information of the user α at this time to obtain estimated cumulative nutritional value information. Here, since the candidate information for eating and drinking is two, “fried chicken” and “vegetable salad”, the information providing apparatus 1 adds information on each nutritional value and the user α ingests each of them. Find estimated cumulative nutritional value information for. Specifically, in the above-described example, as the estimated accumulated nutritional value information when the user α acquires “fried chicken”, the energy is 435 + 100 = 535 kcal, the lipid is 24.25 + 8 = 32.25 grams, and the salt is 1 .2 + 0.4 = 1.6 grams, vitamin E is calculated to be 2.06 + 0.02 = 2.08 milligrams, etc. As the estimated accumulated nutritional value information when the user α acquires “vegetable salad”, the energy is 435 + 25 = 460 kcal, the lipid is 24.25 + 3 = 27.25 grams, and the salt is 1.2 + 0.06 = 1. It is calculated that 26 grams, vitamin E is 2.06 + 0.6 = 2.66 milligrams, and so on.

The information providing apparatus 1 refers to the calculated estimated accumulated nutritional value information and checks whether any one of predetermined information providing conditions is satisfied. As an example, a case will be described below in which information provision conditions are associated with a template of provision information corresponding to the information provision conditions and are set as follows. That is,
(Condition 1) Condition C related to nutritional value: “lipids are 30 grams or more”, provided information template T: “(Food and drink names) should be stopped today”
(Condition 2) Condition C related to nutritional value: “Vitamin E is 2.5 milligrams or more”, provided information template T: “Would you like to eat (name of food and drink)?”
...
It is assumed that a set of information provision conditions such as a provision information template is stored in the storage unit 12 in advance. Hereinafter, this record of information sets is referred to as a provided information database.

  In the case of this example, the information providing apparatus 1 has a condition that the fat is 32.25 grams in the estimated accumulated nutritional value information when the user α acquires “fried chicken”, and the condition C relating to the nutritional value of Condition 1: Since “Fat is 30 grams or more”, we will replace the corresponding part of the corresponding template with “Fried chicken” which is the candidate information for eating and drinking as the food name. The information to be provided is generated. And the sound of this provision object information is sounded, for example.

  In addition, at this time, the information providing apparatus 1 uses the estimated accumulated nutritional value information when the user α acquires “vegetable salad” to have 2.66 milligrams of vitamin E, and the condition C relating to the nutritional value of condition 2: Since “Vitamin E is 2.5 milligrams or more”, “Vegetable salad”, which is candidate information for eating and drinking at that time, is replaced with the corresponding part of the corresponding template, "Would you like to eat?" And the sound of this provision object information is sounded, for example.

  According to this example, even if the user α is trying to reach for fried chicken, the information providing device 1 says “Let ’s stop fried chicken today.” You will hear the voice "Would you like to eat?" That is, in this example, corresponding information is provided to the user at the appropriate time during eating and drinking, and the change in the eating and drinking behavior of the user is promoted.

  In addition, since this Embodiment is not restricted to the example described here, some modifications are demonstrated below.

[Information provision using projectors]
For example, in the above description, the information provision has been described by exemplifying sound, but the present embodiment is not limited to this. For example, the information providing apparatus 1 may include a projector having a screen serving as a table serving food and drink, and may present information on the table. For example, in the above-described operation example, when the provision target information “Keep deep fried today should be stopped” is generated, an icon for calling attention is displayed near the serving position of the fried chicken dish (FIG. 7). You may do it.

  In addition to the projector, information may be provided by a method such as providing a lighting device or the like and applying a spotlight in the vicinity of the serving position of any food or drink.

[Weight measurement]
Furthermore, in the examples so far, the amount of food and drink that the user acquires in one operation is set in advance, so that the amount of food and drink is estimated based on the number of acquisitions of food and drink. The form is not limited to this, for example, a table, a dish, a weight sensor that measures the weight of the food or drink placed on the table, and a wireless that sends the measurement result of the weight to the information providing apparatus 1 wirelessly. The information providing device 1 acquires information on the weight measurement result from a table or a plate, obtains the amount of change in weight before and after the user performs the acquisition operation, and uses the amount actually consumed Nutritional value information, estimated accumulated nutritional value information, and the like may be calculated.

[Information provision based on the provision information history]
In the above example, every time the user α tries to reach for “vegetable salad”, the condition “Vitamin E is 2.5 milligrams or more” in condition 2 is satisfied. "Why don't you eat?" Therefore, in an example of the present embodiment, the information providing apparatus 1 may accumulate and record the provision information provided in the past so that the same provision information is not repeatedly provided.

  Specifically, in the information providing apparatus 1 according to an example of the present embodiment, flag information indicating whether to repeatedly provide information is further associated with each template of provided information included in the information providing condition. For example, in the above example, the provision information template T associated with the condition C related to nutritional value: “lipid is 30 grams or more”: “Let ’s quit (food name) today” Is associated with flag information to be repeatedly provided, and the provisional information T associated with the condition C related to nutritional value: “Vitamin E is 2.5 mg or more”: “(food and drink name) "Would you like to eat?" Is stored in the storage unit 12 in association with flag information that is not repeatedly provided.

  When the information providing apparatus 1 operates as the information provision processing unit 25, each time the provision information is presented to the user, the content of the presented provision information is accumulated and stored in the storage unit 12.

  In addition, when the information providing apparatus 1 operates as the provision information generation unit 24, after obtaining the estimated accumulated nutrition value information, if there is an information provision condition that satisfies the estimated accumulated nutrition value information, the information provision condition The information (T) of the template included in and the flag information associated therewith are read out. Moreover, the information which specifies the food / beverage linked | related with the estimated nutrient value information used for the calculation of the said estimated accumulated nutrient value information is acquired.

  Then, the provided information generating unit 24 uses the information for identifying the food and drink (name of the food and drink) and the information on the model, and provides the provided information such as “(food and drink name) should be stopped today”. Generate. The part of “(Food and drink name)” is replaced with the name of the food and drink.

  If the read flag information indicates that it is not repeatedly provided, the provision information generation unit 24 checks whether or not the generated provision information is stored in the storage unit 12 as information provided in the past. If the provision information generated here is stored in the storage unit 12 as information provided in the past, the provision information generation unit 24 controls not to output the generated provision information. On the other hand, if the generated provision information is not stored in the storage unit 12 as information provided in the past, the generated provision information is output.

  The provided information generation unit 24 outputs the generated provided information if the read flag information indicates that the information is repeatedly provided.

[Considering behavioral changes before and after providing information]
Further, in the present embodiment, when the information provision record is recorded as described above, the food and drink records before and after each time point when the provision information recorded as the information provision record is presented are referred to. You may perform the process which detects the change of eating and drinking action.

  Specifically, in this example, the information providing apparatus 1 holds a plurality of corresponding provision information templates in association with information provision conditions in the provision information database.

For example, for each of a plurality of provision information templates associated with a common information provision condition, information on the strength of each provision information (strength of appeal) is set. As an example,
In relation to the condition C relating to the nutritional value: “lipid is 30 g or more”,
“Strength 1” provided information template T1: “(Food and drink names) should be stopped today”
Informational template T2 for “Strength 2”: “(Food and drink name) should stop today”
“Strength 3” provided information template T3: “If you consume more (name of food and drink), it will harm your health.”
(Whether the level of appeal is strong or not is determined statistically and experimentally, and the contents of the template are determined, or based on past trends for each user. It may be set in machine learning that the strength of the information providing effect is stronger).

  When the information providing apparatus 1 determines that a certain information providing condition CX is satisfied, the information providing apparatus 1 determines whether there is an effect of providing information in the past in relation to the information providing condition, and there is no effect of providing information. If it is determined, the information is related to the satisfied information provision condition CX, is referred to the information on the strength of the provision target information provided in the past, is set as the strength information stronger than the referenced strength, and the information provision condition CX Information to be provided associated with the information is extracted, and information provision processing based on the information to be provided is performed.

  The determination as to whether or not the information provision effect has been achieved can be made, for example, as follows. That is, the control unit 11 of the information providing apparatus 1 refers to the information provision record and acquires information τ1, τ2,... Of the time when the provision information is presented. And, based on the eating and drinking records before and after each of τi (i = 1, 2,...), The intake of each type of nutritional value during the time from the recording start time to τ1, between τi and τi + 1 The intakes (i = 1, 2,...) Of the nutritional value of each type during the time are obtained.

  And the control part 11 calculates the difference of the intake of each kind of nutrition value in the time before and after the time of each information provision, and checks whether there was a change exceeding the threshold value determined for each kind of nutrition value. , Judge whether there is an effect of information provision.

  As an example, the control unit 11 of the information providing apparatus 1 includes the intake amount of carbohydrates consumed during the time period from τi-1 to τi before providing information at the time point of τi, and τi to τi after providing the information. If the absolute value of the difference between the intake of carbohydrates consumed during the time period up to +1 exceeds a predetermined threshold value for carbohydrates, it is judged that there was an effect of providing information at the time of τi. .

  Further, the control unit 11 takes the intake of each nutritional value ingested during the time from τi-1 to τi before the information provision at the time of τi, and after the information provision, from τi to τi + 1. If the absolute value of the difference from the intake of each nutritional value taken in time does not exceed the threshold value predetermined for each nutritional value, the effect of providing information at the time of τi Judge that there was no.

  In this way, for example, when detecting the eating and drinking behavior of fried chicken, when the information providing device 1 determines that the information providing condition relating to the intake of fats and oils (for the first time in the meal) is satisfied, Using the provided information template (lowest intensity), provide information such as “Keep fried deeper today”.

  However, when there is no change in the intake of fats and oils before and after the information provision, the information provision device 1 determines that the information provision is not effective, and provides information provision related to the information provision conditions related to the intake of fats and oils. Increment strength. For example, when the information providing device 1 determines that the information providing condition relating to the intake of fats and oils is satisfied next time, the information providing device 1 uses the template of the provided information of “strength 2”. Information is provided.

Note that the content of the appeal may be changed instead of changing the extent of the appeal by changing the strength. For example,
In relation to the condition C relating to the nutritional value: “lipid is 30 g or more”,
Providing information template T1: “Strength 1”, “Let ’s stop eating today”, “Intensity 2” providing information template T2, You can set it as an advice to reduce the total calorie content. The setting of such a template can be appropriately determined experimentally and empirically.

  As described above, in this example of the present embodiment, the process of providing information is performed while continuously detecting the user's eating and drinking behavior with a camera or the like and checking whether the user's predetermined eating and drinking behavior is effective in a timely manner. Go and shorten the period until the user's behavior changes.

  Whether or not the eating and drinking behavior is changed may be determined based on the change in the intake tendency as described above, or may be determined by machine learning based on various obtained information. May be.

[Consideration of eating and drinking time]
Further, in the description so far, the information provision condition includes the condition relating to the nutritional value, but the information provision condition in the present embodiment is not limited to this. For example, as information provision conditions, in addition to conditions related to nutritional value, conditions related to eating and drinking time may be included. In this example, specifically

Condition C1: related to eating and drinking time “Time is between 21:00 and 3:00”, Condition C2 related to nutritional value: “Energy is 100 kcal or more”, Model T of provided information: “(name of food and drink) Let's do it tomorrow "
As described above, the condition relating to the eating and drinking time and the condition relating to the nutritional value are stored in the storage unit 12 in association with the template of the provided information.

  Then, the information providing apparatus 1 of this example calculates the estimated accumulated nutritional value information and refers to the current time (the information on the current time may be obtained from a time measuring means such as a calendar IC (not shown)). Check whether any of the information provision conditions are satisfied.

  For example, if the information provision conditions described above are set, the estimated cumulative nutritional value information when the user is trying to reach for the food “Kakiage” When the energy information exceeds 100 kcal, if the current time is after 3:00 and before 21:00, the information providing apparatus 1 determines that the information providing condition is not satisfied. On the other hand, if the information on energy exceeds 100 kcal in the estimated accumulated nutritional value information when ingesting “fried chicken” and the current time is after 21:00 and before 3:00, the information providing apparatus 1 Determines that the information provision condition is satisfied, and obtains a template of the provision information associated with the condition “(Food and drink name) let's tomorrow”. And the information provision apparatus 1 replaces the part of "(Food and drink name)" with "Fried food" which is the name of the food which the user is going to ingest, and produces | generates and outputs provision information. As a result, information such as “Let's make fried chicken tomorrow” is provided, for example, by voice.

[Consideration of eating and drinking speed]
Moreover, the information provision apparatus 1 may use the conditions concerning the eating and drinking speed as information provision conditions. In this example, the information providing apparatus 1 refers to the food / beverage record or the record of the intake state information, obtains the user's food / drink speed as an example of the food / beverage mode information, and provides information based on the food / beverage speed. Specifically, the information providing apparatus 1 refers to the time information of each data record included in the food and beverage record or the intake state information record, and in the i-th recorded data record and the i + 1-th recorded data record. A difference Δti in time information is obtained (where i = 1, 2,...). Further, the information providing apparatus 1 obtains an average Δtav for the latest n times (n is a positive integer determined arbitrarily) of the information Δti at this time, and the condition that the average Δtav falls below a predetermined threshold (information When the provision condition is satisfied, it is determined that the eating and drinking speed is fast, and information such as “How about being a little slower?” Is provided.

  Moreover, the information provision apparatus 1 calculates | requires the increase rate (average amount of increase per unit time) for every nutrition value from the record of intake state information, and uses the information provision conditions concerning the increase rate for each calculated nutrition value. Information provision processing may be executed. In this example, for each nutritional value, a threshold for the average amount of increase per unit time (upper threshold that provides information when the threshold is exceeded, and lower than the threshold And a lower threshold value for providing information) may be defined as an information provision condition.

  The information providing apparatus 1 in this example refers to the record of the intake state information, and each nutrition value νj [i] in the i-th recorded data record and each nutrition value νj [in the i + 1-th recorded data record. The difference value Δνj [i] from i + 1] is obtained (where i = 1, 2,..., j = 1, 2,...). Then, the information providing apparatus 1 obtains an average Δνj_av for the latest n times (n is a positive integer determined arbitrarily) of the information Δνj [i] at this time, and the average Δνj_av is determined in advance for the corresponding nutritional value. Information when the condition of exceeding the threshold (upper threshold) is satisfied, or when the condition of falling below a predetermined threshold (lower threshold) for the corresponding nutritional value is satisfied Perform the provided process.

  As an example, assuming that the nutritional value of “vitamin A” is set in advance such that the lower threshold value is set to 9 micrograms per minute and the upper threshold value is set to 300 micrograms, the information providing apparatus 1 is ingested. Referring to the state information, when the average difference Δν (va) _av for vitamin A is below the lower threshold value of 9 micrograms, vitamin A is included in the food and drink being served Provide information such as "You should take (food and drink) a little more" in order from foods and drinks randomly or in order of increasing vitamin A content. In addition, when the information providing apparatus 1 has an average difference Δν (va) _av of vitamin A that exceeds the upper threshold value of 300 micrograms, the information providing apparatus 1 randomly selects a food or drink containing vitamin A, or In order of increasing vitamin A content, information such as “(Food and drink) should be eaten too soon?” Is provided.

  Moreover, although the frequency | count of food / drink per unit time and the increase amount for every nutrition value were used as information on the speed of food / drink here, this Embodiment is not restricted to this. For example, information such as the number of appearances for each food and drink may be used. For example, the information providing apparatus 1 according to the present embodiment uses the number of times of eating and drinking F (k, j) for each food and drink j (j = 1, 2,...) Up to the current time, and the total number FT of eating and drinking for each user k. (K) = ratio of the number of times of eating and drinking of each food to Σ_j (F (k, j)) FR (k, j) = F (k, j) / FT (k) (Note that Σ_j (X) is X Means the sum of j).

  And when the ratio FR (k, j) of the number of times of eating and drinking exceeds a predetermined threshold value, the information providing device 1 is not eating “(user k), (food and drink j), "Let's eat stuff!" Here, “(user k)” is replaced with information such as the name of the user k. Also, “(food / drink j)” is replaced with information on the food / drink name of the food / beverage j.

[Eating and drinking order]
In an example of the present embodiment, conditions relating to the order of eating and drinking may be used as the information providing conditions. In this example, the information providing apparatus 1 refers to the food and drink record and checks the order of eating and drinking of the user's food and drink as an example of the food and drink mode information. Then, when the order of eating and drinking satisfies the predetermined condition, the information providing apparatus 1 provides information set in advance in association with the condition. For example, as a condition related to the order of eating and drinking, when the condition is that “the same food appears three or more times in succession”, the information to be provided is “Don't eat the same thing, eat other things. ”And related information. The information providing apparatus 1 refers to the food and drink record, and when the same food and drink is recorded continuously three or more times in the order of food and drink of a certain user, It is judged that the condition “appears” is satisfied, and the sound based on the character string information set in association with the condition, “Let's eat only the same thing and eat other things” is sounded. Etc. to provide information.

[Per-user settings]
In the description so far, the information provision conditions may be set differently for each user. As an example, in the present embodiment, the information provision condition may include a condition related to the user. For example, regarding conditions related to estimated accumulated nutritional value information, taking into account gender differences and age differences,
(Condition 1a) Condition C related to nutritional value: “Zinc is 3 milligrams or less” and condition Cu related to users: Age range is “17 years old or younger” or gender is “female”, provided information model T: “ Would you like to eat a little more (food name)? "
(Condition 1b) Condition C relating to nutritional value: “Zinc is 4 milligrams or less” and Condition Cu relating to users: Age range is “over 17 years old” and sex is “male”, provided information template T: “Would you like to eat a little more (food and beverage names)?”
And so on.

  In this example, the information providing apparatus 1 receives and stores in advance information such as age for each user (may be calculated from the date of birth and current date and time), gender, height, weight, physical condition, and the like. deep. Further, the user is recognized from the image data obtained by imaging, and the position of the head and the position of the hand are identified for each user. Here, the recognition of the user may be explicitly specified by the user, or a process of recognizing from the image data may be performed. Further, a weight scale may be installed on a chair or the like, and the measurement result may be accepted and the seating position of the user may be recognized by comparing with the weight information set for each user.

  When the estimated accumulated nutritional value information about the user k is obtained, the information providing apparatus 1 acquires information on the age and gender of the user k and is included in the condition (1a) or (1b). When the condition C relating to the nutritional value is satisfied and the condition Cu relating to the user is satisfied, the provided information template associated with the satisfied condition is read and information is provided using the template. Perform the process.

  Specifically, when the value of the nutritional value “zinc” included in the estimated accumulated nutritional value information obtained for the user k is 3.5 milligrams and the user k is a 20-year-old woman, the condition ( Since neither 1a) nor (1b) is satisfied, information provision based on the condition (1a) or (1b) is not performed.

  On the other hand, when the value of the nutritional value “zinc” included in the estimated accumulated nutritional value information obtained for the user k ′ is 3.5 milligrams and the user k ′ is a 20-year-old male, the condition ( Since 1b) is satisfied, information such as "Would you like to eat the lever a little more?" Is provided (the lever is on the table and the hand of the user k 'is extended in the direction of the serving position of the lever. When).

[Caching behavior]
Furthermore, in the examples so far, information provision during eating and drinking has been described, but the information provision apparatus 1 according to the present embodiment is not necessarily limited to information provision during eating and drinking. In the information providing apparatus 1 of this example, the value obtained by dividing each nutrition information included in the total by the number of users using the table as the information providing condition is a value that should be taken by each user for each meal. For each condition that there is a nutritional value that does not reach the limit, or that there is a nutritional value that reaches an excessive value when each user consumes each meal. Associate and set.

  For example, as one of the information provision conditions, provisions such as "Is there any food containing iron?" For the condition that iron is 10 x N milligrams or less at a table where N adults eat and drink Associate and set information templates. In another information provision condition, in the table where N adults eat and drink, the provision of information such as “It seems that there is too much amount” for the condition that vitamin A is 2700 × N micrograms or more. Set the template in association.

  The information providing apparatus 1 of this example recognizes food and drink served from image data obtained by imaging a table. As the behavior of the catering, it may be determined that the user has been catered when there is an act of moving food and drink from outside the angle of view of the image data to the table. In addition, the end of the serving may be determined as having been finished when the user stays in the vicinity of the table for a time equal to or greater than a predetermined threshold. Moreover, the information provision apparatus 1 recognizes the food and drink on the table at the end of the serving, and further recognizes the number N of users located near the table at the end of the serving. As described above, widely known techniques can be adopted for the processing of these recognitions.

  The information providing apparatus 1 obtains the sum total of the nutrition value information of each recognized food and drink, and if there is an information provision condition including a condition that the value of the nutrition value included in the sum and the number N of users are satisfied Then, an information provision process based on the provision information included in the information provision condition is executed.

  Specifically, when the information provision conditions regarding iron are set as described above, the recognized food and drink is divided into a portion of rice (total 0.2 mg of iron) and deep-fried (iron) If the number N of recognized users is N = 2, the information providing apparatus 1 has a total iron content of 1.1 milligrams as one of nutritional value information. Yes, in order to satisfy the condition that the iron content is one of the information provision conditions of 10 × N = 10 × 2 = 20 milligrams or less, based on the model associated with the condition, “Iron content is included. A sound like "Is there any food?"

  In this example as well, the information provision condition may include the time of serving. For example, the condition relating to calories included in the information provision condition regarding food and drink distributed around 11 pm is stricter than the condition relating to calories included in the information provision condition serving as food and drink distributed around 7 pm ( Information provision device 1 obtains time information at the end of serving and provides information that also satisfies the conditions related to this time information. Search for conditions.

[Provision of information based on past food and drink records]
Furthermore, as information on the food and beverage mode, information on foods and beverages eaten and consumed in the user's past meals (last meal, last week, meals of the last month, etc.) (types of each food and drink, Records of eating order, eating speed, etc.) may be included.

  In this example, the information provision apparatus 1 calculates | requires the nutrition value information etc. which were ingested based on the information of the food and drink contained in the user's past meal record. As an example, the information providing apparatus 1 uses the information of the food and drink name as a key when the food and drink name and the information on the amount of food and drink are included in the record of food and drink for the most recent week from the day when the information providing process is performed. Then, the nutrition value database is referred to, corrected by the information on the amount of food and drink, and the information on the nutrition value ingested in the most recent week (weekly cumulative nutrition value information) is generated.

  In this example, the information provision condition may be changed based on this weekly cumulative intake nutrition value information. As an example, when the condition that “the value of energy contained in the cumulative nutritional value information for each day is 3000 kcal or more” is set, the average day of the most recent 7 days is determined based on the cumulative nutritional value information for the week. If the energy intake per unit exceeds 3000 kilocalories (when the energy information included in the 7-day cumulative nutritional value information exceeds 21,000 kilocalories), the provided information template will be “Food and drink soon. ”Let ’s refrain” ”, and corrected the condition that“ the value of energy included in the daily nutritional value information is 3000 kilocalories or more ”. The energy value included in the intake nutrition value information is 2800 kcal or more.

  Conversely, if the average daily energy intake for the most recent 7 days is below 3000 kilocalories, exceeding a predetermined threshold (for example, 500 kilocalories) (in this case, it is included in the cumulative total nutritional value for 7 days) If the energy information is less than 17500 kilocalories), the condition that “the energy value included in the daily cumulative nutritional value information is 3000 kilocalories or more” is corrected. The energy value included in the intake nutrition value information is 3200 kilocalories or more.

  According to this example, it is possible to perform processing according to the daily intake of nutritional value in consideration of information on recent nutritional value.

  In addition, although the example which correct | amends information provision conditions based on the information of nutrition value was described here, correction of information provision conditions is not restricted to this, It is information of a user's attribute, ie, the latest height, weight, You may perform based on the body fat rate, the amount of exercise, the change of activity amount, etc., or the information of a medical history.

  For example, when there is a history of calculus in the past, the information provision conditions regarding the intake of meat may be corrected so that information is provided with a smaller intake.

[Other examples of eating and drinking]
Moreover, the food-drinking mode information generation part 23 which the control part 11 of the information provision apparatus 1 of this Embodiment implement | achieves as the description so far, a user "reaching for food and drink" In addition to recognizing the action of “taking food and drink” and recording it as a food and drink record, the following actions may be recognized and included in the food and drink record.

For example, the eating and drinking mode information generation unit 23 is for each user,
"I replaced the rice"
"I drank a drink"
"I added a drink"
"I finished eating and drinking"
May be included in the food and drink record. The recognition of these actions can be performed by using a known technique, for example, based on the fact that the user has moved out of the angle of view of the image data, the increase / decrease of the contents such as teacups and cups, etc. Detailed description is omitted.

[Modification]
Furthermore, in the above example of the present embodiment, the case of providing information about eating and drinking behavior at the table has been described as an example. However, the present embodiment is not limited to this, for example, the eating and drinking behavior of a customer for each table in a restaurant May be used for the service.

[Communication with server]
In the above description, the information provision conditions, the nutritional value database, and the like are stored in the storage unit 12. However, the present embodiment is not limited to this, and communication is possible via the communication unit 15. These may be stored in a server on a simple network. In this case, the control part 11 will acquire the information provided in information provision conditions or a nutrient value database from a server.

  Furthermore, the record which the information provision apparatus 1 produces | generates, such as a food / beverage record, may be recorded on the server which can communicate via a network.

[Cooperation with restaurant records]
Furthermore, in an example of this Embodiment, the information provision apparatus 1 acquires the user's eating-out record, for example from the smart phone which a user has, or the server connected via the network, with the content of the food-and-drink record, You may use for the process of information provision.

  In this example, the information providing apparatus 1 uses information on food and drink included in the eating-out record, ingested nutrition value information, and the like together with the content of the food and drink record and ingested nutrition value information. As an example, if the information on the name of the eaten food is included in the record of eating out on the day of the information providing process, the information providing apparatus 1 uses the information on the name of the food and drink as a key for the nutrition value database. Refer to and obtain information on nutritional value ingested by eating out.

  And the information provision apparatus 1 adds the information of the nutrition value contained in the nutritional value information included based on the nutritional value information generated based on the food and beverage record, the information on the corresponding nutritional value ingested by eating out, and the food and beverage record at the time of another food and beverage (If the current dinner is dinner, breakfast and lunch eaten at the same table or lunch records) add the corresponding nutrition value information and generate daily cumulative nutrition value information To do.

  Further, in this example, the condition based on the daily intake nutrition value information is included as the condition to be included in the information provision condition. As an example, in response to the condition that “the energy value included in the daily cumulative nutritional value information is 3000 kilocalories or more”, information such as “let's refrain from eating and drinking today” as a template of the provided information Associate and set. According to this example, it is possible to perform processing according to the daily intake of nutritional value in consideration of eating out.

[Example using machine learning]
Furthermore, as described above, the information providing apparatus 1 according to the present embodiment sets information providing conditions and provides information to be provided using machine-learned results instead of processing to provide information by conditional branching. It may be determined. The provided information generating unit 24 of the control unit 11 of the information providing apparatus 1 according to this example includes the eating and drinking mode information, information that specifies past provided information (provided for the most recent predetermined number of times), the current time, Is included. This neural network outputs information for selecting one of a plurality of predetermined provision information.

  In this example of the present embodiment, the provision information generation unit 24 includes, for example, the eating and drinking mode information x [ti], the provision information a [ti-1] immediately before as past provision information, and the current time ti. Based on this, vector information st = f (x [ti], a [ti-1], ti) representing the state of the user is generated for each user. Further, this vector information may include vector information representing past user states. That is, st = f (x [ti], a [ti-1], ti, st-1) may be used.

  The provision information generation unit 24 outputs information for selecting any one of a plurality of predetermined provision information using a neural network that receives the vector information st.

  The user or the administrator of the information providing apparatus 1 actually inputs some food / beverage mode information, information for specifying past provided information, and the current time, the food / beverage mode information, and the past provision Information for selecting information and information for selecting provided information to be output when the current time is input are given to the neural network to perform so-called supervised learning, To learn the output. Since the learning process of such a neural network is a well-known technique, description thereof is omitted here.

  In this example of the present embodiment, state information in the neural network (information such as vector conversion parameters and biases in each layer) is obtained by learning, but the neural network is not necessarily stored in the information providing apparatus 1. Rather, it may be stored on the server side connected via the network. In this case, the state information in the neural network stored in the server is learned from information from a plurality of information providing apparatuses 1 arranged at various places, and learning is performed across users, thereby improving learning efficiency. To do.

  In addition, as in the example here, machine learning is performed using a vector based on the provided information and the time series information of the user's behavior as input, so that appropriate provided information can be selected (at an earlier stage of effective provided information). Possibility of selection) is increased.

[Modification of provided information]
It should be noted that, in the description so far, the provided information template and the plurality of provided information set in advance as the output of the neural network may include information having the same purpose but different expression methods. Good.

For example, even if the offer information is intended to urge you to refrain from taking certain food and drink,
"Let's stop (name of food and drink) today."
"I eat too much (name of food and drink)!"
"Let's stop!"
It is also possible to include different expressions. Furthermore, even if it is the same as a character string, provision information in which the utterance method is different (accents and voices are different) may be included.

[Timing for providing information]
Furthermore, in the examples so far, information was provided when food / drink mode information was generated. However, the present embodiment is not limited to this, and information is provided at a predetermined timing after the end of food / drink. It may be controlled as follows. In the information providing apparatus 1 of this example, for example, information related to provision timing is included in a model of provision information or a plurality of provision information set in advance as an output of a neural network. The provision timing information is set to “immediately”, “after meal”, “morrow morning”, and the information provision processing unit 25 converts the provision information including the provision timing information into the provision information generation unit 24. Accept from. Then, the information provision victory unit 25 provides the accepted provision information to the user at the accepted provision timing.

[Effect of the embodiment]
According to the example of this Embodiment, the information regarding eating and drinking action can be provided in the case of a user's eating and drinking action.

DESCRIPTION OF SYMBOLS 1 Information provision apparatus, 11 Control part, 12 Storage part, 13 Imaging part, 14 Information provision part, 15 Communication part, 21 Image | video reception part, 22 Recognition processing part, 23 Food-drinking aspect information generation part, 24 Provision information generation part, 25 Information provision processing section.

Claims (5)

An imaging means for imaging a scene in which a user eats or drinks at least one food and drink served;
An identification means for identifying the served food and drink;
Based on the video imaged by the imaging means, the generating means for generating the food and beverage mode information relating to the food and beverage mode of each identified food and drink,
Including
An information providing apparatus in which a predetermined information providing process is executed based on the generated eating and drinking mode information during imaging of a eating and drinking scene by the imaging unit. The information providing device according to claim 1,
The predetermined information providing process includes:
An information providing apparatus that is executed based on a difference between the eating and drinking mode information generated by the generating unit before execution of the past information providing process and the eating and drinking mode information generated after execution of the past information providing process. The information providing device according to claim 1 or 2,
The eating and drinking mode information is
The amount of food and drink of each identified food and drink by the user,
Eating and drinking order of each identified food and drink by the user,
Eating and drinking speed of each identified food and drink by the user,
The user's past meal records,
Information related to user attributes,
Information provided in the past,
An information providing apparatus including at least one of the above. The information providing device according to any one of claims 1 to 3,
The information providing process includes:
Based on the nutrition value information of each identified food and drink and the generated food and beverage mode information, the user's intake including at least one of the amount of food and drink for each predetermined nutrition value, the order of eating and drinking or the speed of eating and drinking An information providing apparatus that obtains state information and executes information providing processing based on the intake state information. A computer connected to an imaging means for imaging a scene in which a user eats or drinks at least one food and drink served,
An identification means for identifying the served food and drink;
Based on the video imaged by the imaging means, the generating means for generating the food and beverage mode information relating to the food and beverage mode of each identified food and drink,
Function as
A program for executing a predetermined information providing process based on the generated eating and drinking mode information during imaging of a eating and drinking scene by the imaging means.

Images