JP2024076257A - In-vehicle systems - Google Patents

Abstract

To provide an on-vehicle system that is able to improve the accuracy of the prediction of a future emotion of an occupant.SOLUTION: An on-vehicle system according to the present invention incudes: a control device; an in-vehicle sensor that detects a seating position of an occupant present in a vehicle; a spatial information acquisition device that acquires spatial information in the vehicle; a measurement data acquisition device that acquires measurement data of the occupant; and a storage device that stores a learned model for predicting a future emotion of the occupant; wherein the control device is configured such that, using the learned model, the future feeling of the occupant is predicted from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device, and vehicle control is executed in accordance with the predicted future feeling of the occupant.SELECTED DRAWING: Figure 3

Description

The present invention relates to an in-vehicle system.

Patent document 1 proposes a technology that estimates the emotions of passengers based on biometric information obtained from a wearable device.

JP 2018-083583 A

Image data and biometric information directly reflect the occupant's emotions. Image data and biometric information are effective for estimating the occupant's emotions at a given time. However, the characteristics expressed in the image data and biometric information may have little or no relevance to future emotions. For this reason, it is difficult to predict the occupant's future emotions with a high degree of accuracy from the image data and biometric information.

The present invention has been made in consideration of the above problems, and its purpose is to provide an in-vehicle system that can improve the accuracy of predicting the future emotions of occupants.

In order to solve the above-mentioned problems and achieve the object, the in-vehicle system of the present invention is an in-vehicle system including a control device, an in-vehicle sensor that detects the seating position of an occupant present in the vehicle, a spatial information acquisition device that acquires spatial information inside the vehicle, a measurement data acquisition device that acquires measurement data of the occupant, and a storage device that stores a trained model for predicting the future emotions of the occupant, and the control device is configured to predict the future emotions of the occupant using the trained model from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device, and to execute vehicle control according to the predicted future emotions of the occupant.

The in-vehicle system of the present invention can improve the accuracy of predicting the future emotions of occupants.

In addition, in the above, the spatial information is _CO2 concentration information, the vehicle control is air conditioning control that circulates air inside the vehicle or ventilates the vehicle, the trained model is generated by machine learning to derive a predicted result of the occupant's future emotions from the seating position, the _CO2 concentration information, and the measurement data, and predicting the occupant's future emotions using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the _CO2 concentration information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to obtain a predicted result of the occupant's future emotions from the trained model by executing calculation processing of the trained model.

This makes it possible to predict the future emotions of the occupants based on the _CO2 concentration information inside the vehicle.

In the above, the spatial information is odor information, the vehicle control is air conditioning control that circulates air inside the vehicle or ventilates the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotions of the occupant from the seating position, the odor information, and the measurement data, and predicting the future emotions of the occupant using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the odor information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to obtain a result of predicting the future emotions of the occupant from the trained model by executing a calculation process of the trained model.

This makes it possible to predict the future emotions of occupants based on odor information inside the vehicle.

In the above, the spatial information is temperature information, the vehicle control is air conditioning control for adjusting the temperature inside the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotions of the occupant from the seating position, the temperature information, and the measurement data, and predicting the future emotions of the occupant using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the temperature information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to obtain a result of predicting the future emotions of the occupant from the trained model by executing a calculation process of the trained model.

This makes it possible to predict the future emotions of occupants based on temperature information inside the vehicle.

In the above, the spatial information is sunlight information, the vehicle control is sunlight control that adjusts the sunlight inside the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotions of the occupant from the seating position, the sunlight information, and the measurement data, and predicting the future emotions of the occupant using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the sunlight information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to obtain a result of predicting the future emotions of the occupant from the trained model by executing a calculation process of the trained model.

This makes it possible to predict the future emotions of occupants based on solar radiation information inside the vehicle.

The in-vehicle system according to the present invention has the effect of improving the accuracy of predicting the future emotions of occupants.

1 is a diagram showing an outline of a vehicle equipped with an in-vehicle system according to a first embodiment; 1 is a block diagram showing a schematic configuration of an in-vehicle system 2 according to a first embodiment. 4 is a flowchart showing an example of control performed by a control device in the in-vehicle system according to the first embodiment. FIG. 11 is a block diagram showing a schematic configuration of an in-vehicle system according to a second embodiment. 10 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a second embodiment. FIG. 11 is a block diagram showing a schematic configuration of an in-vehicle system according to a third embodiment. 13 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a third embodiment. FIG. 11 is a block diagram showing a schematic configuration of an in-vehicle system according to a fourth embodiment. 13 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a fourth embodiment.

(Embodiment 1)
Hereinafter, a first embodiment of an in-vehicle system according to the present invention will be described. Note that the present invention is not limited to this embodiment.

Figure 1 is a diagram showing an overview of a vehicle 1 equipped with an in-vehicle system 2 according to embodiment 1. Figure 2 is a block diagram showing a schematic configuration of the in-vehicle system 2 according to embodiment 1.

As shown in FIG. 1, the vehicle 1 according to the embodiment includes an in-vehicle system 2, a steering wheel 4, front seats 31 and 32, and a rear seat 33. Note that the arrow A in FIG. 1 indicates the traveling direction of the vehicle 1.

Passengers 10A, 10B, and 10C sit in the front seats 31, 32 and the rear seat 33, respectively. Passenger 10A, who sits in the front seat 31 opposite the steering wheel 4, is the driver of the vehicle 1. In the following description, when there is no particular distinction between passengers 10A, 10B, and 10C, they will simply be referred to as passenger 10.

The in-vehicle system 2 is composed of a control device 21, a storage device 22, an in-vehicle camera 23, and an operation panel 24.

The control device 21 is configured, for example, by an integrated circuit including a CPU (Central Processing Unit). The control device 21 is communicatively connected to the storage device 22, the in-vehicle camera 23, and the operation panel 24. The control device 21 executes programs and the like stored in the storage device 22. The control device 21 also acquires image data, for example, from the in-vehicle camera 23.

The storage device 22 includes at least one of, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), and an HDD (Hard Disk Drive). The storage device 30 does not need to be a single physical element, and may include multiple elements that are physically separated from one another. The storage device 22 stores programs executed by the control device 21, etc. The storage device 22 also stores various data used when executing programs such as a trained model for predicting the future emotions of the occupant 10 and a trained model for vehicle control, which will be described later.

As shown in FIG. 1, the in-vehicle camera 23 is an imaging device arranged in a position inside the vehicle where it can capture an image of multiple occupants 10A, 10B, 10C. The in-vehicle camera 23 functions as an in-vehicle sensor for detecting a target occupant 10 whose emotion is to be predicted from the multiple occupants 10A, 10B, 10C inside the vehicle, and outputs image data as sensor data. The image data captured by the in-vehicle camera 23 is transmitted to the control device 21 and temporarily stored in the storage device 22.

The operation panel 24 is an input/output device such as a touch panel display located near the driver's seat, which accepts operation instructions from an occupant 10 such as a driver and provides information to the occupant 10.

The _CO2 concentration information acquisition device 25 includes a _CO2 concentration sensor, an infrared sensor, etc., and acquires information about the _CO2 concentration inside the vehicle. In addition, the _CO2 concentration information acquisition device 25 can acquire information about the CO2 concentration in the vicinity of the seating position of the occupant 10 in the front seats 31, 32 and the rear seat 33 by using vehicle structure information that is information specific to the vehicle ₁ .

The air conditioner 5 includes an air compressor and a heater, and adjusts the temperature inside the vehicle by operating the air compressor or the heater according to a control signal from the control device 21. For example, when the air conditioner 5 is in an operating state, the control device 21 controls the air conditioner 5 so that the temperature inside the vehicle becomes a target temperature. The target temperature may be a temperature set in advance or a temperature set by the most recent user. The air conditioner 5 is configured to be able to introduce outside air into the vehicle, and is capable of ventilating the vehicle by introducing outside air into the vehicle. The air conditioner 5 is also able to perform air conditioning control, such as blowing air so as to prioritize and optimize the _CO2 concentration at a specific location inside the vehicle, using vehicle structure information, which is information unique to the vehicle 1.

The control device 21 can detect, for example, the attributes and facial expressions of the occupant 10, the seating position (riding position) of the occupant 10, etc., based on the image data captured by the in-vehicle camera 23. That is, the control device 21 can determine, based on the image data, the attributes and facial expressions of the occupant 10, the seating position (riding position) of the occupant 10, etc., by AI (Artificial Intelligence) using a learned model that has been machine-learned to learn the attributes and facial expressions of the occupant 10, the seating position (riding position) of the occupant 10, etc. In addition, the control device ₂₁ can determine, based on the image data, a prediction of the future emotions of the occupant 10 by AI using a learned model that has been machine-learned to learn the seating position of the occupant 10, CO2 concentration information, and measurement data of the occupant 10, etc. In addition, the measurement data of the occupant 10 includes, for example, the facial expressions of the occupant 10 captured by the in-vehicle camera 23, and biological information such as heart rate and blood pressure detected by a wearable device worn by the occupant 10. Furthermore, the control device 21 is capable of using AI to determine the content of vehicle control based on predictions of the future emotions of the occupant 10 using a machine-learned model.

The trained model for predicting future emotions is trained to output emotion estimation results from input data by supervised learning, for example, according to a neural network model. The trained model for predicting future emotions is generated by repeatedly executing a learning process using a training data set that is a combination of input and result data. The training data set in the trained model for predicting future emotions includes a plurality of training data in which input data such as the seating position of the occupant 10, _CO2 concentration information, and measurement data of the occupant 10 are labeled with a future emotion prediction of the occupant 10, which is the output. The labeling of the future emotion prediction for the input data is performed, for example, by a person skilled in the art. In this way, the trained model for predicting future emotions trained using the training data set receives input data, and outputs a future emotion prediction of the occupant 10 by executing a calculation process of the trained model.

The trained model for vehicle control is trained to output the result of the content of vehicle control from input data by supervised learning according to, for example, a neural network model. The training data set in the trained model for vehicle control includes a plurality of training data in which the content of vehicle control to be output is labeled for input data such as the seating position of the occupant 10, _CO2 concentration information, and future emotion prediction of the occupant 10 given as input. The labeling of the content of vehicle control for the input data is performed, for example, by a person skilled in the art. In this way, the trained model for vehicle control trained using the training data set outputs the content of vehicle control when it receives input data. As the content of vehicle control, for example, when it is predicted that the occupant 10 will become uncomfortable in the future based on the _CO2 concentration information, air conditioning control such as air circulation and ventilation is performed.

In addition, the control device 21 is not limited to determining the content of vehicle control using a trained model for vehicle control when determining the content of vehicle control. For example, the control device 21 may determine the content of vehicle control from _CO2 concentration information and a prediction of the future emotion of the occupant 10 based on a rule that associates the _CO2 concentration information in the vehicle and the prediction of the future emotion of the occupant 10 with the content of vehicle control.

The control device 21 identifies the seating position (riding position) of the occupant whose emotion is to be predicted based on the detection results of the in-vehicle sensors such as the in-vehicle camera 23 and the seating sensor. That is, the control device 21 detects the seating position of the target occupant by the seating sensors provided in the front seats 31, 32 and the rear seat 33, respectively, or by image processing of image data captured by the in-vehicle camera 3. In addition, in the in-vehicle system 2 according to the first embodiment, the seat positions in the vehicle are displayed on the display of the operation panel 24, and the seat position where the target occupant is sitting may be specified by the occupant 10, such as the driver.

In the in-vehicle system 2 according to the first embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on the _CO2 concentration information inside the vehicle, the control device 21 performs air conditioning control such as air circulation and ventilation using the air conditioner 5. Note that, as the vehicle control, the control device 21 may execute control to open an openable window provided in the vehicle 1, instead of the air conditioning control by the air conditioner 5, to ventilate the interior of the vehicle.

Figure 3 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the first embodiment.

First, in step S1, the control device 21 identifies the seating position of the target occupant based on the detection result of the in-vehicle sensor. Next, in step S2, the control device 21 acquires _CO2 concentration information, which is spatial information inside the vehicle. Next, in step S3, the control device 21 acquires measurement data of the target occupant, such as facial expression, from image data of the in-vehicle camera 23. Next, in step S4, the control device 21 determines whether or not the target occupant's emotional deterioration is predicted based on the seating position of the target occupant, the _CO2 concentration information inside the vehicle, and the measurement data of the target occupant. If the control device 21 determines that the target occupant's emotional deterioration is not predicted, the control device 21 determines No in step S4 and ends a series of controls without changing the in-vehicle environment. If the control device 21 determines that the target occupant's emotional deterioration is predicted, the control device 21 determines Yes in step S4 and proceeds to step S5. In step S5, the control device 21 predicts the future emotions of the target occupant from the seating position of the target occupant, _CO2 concentration information in the vehicle interior, and the measurement data of the target occupant using the trained model for predicting future emotions. Next, in step S6, the control device 21 determines the content of air conditioning control such as air circulation and ventilation in the vehicle from the seating position of the target occupant, _CO2 concentration information in the vehicle interior, and the predicted future emotions of the target occupant using the trained model for vehicle control. Next, in step S7, the control device 21 executes air conditioning control based on the determined content of air conditioning control.

As a result, in the in-vehicle system 2 according to the first embodiment, it is possible to prevent the target occupant from becoming uncomfortable in the future due to a deterioration of the in-vehicle environment, such as an increase in the _CO2 concentration in the vehicle near the seating position of the target occupant.

(Embodiment 2)
Hereinafter, a second embodiment of the in-vehicle system according to the present invention will be described. Note that the description of the second embodiment common to the first embodiment will be omitted as appropriate.

Figure 4 is a block diagram showing the schematic configuration of an in-vehicle system 2 according to the second embodiment.

In the in-vehicle system 2 according to the second embodiment, the spatial information inside the vehicle is odor information inside the vehicle, and the vehicle is provided with an odor information acquisition device 26 for acquiring the odor information inside the vehicle. The odor information acquisition device 26 includes, for example, an odor sensor, and acquires odor information inside the vehicle. The odor information includes, for example, odor information of cosmetics and odor information of food. In addition, the odor information acquisition device 26 can acquire odor information near the seating position of the occupant 10 in the front seats 31, 32 and the rear seat 33 by using vehicle structure information, which is information unique to the vehicle 1.

The control device 21 predicts the future emotions of the occupant 10 using a trained model for predicting future emotions based on the seating position of the occupant 10, odor information inside the vehicle, and measurement data of the occupant 10. If it is predicted that the occupant 10 will become uncomfortable in the future, the control device 21 uses the trained model for vehicle control to perform air conditioning control such as air circulation and ventilation inside the vehicle.

The learning data set in the trained model for predicting future emotions includes multiple pieces of learning data that are labeled with a prediction of the occupant's 10's future emotions as output for input data such as the seating position of the occupant 10, odor information, and measurement data of the occupant 10.

The learning dataset in the trained model for vehicle control includes multiple pieces of learning data in which the vehicle control content that is the output is labeled for input data such as the seating position of the occupant 10, odor information inside the vehicle, and predictions of the occupant 10's future emotions. The vehicle control content, for example, is to perform air conditioning control such as air circulation and ventilation when it is predicted that the occupant 10 will become uncomfortable in the future based on odor information inside the vehicle.

In the in-vehicle system 2 according to the second embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on odor information inside the vehicle, the control device 21 performs air conditioning control such as air circulation and ventilation using the air conditioning device 5. The air conditioning device 5 can perform air conditioning control such as blowing air to prioritize and optimize the odor at a specific location inside the vehicle using vehicle structure information, which is information unique to the vehicle 1. Furthermore, the control device 21 may control the odor inside the vehicle by executing opening and closing control of an openable window provided in the vehicle 1 instead of the air conditioning control by the air conditioning device 5 as the vehicle control.

Figure 5 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the second embodiment.

First, in step S11, the control device 21 identifies the seating position of the target occupant 10, whose future emotions are predicted, based on the detection results of the in-vehicle sensors. Next, in step S12, the control device 21 acquires odor information, which is spatial information within the vehicle. Next, in step S13, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the in-vehicle camera 23. Next, in step S14, the control device 21 determines whether or not the target occupant's emotional deterioration is predicted based on the seating position of the target occupant, the odor information within the vehicle, and the measurement data of the target occupant. If the control device 21 determines that the target occupant's emotional deterioration is not predicted, the control device 21 determines No in step S14 and ends the series of controls without changing the in-vehicle environment. On the other hand, if the control device 21 determines that the target occupant's emotional deterioration is predicted, the control device 21 determines Yes in step S14 and proceeds to step S15. In step S15, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant from the seating position of the target occupant, odor information in the vehicle interior, and the measurement data of the target occupant. Next, in step S16, the control device 21 uses a trained model for vehicle control to determine the content of air conditioning control, such as air circulation and ventilation in the vehicle, from the seating position of the target occupant, odor information in the vehicle interior, and the predicted future emotions of the target occupant. Next, in step S17, the control device 21 executes air conditioning control based on the determined content of air conditioning control.

As a result, in the in-vehicle system 2 according to the second embodiment, it is possible to prevent the target occupant from becoming uncomfortable in the future due to a deterioration in the in-vehicle environment, such as an increase in the intensity of odors in the vicinity of the target occupant's seating position.

(Embodiment 3)
Hereinafter, an in-vehicle system according to a third embodiment of the present invention will be described. Note that the description of the third embodiment common to the first embodiment will be omitted as appropriate.

Figure 6 is a block diagram showing the schematic configuration of an in-vehicle system 2 according to embodiment 3.

In the in-vehicle system 2 according to the third embodiment, the spatial information within the vehicle is temperature information within the vehicle, and the system is provided with a temperature information acquisition device 27 for acquiring the temperature information within the vehicle. The temperature information acquisition device 27 includes, for example, a temperature sensor, and acquires temperature information within the vehicle, such as the temperature distribution (air temperature distribution) within the vehicle. In addition, the temperature information acquisition device 27 can acquire temperature information around the seating position of the occupant 10 in the front seats 31, 32 and the rear seat 33, using vehicle structure information, which is information specific to the vehicle 1.

The control device 21 predicts the future emotions of the occupant 10 using a trained model for predicting future emotions based on the seating position of the occupant 10, temperature information inside the vehicle, and measurement data of the occupant 10. If it is predicted that the occupant 10 will become uncomfortable in the future, the control device 21 uses a trained model for vehicle control to control the temperature inside the vehicle by turning the air conditioner 5 on and off, changing the set temperature of the air conditioner 5, and the like. The control device 21 may also control the temperature inside the vehicle by turning the seat coolers provided in the front seats 31, 32 and the rear seat 33 on and off, setting the temperature of the seat coolers, and the like.

The learning data set in the trained model for predicting future emotions includes multiple pieces of learning data that are labeled with a prediction of the future emotions of the occupant 10 as output for input data such as the seating position of the occupant 10, temperature information inside the vehicle, and measurement data of the occupant 10.

The learning dataset in the trained model for vehicle control includes multiple pieces of learning data in which the vehicle control content that is the output is labeled for input data such as the seating position of the occupant 10, temperature information inside the vehicle, and a prediction of the future emotions of the occupant 10. The vehicle control content, for example, is to control the temperature inside the vehicle when it is predicted that the occupant 10 will become uncomfortable in the future based on the seating position of the occupant 10, temperature information inside the vehicle, and a prediction of the future emotions of the occupant 10.

In the in-vehicle system 2 according to the third embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on temperature information inside the vehicle, the control device 21 performs temperature control to adjust the temperature inside the vehicle using the air conditioner 5. The air conditioner 5 can perform air conditioning control such as blowing air so as to prioritize and optimize the temperature at a specific location inside the vehicle using vehicle structure information, which is information unique to the vehicle 1. The control device 21 may also adjust the temperature inside the vehicle by performing control to open and close an openable window provided in the vehicle 1, instead of temperature control by the air conditioner 5, as the vehicle control.

Figure 7 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the third embodiment.

First, in step S21, the control device 21 identifies the seating position of the target occupant, who is the occupant 10 for whom future emotions are predicted, based on the detection results of the in-vehicle sensors. Next, in step S22, the control device 21 acquires temperature information inside the vehicle, which is spatial information inside the vehicle. Next, in step S23, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the in-vehicle camera 23. Next, in step S24, the control device 21 determines whether or not the target occupant's emotional deterioration is predicted based on the seating position of the target occupant, the temperature information inside the vehicle, and the measurement data of the target occupant. If the control device 21 determines that the target occupant's emotional deterioration is not predicted, the control device 21 determines No in step S24 and ends the series of controls without changing the in-vehicle environment. On the other hand, if the control device 21 determines that the target occupant's emotional deterioration is predicted, the control device 21 determines Yes in step S24 and proceeds to step S25. In step S25, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant from the seating position of the target occupant, temperature information inside the vehicle, and the measurement data of the target occupant. Next, in step S26, the control device 21 uses a trained model for vehicle control to determine the content of temperature control for adjusting the temperature inside the vehicle from the seating position of the target occupant, temperature information inside the vehicle, and the predicted future emotions of the occupant 10. Next, in step S27, the control device 21 executes temperature control based on the determined content of temperature control.

As a result, in the in-vehicle system 2 according to the third embodiment, it is possible to prevent the target occupant from becoming uncomfortable in the future due to a deterioration in the in-vehicle environment, such as the temperature inside the vehicle becoming too high or too low near the seating position of the target occupant.

(Embodiment 4)
Hereinafter, an in-vehicle system according to a fourth embodiment of the present invention will be described. Note that the description of the fourth embodiment common to the first embodiment will be omitted as appropriate.

Figure 8 is a block diagram showing the schematic configuration of an in-vehicle system 2 according to embodiment 4.

In the in-vehicle system 2 according to the fourth embodiment, the spatial information inside the vehicle is solar radiation information inside the vehicle, and the vehicle is provided with a solar radiation information acquisition device 28 for acquiring the solar radiation information inside the vehicle. The solar radiation information acquisition device 28 includes, for example, an illuminance sensor, and acquires solar radiation information (amount of solar radiation and direction of sunlight) for the entire vehicle interior. The solar radiation information includes, for example, information on the amount of solar radiation and information on the direction of sunlight entering the vehicle interior. The solar radiation information acquisition device 28 can acquire solar radiation information for each of the front seats 31, 32 and the rear seat 33 near the seating position of the occupant 10 using vehicle structure information, which is information specific to the vehicle 1. The solar radiation information acquisition device 28 may also estimate and acquire current and future solar radiation information inside the vehicle from the moving direction and vehicle position of the vehicle 1 using GPS information and weather information received by the car navigation system.

The control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant based on the seating position of the target occupant, future solar radiation information inside the vehicle, and the measurement data of the target occupant. If it is predicted that the occupant will become uncomfortable in the future, the trained model for vehicle control is used to perform solar radiation control by the solar radiation control device 6 as vehicle control. Examples of solar radiation control include adjusting the visible light transmittance of the window glass on the rear seat 33 side, raising and lowering the sunshade of the window glass on the rear seat 33 side, and automatically operating the sun visors on the front seats 31 and 32 sides.

The learning dataset in the trained model for predicting future emotions includes multiple pieces of learning data that are labeled with a prediction of the future emotions of the occupant 10 as output for input data such as the seating position of the target occupant, future solar radiation information inside the vehicle, and measurement data of the occupant 10.

The learning dataset in the trained model for vehicle control includes multiple pieces of learning data in which the vehicle control content that is output is labeled for input data such as the seating position of the target occupant, future solar radiation information inside the vehicle, and future emotion predictions of the occupant 10. The vehicle control content, for example, is to control solar radiation inside the vehicle by the solar radiation control device 6 when it is predicted that the occupant 10 will become uncomfortable in the future based on the seating position of the target occupant, future solar radiation information inside the vehicle, and future emotion predictions of the occupant 10.

Figure 9 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the fourth embodiment.

First, in step S31, the control device 21 identifies the seating position of the target occupant, who is the occupant 10 for whom future emotions are predicted, based on the detection results of the in-vehicle sensors. Next, in step S32, the control device 21 acquires solar radiation information in the vehicle, which is spatial information in the vehicle. Next, in step S33, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the in-vehicle camera 23. Next, in step S34, the control device 21 determines whether or not the target occupant's emotional deterioration is predicted based on the seating position of the target occupant, the solar radiation information in the vehicle, and the measurement data of the target occupant. If the control device 21 determines that the target occupant's emotional deterioration is not predicted, the control device 21 determines No in step S34 and ends the series of controls without changing the in-vehicle environment. On the other hand, if the control device 21 determines that the target occupant's emotional deterioration is predicted, the control device 21 determines Yes in step S34 and proceeds to step S35. In step S35, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the occupant 10 based on the seating position of the target occupant, solar radiation information inside the vehicle, and the measurement data of the target occupant. Next, in step S36, the control device 21 uses a trained model for vehicle control to determine the content of solar radiation control for adjusting the solar radiation into the vehicle based on the seating position of the target occupant, solar radiation information inside the vehicle, and the predicted future emotions of the target occupant. Next, in step S37, the control device 21 executes solar radiation control based on the determined content of solar radiation control.

As a result, in the in-vehicle system 2 according to the fourth embodiment, it is possible to prevent the target occupant from becoming uncomfortable in the future due to a deterioration in the in-vehicle environment caused by the future amount of solar radiation or sunlight in the vicinity of the seating position of the target occupant.

As described above, in the in-vehicle system 2 according to each embodiment, the state of the space in which the occupant 10 is seated (e.g., _CO2 concentration, temperature, odor, and solar radiation) can be grasped based on the spatial information in the vehicle and the seating position of the occupant 10. In addition, the in-vehicle system 2 can predict the future state of the vehicle interior space based on the vehicle structure information and the spatial situation information. The state of the vehicle interior space can be gradually reflected in the emotions of the occupant 10. For example, if an inappropriate temperature state continues, the emotions of the occupant 10 will change to an uncomfortable one. Therefore, by using the spatial information in the vehicle and the information on the seating position of the occupant 10 as input data (explanatory variables) for machine learning, the accuracy of predicting the future emotions of the occupant 10 can be improved.

Reference Signs List 1 Vehicle 2 In-vehicle system 4 Steering wheel 5 Air conditioning device 6 Solar radiation control device 10, 10A, 10B, 10C Occupant 21 Control device 22 Storage device 23 In-vehicle camera 24 Operation panel 25 _CO2 concentration information acquisition device 26 Smell information acquisition device 27 Temperature information acquisition device 28 Solar radiation information acquisition device 31, 32 Front seats 33 Rear seats

Claims (5)

A control device;
An in-vehicle sensor for detecting the seating positions of occupants present in the vehicle;
A spatial information acquisition device for acquiring spatial information inside a vehicle;
a measurement data acquisition device for acquiring measurement data of the occupant;
A storage device that stores a trained model for predicting future emotions of the occupant;
An in-vehicle system comprising:
The control device includes:
Using the trained model, predicting future emotions of the occupant from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device; and
Executing vehicle control in response to the predicted future emotion of the occupant.
It is configured as follows:
In-vehicle systems. The spatial information is _CO2 concentration information,
The vehicle control is an air conditioning control for circulating air inside the vehicle or ventilating the vehicle,
The trained model is generated by machine learning to derive a result of predicting the future emotions of the occupant from the seating position, the _CO2 concentration information, and the measurement data,
Predicting future emotions of the occupant using the trained model
The seating position detected by the in-vehicle sensor, the CO ₂ concentration information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and
Executing a calculation process of the trained model to obtain a prediction result of the future emotion of the occupant from the trained model;
Consists of:
The in-vehicle system according to claim 1 . The spatial information is odor information,
The vehicle control is an air conditioning control for circulating air inside the vehicle or ventilating the vehicle,
The trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the odor information, and the measurement data,
Predicting future emotions of the occupant using the trained model
The seating position detected by the in-vehicle sensor, the odor information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and
Executing a calculation process of the trained model to obtain a prediction result of the future emotion of the occupant from the trained model;
Consists of:
The in-vehicle system according to claim 1 . the spatial information is temperature information,
The vehicle control is an air conditioning control for adjusting a temperature inside the vehicle,
The trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the temperature information, and the measurement data,
Predicting future emotions of the occupant using the trained model
The seating position detected by the in-vehicle sensor, the temperature information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and
Executing a calculation process of the trained model to obtain a prediction result of the future emotion of the occupant from the trained model;
Consists of:
The in-vehicle system according to claim 1 . The spatial information is solar radiation information,
The vehicle control is a solar radiation control for adjusting solar radiation inside the vehicle,
The trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the solar radiation information, and the measurement data,
Predicting future emotions of the occupant using the trained model
The seating position detected by the in-vehicle sensor, the solar radiation information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and
Executing a calculation process of the trained model to obtain a prediction result of the future emotion of the occupant from the trained model;
Consists of:
The in-vehicle system according to claim 1 .

Images