WO2019194364A1 - Automobile heating, ventilation, and air conditioning control system using emotion recognition - Google Patents

Abstract

Provided is an automobile heating, ventilation, and air conditioning control system using emotion recognition for: automatically controlling adjustment of heating or air conditioning and a direction and a strength of a wind of an interior room of an automobile, according to emotional information configured by a user; and controlling an additional function provided by the automobile, according to the constitution, biorhythm, health information, or taste of the user.

Description

Automotive air conditioning control system using emotion recognition

The present invention relates to an air conditioning control system for automobiles using emotion recognition, and more particularly, to a vehicle using emotion recognition that automatically changes the direction and intensity of heating and cooling of a car and wind according to emotion information set by a user. It relates to an air conditioning control system.

In general, emotion refers to a feeling or a feeling that is felt about something, a phenomenon, or an object. Sensibility refers to the ability to induce emotion through the mind or senses (vision, hearing, smell, taste, touch). If the peripheral device of the human being, for example, the brightness and color of the light, the volume of the sound source playback device, the genre of the sound source to be reproduced, etc., the emotion is expressed to the outside according to the human emotion.

Emotional engineering has become a very important subject in the industry and economy in recent years. Emotional engineering is a technique of recognizing emotions expressed by human emotion through physical or sensory stimulation. Emotional engineering received a lot of attention from scientists in the late 1990s and early 2000s. Recently, researches to improve the accuracy of biometric technology, which is the basic technology in the field of emotional engineering, by using the technology of detecting human emotional state by recognizing facial expressions and body movements and mathematical techniques.

On the other hand, the air conditioner is a device for controlling the temperature and humidity of the indoor air. An air conditioner is essential for a vehicle to control heating and cooling. The conventional air conditioner operates only at a temperature set by a user, and does not consider the user's emotion at all. In the future, the era of autonomous cars, in which all the functions of the car operate for people, will come. Accordingly, there is an urgent need for an air conditioning control technology for automobiles in consideration of the user's emotion.

The present invention has been proposed in the background as described above, and provides a vehicle air conditioning control system using emotion recognition to automatically control the heating or cooling of the car interior and the direction and intensity of the wind according to the emotion information set by the user. .

In addition, the present invention provides a vehicle air conditioning control system using emotion recognition to automatically control the heating or cooling of the vehicle interior and the direction and intensity of the wind according to the user's constitution, biorhythm, or health information.

Other objects of the present invention will be readily understood through the following description of the embodiments.

In order to achieve the above technical problem, the vehicle air conditioning control system using an emotion recognition according to an aspect of the present invention, is installed around the driver's seat and passenger seat and includes an operation unit, an air conditioner, an internal temperature sensor and the solar radiation sensor and an infrared camera An information collecting unit; Emotion setting information receiving unit for receiving the emotion setting information set by the user from the seat-specific operation unit; A sensor information receiver configured to receive internal temperature and insolation information from an internal temperature sensor for each seat and an insolation sensor for each seat;

A seat-specific emotion recognition unit that recognizes a person's emotion by using an image captured by an infrared camera for each seat and the internal temperature and solar radiation information; An information processing unit which transmits the emotion setting information for each seat inputted from the emotion setting information receiving unit, internal temperature and insolation amount information for each seat to the emotion recognition unit for each seat, and receives emotion recognition information for each seat from the emotion recognition unit for each seat;

A seat-specific air conditioning controller configured to generate air-conditioner control information for each seat using the seat-specific emotion setting information, emotion recognition information, internal temperature, and solar radiation information inputted from the information processor; And a seat-specific device controller for driving the air conditioner or the heater, driving the blower fan, and driving the actuator to change the direction of the wind by receiving the seat-specific air conditioner control information from the seat-specific air conditioning control unit.

According to the present invention, the operation unit includes a mode key, a display, and a controller for emotion setting, wherein the controller is configured to display the emotion setting information and the air conditioner on the emotional display area of the display screen when the MODE key is pressed by the user. Alternatively, the strength display area, the wind intensity display area, and the ventilation display area of the heater may respectively display the strength of the air conditioner or the heater, the wind intensity, and the ventilation information.

According to the present invention, an emotion recognition unit for each seat extracts an iris pattern from an eye image of a person photographed by an infrared camera for each seat and recognizes a passenger using an iris pattern, and an information processing unit identifies passengers and passengers identified by date and time. The emotion setting information, the emotion recognition information for each seat, internal temperature, and solar radiation information are stored in the storage unit.

According to the present invention, the information processor receives the passenger's constitution, biorhythm, or health information from the smartphone possessed by the passenger and transfers it to the emotion recognition unit for each seat or the air conditioner for each seat, and the emotion recognition unit for each seat is a biorhythm. Is used as weight information for emotion recognition, and the air conditioning control unit for each seat determines the strong and weak body parts according to the passenger's constitution, and changes the direction and intensity of the wind.

According to the air conditioning control system for cars using the emotion recognition of the present invention has the following advantages.

First, the driver or the passenger can obtain the effect of emotional satisfaction by forming their own air conditioning environment through the emotion setting information rather than the temperature setting.

Second, the car can increase the human intimacy and comfort of the car by creating an air-conditioning environment suitable for the constitution, biorhythm or health of the driver or passenger.

1 shows a vehicle interior system according to the invention.

2 is an exemplary view for explaining an arrangement of a camera for recognizing user emotion according to the present invention.

3 is an exemplary view for explaining an operation unit for setting a user emotion according to the present invention.

4 is an exemplary view for explaining a configuration of a vehicle air conditioning control system using emotion recognition according to the present invention.

5 is a flowchart illustrating a vehicle air conditioning control method using emotion recognition according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily understand and reproduce the present invention.

As shown in FIG. 1, the entire system according to the present invention includes a display unit 105, an input / output control board unit 110, a trip control unit 115, an input selector 120, an AVM system 125, and a smartphone. 130, DMB / DVI 135, navigation 145, codec 150, communication interface 155, infrared camera 160, temperature sensor 165, solar radiation sensor 170, and air conditioner 175 It may include.

The display unit 105 may include an image display unit displaying an image and a touch screen receiving a touch input. The image display unit may be implemented as a liquid crystal display (LCD) or a head-up display (HUD).

The input / output control board unit 110 inputs / outputs communication, control, and data with a device mounted in a vehicle. The input / output control board unit 110 includes a CVBS communication unit, an MHL / HDMI communication unit, an RGB communication unit, a DVI communication unit, an audio communication unit, a wireless communication unit, and a power supply unit.

The CVBS communication unit is for connecting with an electronic device for performing CVBS communication, such as a camera and an AVM system 125, and has a corresponding connector. The MHL / HDMI communication unit is for performing HML / HDMI communication with the smartphone 130 and has a corresponding connector. The RGB communication unit is in charge of communication with a device that provides image information through an RGB cable. The RGB communication unit may receive a DMB image or a Divx image and transmit a signal for controlling a device that provides the image. The DVI communication unit is in charge of communicating with a device that provides video information through a DVI cable.

The audio communication unit is in charge of audio communication with the electronic device. The audio communication unit has an audio connection connector. The device may be connected to a device providing only an audio signal through the audio communication unit or may receive an audio signal of a device connected through another communication unit. The wireless communication unit is responsible for communication with an external wireless communication device, and it is effective that the wireless communication unit is composed of various modules capable of responding to various types of wireless communication. For example, Bluetooth, infrared communication, RF communication, or Wi-Fi communication are possible.

The power supply unit supplies power to electronic devices connected to each communication unit. The power supplied is the power used for the electronic device to provide the DC power, the power inside the vehicle. Therefore, it is preferable that the power applying unit provides a power source suitable for operating power of the device by controlling voltage and current of various levels.

The trip controller 115 may include an OBD communication unit, a K-Line communication unit, a GPIO communication unit, a navigation communication unit, a touch signal communication unit, an external memory communication unit, a trip communication unit, and a diagnostic USB communication unit. The trip control unit 115 may communicate with devices inside the vehicle, information for driving and driving the vehicle, and devices for checking and diagnosis.

The OBD communication unit is for communication between the internal system for the vehicle inspection and the sensors and the inspection devices, and has a corresponding connector. Through such an OBD communication unit it is possible to know the state of the vehicle. The K-line communication unit may be provided with inputs of various sensors of the diagnosis device and output values of the actuators through can communication. K-line communication unit can communicate with the in-vehicle device through can communication.

The GPIO communication unit is for communication with an external chipset and has a corresponding connector, that is, an input / output terminal. The navigation communication unit is for communication with the navigation mounted on the vehicle, and receives information collected from the navigation, and may provide a signal for navigation control. The navigation communication unit may project the screen of the navigation in a mirroring form. When the navigation is integrated with the AVM in the vehicle, the function of the navigation communication unit may be performed by the input / output control board 110.

The touch signal communication unit receives a coordinate value touched by the touch screen of the display unit 105. The external memory communication unit is connected to an external memory (including an SD card) to store information in the corresponding memory or to read information stored in the memory. The trip communication unit communicates with a trip computer mounted in the vehicle. Through this, information related to driving of the vehicle may be provided. The diagnostic USB communication unit is for USB communication for vehicle diagnosis, and a separate connection terminal is provided therefor. Diagnostics You can also perform vehicle self-test via the USB communication unit.

The input selector 120 selects and transmits at least one signal from an input signal of the electronic device input through the trip controller 115 and the input / output control board 110. The input selector 120 transmits only the selected input signal so that a switch or a mux may be used.

The communication interface 155 includes an OBD interface, a K-Line interface, and a GPIO interface. The infrared camera 160 provides an image image of a human face. The image image captured by the infrared camera 160 is used in an algorithm that can determine the emotional state of a person. The temperature sensor 165 detects an internal temperature and an outside temperature of the vehicle. The solar radiation sensor 170 measures the amount of solar energy radiated for 1 minute in 1 cm 2 area. The air conditioner 175 includes an air conditioner, a heater, a blowing fan, and an actuator. The actuator changes the direction of the wind.

As shown in FIG. 2, the infrared camera 160 includes an infrared camera 161 for photographing an automobile driver, an infrared camera 162 for photographing a passenger seated in an auxiliary seat, an infrared camera 163 for photographing a left passenger, and a right side. Infrared camera 164 for photographing passengers. An inner temperature sensor 1651 to 1654 and an insolation sensor 171 to 174 are installed at the cockpit seat, the auxiliary seat, the left side, and the right side of the vehicle, respectively, to sense the temperature inside the vehicle. An actuator for changing the wind strength and direction of the air conditioner is installed at the cockpit seat, the auxiliary seat, the left side, and the right side of the vehicle, respectively. The manipulator seats, auxiliary seats, left and right sides of the vehicle are provided with operation units 181, 812, 183, and 184 for setting user emotion.

Referring to FIG. 3, the operation unit includes a mode (311) key for setting user emotion, a display, and a controller. When the MODE 311 key is pressed by the user, the controller displays one of warmth, comfort, and coolness on the emotional display area 312 of the display screen. The control unit may be implemented to display the intensity, wind strength, and ventilation information of the air conditioner or heater, respectively, on the intensity display area 313 of the air conditioner or heater, the wind intensity display area 314, and the ventilation display area 315 of the display screen. Can be.

The air conditioning control system for automobiles using the emotion recognition according to the present invention may be implemented to operate according to the strength of the air conditioner or heater and the wind strength which are basically set in advance when the emotion mode set by the user is input. The air conditioning control system for automobiles using the emotion recognition according to the present invention is implemented to determine the emotional state of a person from an image image photographed by an infrared camera and to adjust the strength and wind intensity of the air conditioner or heater which are basically set in advance according to the determination result. Can be.

As shown in FIG. 4, the vehicle air conditioning control system using emotion recognition according to the present invention has a driver information collecting unit 411, a passenger seat information collecting unit 412, a left passenger information collecting unit 413, and a right side. Passenger information collecting unit 414, emotion setting information receiving unit 415, sensor information receiving unit 416, seat emotion recognition unit 417, information processing unit 418, seat-specific air conditioning control unit 419 and seat-specific device controller 420.

The driver information collecting unit 411, the auxiliary passenger information collecting unit 412, the left passenger information collecting unit 413, and the right passenger information collecting unit 414 all use an operation unit, an air conditioner, an internal temperature sensor, an insolation sensor, and an infrared camera. Include.

The emotion setting information receiving unit 415 may receive an emotion mode set by the user, for example, warmth, comfort, or coolness, from the seat-specific operation unit. Of course, the invention is not limited thereto, and the emotional mode may be implemented including comfort, liveliness, and the like. The conventional emotion setting information receiving unit 415 receives the driver setting information on the heater temperature or the air conditioner temperature, but the present invention receives the emotion setting information for each seat.

The sensor information receiving unit 416 receives the internal temperature and the solar radiation information from the internal temperature sensor for each seat and the solar radiation sensor for each seat. Although not shown in FIG. 4, the vehicle may be provided with an outside air temperature sensor for measuring the outside air temperature and a humidity sensor on the windshield of the car. The sensor information receiver 416 may receive outside temperature and humidity information from an outside temperature sensor and a humidity sensor.

The emotion recognition unit 417 for each seat is photographed from an infrared camera installed in the driver information collecting unit 411, the auxiliary passenger information collecting unit 412, the left passenger information collecting unit 413, and the right passenger information collecting unit 414. Receive image images to recognize emotions.

Seat-specific emotion recognition unit 417 measures the degree of fatigue by measuring the state of the eye with an eye image of a person taken with an infrared camera, for example, how often the eye blinks in one minute and fatigue using a change in the pupil Judge. The iris muscles that control the pupil slow down when the eye becomes tired, and this is measured.

The emotion recognition unit 417 for each seat may also recognize an identity using an iris pattern from an eye image of a person photographed by an infrared camera. The emotion recognition unit 417 for each seat may diagnose a person's health through an iris pattern analysis. The seat-specific emotion recognition unit 417 may recognize anxiety of a person through eye tracking following the center of the pupil, for example, moving the pupil up and to the right.

The emotion recognition unit 417 for each seat may recognize a person's emotion by using the internal temperature and insolation information and the outside temperature and humidity information for each seat in addition to the infrared image information for each seat. In general, when the amount of insolation is high, a person's eyes or face are frowned. On the other hand, even when the amount of insolation is high, facial expressions or biometric changes may not occur in some people. The seat-specific emotion recognition unit 417 may be implemented to generate emotion recognition error information using outdoor environment information and to remove the error information, and then recognize a person's emotion. The emotion recognition error information may be generated through, for example, human facial expressions or biometric changes in a situation in which an external shock detection signal is input from a shock sensor mounted in a vehicle.

The information processing unit 418 receives the emotion mode set for each seat from the emotion setting information receiving unit 415, the internal temperature and insolation information for each seat, and the outside temperature and humidity information for each seat from the sensor information receiving unit 416. To pass). The information processor 418 receives the emotion recognition information for each seat from the emotion recognition unit 417 for each seat and transmits the emotion recognition information for each seat 419. The information processor 418 may be implemented to store information input from the emotion setting information receiving unit 415, the sensor information receiving unit 416, and the seat-specific emotion recognition unit 417 in a storage unit.

The information stored in the storage unit by the information processing unit 418 may include identification information for each date and time, an emotion mode set by the passenger, internal temperature and insolation information, and outdoor temperature and humidity information. This information can be used to derive the optimal air control information corresponding to the emotion mode for each passenger through a big data analysis technique. The information processor 418 stores, in the storage, the optimum air conditioner control information corresponding to the emotion mode set by the passenger derived through the big data analysis technique, and when the emotion mode set by the passenger is input, the air conditioner stored in the storage unit. It can be implemented to output the control information.

The air conditioning control unit 419 for each seat generates air conditioner control information for each seat using the emotion mode set by the passenger from the information processor 418, the emotion recognition information for each seat, the internal temperature and the solar radiation information, and the outside temperature and humidity information. The air conditioner control information for each seat includes air conditioner or heater operation control information, information on wind strength and direction, and ventilation information. The air conditioning controller 419 for each seat operates the air conditioner or the heater until the emotion recognition information for each seat is similar to, for example, 90% or more set by the user, and is implemented to control the strength and direction of the wind and the ventilation.

The seat-specific device control unit 420 receives air-conditioner control information for each seat from the seat-specific air conditioning control unit 419 to drive an air conditioner or a heater, to drive a blower fan, and to drive an actuator for changing the direction and strength of the wind.

The information processor 418 may receive the passenger's constitution, biorhythm, or health information from the smart phone possessed by the passenger and transmit the received information to the seat emotion recognition unit 417 or the seat air conditioning controller 419. For example, the emotion recognition unit 417 for each seat may use biorhythm as weight information for emotion recognition. Biorhythms include physical rhythms, emotional rhythms, and intellectual rhythms. The emotion recognition unit 417 for each seat may increase or decrease an emotion index of a person generated by using infrared image information, internal temperature, and solar radiation information for each seat, depending on whether the emotion rhythm is low or high.

Human constitution is divided into eight constitutions based on the strength and weakness of five parts of the heart, pancreas, lung and kidney, and five parts of the gallbladder, stomach, small intestine, large intestine, and bladder. For example, the air conditioning control unit 419 for each seat may be implemented to determine strong and weak body parts according to the passenger's constitution and to change the direction and intensity of the wind. As another example, the air conditioning control unit 419 for each seat may be implemented to change the direction and intensity of the wind by using the health information. For example, if the passenger is a patient usually with a bronchial or nasal ENT, the wind may be directed to the side instead of the passenger's face.

Thus far, the present specification has been described with reference to the embodiments shown in the drawings so that those skilled in the art to which the present invention pertains can easily understand and reproduce the present invention. Those skilled in the art will understand that various modifications and equivalent other embodiments are possible from the embodiments of the present invention. Therefore, the true technical protection scope of the present invention should be defined only by the appended claims.

The invention is applicable to the automotive air conditioning control industry.

Claims (7)

An information collection unit installed around a driver's seat and a passenger seat and including an operation unit, an air conditioner, an internal temperature sensor, an insolation sensor, and an infrared camera; Emotion setting information receiving unit for receiving the emotion setting information set by the user from the seat-specific operation unit; A sensor information receiver configured to receive internal temperature and insolation information from an internal temperature sensor for each seat and an insolation sensor for each seat; A seat-specific emotion recognition unit that recognizes a person's emotion by using an image captured by an infrared camera for each seat and the internal temperature and solar radiation information; An information processing unit which transmits the emotion setting information for each seat inputted from the emotion setting information receiving unit, internal temperature and insolation amount information for each seat to the emotion recognition unit for each seat, and receives emotion recognition information for each seat from the emotion recognition unit for each seat; A seat-specific air conditioning controller configured to generate air-conditioner control information for each seat using the seat-specific emotion setting information, emotion recognition information, internal temperature, and solar radiation information inputted from the information processor; A seat-specific device control unit which receives the air conditioner control information for each seat from the air conditioning control unit for each seat, drives an air conditioner or a heater, drives a blower fan, and drives an actuator for changing the direction and intensity of the wind; Air conditioning control system for cars using emotion recognition comprising a. The method according to claim 1, The operation unit includes a mode key, a display and a control unit for setting emotion, When the MODE key is pressed by the user, the control unit displays the emotion setting information on the emotional display area of the display screen, the strength display area of the air conditioner or heater, the strength display area of the air conditioner, and the wind power display area and the ventilation display area, respectively. Displaying wind strength and ventilation information, Automotive air conditioning control system using the emotion recognition characterized in that. The method according to claim 1, The sensor information receiver, Receives outside air temperature and humidity information from the outside air temperature sensor for measuring the outside air temperature of the vehicle and the humidity sensor installed on the windshield of the car and transmits it to the information processing unit The emotion recognition unit for each seat, Recognizing a person's emotion by using an image image taken by an infrared camera for each seat and the internal temperature and insolation information per seat and outside temperature and humidity information from the information processor, Automotive air conditioning control system using the emotion recognition characterized in that. The method according to claim 1, The emotion recognition unit for each seat extracts an iris pattern from an eye image of a person photographed by an infrared camera for each seat, and recognizes a passenger using an iris pattern, Automotive air conditioning control system using the emotion recognition characterized in that. The method according to claim 4, The information processing unit stores the identification information for each date and time, the emotion setting information set by the passenger, the emotion recognition information for each seat, and the internal temperature and insolation information in a storage unit. Automotive air conditioning control system using the emotion recognition characterized in that. The method according to claim 1, The information processing unit receives a passenger's constitution, biorhythm, or health information from the smartphone possessed by the passenger and transmits the information to the seat emotion recognition unit or the seat air conditioning controller, The emotion recognition unit for each seat is to use the biorhythm as weight information for emotion recognition, Automotive air conditioning control system using the emotion recognition characterized in that. The method according to claim 6, The air conditioning control unit for each seat, to determine the strong and weak body parts according to the passenger's constitution, to change the direction and strength of the wind, Automotive air conditioning control system using the emotion recognition characterized in that.

Images