JP2018118671A - Occupant monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an occupant monitoring device capable of appropriately monitoring a occupant's riding condition of a vehicle. SOLUTION: An occupant monitoring device 1 includes an infrared camera 10 that shoots from the front of a seat provided in a vehicle toward the seat, and a material that is provided so that an occupant seated on the seat can wear and reflects infrared rays. An occupant's seatbelt based on the formed seatbelt 20, the specific portion 30 that identifies the seatbelt 20 included in the image captured by the infrared camera 10, and the seatbelt 20 that is identified by the specific portion 30. An estimation unit 40 that estimates the wearing state of the 20 is provided, and the riding state of the occupant is monitored by the mounting state estimated by the estimation unit 40. [Selection diagram] Fig. 1

Description

  The present invention relates to an occupant monitoring device that monitors a boarding state of an occupant of a vehicle.

  Conventionally, techniques for detecting the situation in the passenger compartment, specifically the situation of the vehicle occupant have been studied. As this type of technology, for example, there are those described in Patent Documents 1 and 2 shown below.

  The detection system described in Patent Literature 1 detects the temperature distribution of a space including a vehicle seat by an infrared array sensor, and determines whether the occupant is sleeping based on the detection result. If it is determined that the user is sleeping, an icon indicating that the occupant is sleeping is displayed on the display. Moreover, based on the detection result of the infrared array sensor, an icon for controlling an air conditioner installed in the vicinity of the passenger is displayed on the display. Furthermore, based on the detection result of the infrared array sensor, the presence / absence of the occupant's seat belt is detected, and if the occupant is not wearing the seat belt, the user may be notified that the seat belt is not worn. To do.

  The passenger sitting posture detecting / determining device described in Patent Document 2 detects the passenger's sitting posture using a camera that captures the passenger's face and a seat belt wearing sensor. Specifically, the face image area extracted from the image data acquired by the camera is compared with the face image area in the previously acquired image data while the passenger sitting on the seat is wearing the seat belt. If the direction of movement or the amount of movement exceeds the allowable range, it is determined whether or not it is dangerous. The passenger seating posture detection / determination device described in Patent Document 2 detects a situation in which the passenger suddenly stands up from the seat or faces out of the window.

Japanese Patent Laying-Open No. 2015-96413 JP 2008-199515 A

  For example, the temperature inside the vehicle rises considerably during the daytime in summer, etc., and in the configuration in which the device is controlled by the temperature distribution as in the technique described in Patent Document 1, the influence of disturbance such as temperature change in the vehicle is affected. May lead to malfunction. Moreover, in the technique described in Patent Document 2, it is limited to detecting the passenger's situation, and it is not assumed that the behavior of the driver is detected. In addition, the seat belt is detected by the engagement between the tongue and the buckle. For example, the occupant does not wear the seat belt, and the seat belt is placed between the back of the occupant and the seat (the back of the seat). Even when the tongue and the buckle are engaged, the seat belt mounting sensor determines that the seat belt is correctly mounted. As described above, the techniques described in Patent Documents 1 and 2 cannot properly determine the boarding status of the vehicle occupant, and thus cannot properly monitor the occupant.

  Therefore, an occupant monitoring device capable of appropriately monitoring the boarding situation of a vehicle occupant is required.

  A characteristic configuration of an occupant monitoring device according to the present invention is an infrared camera that captures images from the front of a seat provided in a vehicle toward the seat, and a material that is provided so that an occupant seated on the seat can be mounted and reflects infrared rays. Based on the form of the seat belt specified by the specifying unit, the specifying unit for specifying the seat belt included in the captured image acquired by the infrared camera, and the seat belt specified by the specifying unit An estimator that estimates the seat belt wearing state, and monitors the occupant's riding state based on the wearing state estimated by the estimating unit.

  With such a characteristic configuration, a material that easily reflects infrared rays is added to the seat belt, and the seat belt is detected by the infrared camera, so that it can be detected without being affected by disturbance. Further, it is possible to project the seat belt and the occupant with an infrared camera and determine whether or not the occupant is wearing the seat belt appropriately. Therefore, it is possible to recognize the actual position and posture of the occupant and appropriately monitor or warn about the riding situation of the occupant of the vehicle. Further, by specifying the seat belt, it becomes possible to prevent a violation of the duty of wearing the seat belt.

  Further, the seat belt is provided with a pattern that can be discriminated at the time of photographing with the infrared camera using the material, and the specifying unit specifies the seat belt based on the form of the pattern included in the captured image, Preferably, the estimation unit estimates the wearing state based on the pattern form in the captured image when the occupant wears the seat belt acquired in advance and the pattern form specified by the specifying unit. It is.

  In this way, by using a material that easily reflects infrared rays, a characteristic pattern (for example, a pattern given at a predetermined interval) is formed on the seat belt, so that the change in the position and posture of the occupant It is possible to detect the position and posture of the occupant with high accuracy by checking both the amount of withdrawal and the change in the interval.

  Further, the infrared camera acquires the captured image at predetermined time intervals, and the estimation unit is based on a time-series change in the form of the seat belt in the plurality of captured images acquired by the infrared camera. It is preferable to estimate the wearing state.

  With such a configuration, it is easy to estimate changes in the position and posture of the occupant based on time-series changes in the position of the seat belt.

  In addition, it is preferable that the seat belt is provided with a biosensor that detects at least one of heartbeat and respiration, and the estimation unit estimates the wearing state based on a detection result of the biosensor.

  With such a configuration, it is possible to estimate the wearing state of the seat belt of the occupant and the position and posture of the occupant with high accuracy. It is also possible to detect the physical condition of the occupant.

It is a block diagram which shows typically the structure of a passenger | crew monitoring apparatus. It is the figure which showed an example of the captured image compared with an estimation part. It is a flowchart which shows a process by a passenger | crew monitoring apparatus.

  The occupant monitoring apparatus according to the present invention has a function of monitoring the occupant's boarding state. Hereinafter, the passenger | crew monitoring apparatus 1 of this embodiment is demonstrated.

  FIG. 1 is a block diagram schematically showing the configuration of an occupant monitoring device 1 according to this embodiment. As shown in FIG. 1, the occupant monitoring device 1 includes the functional units of an infrared camera 10, a seat belt 20, a specification unit 30, an estimation unit 40, a storage unit 50, a face recognition unit 60, and a notification unit 70. In particular, each of the functional units of the specifying unit 30, the estimating unit 40, the storage unit 50, the face recognizing unit 60, and the notifying unit 70 is a hardware that uses a CPU as a core member in order to perform processing related to the monitoring of the passenger's boarding state Or built in software or both.

  The infrared camera 10 captures images from the front of the seat provided in the vehicle toward the seat. The infrared camera 10 is configured to be able to discriminate what appears in the acquired captured image even when the amount of light is small (at night or in backlight). The detailed explanation is omitted. In addition, the infrared camera 10 used in the occupant monitoring apparatus 1 can use an infrared camera conventionally provided in a vehicle. In the present embodiment, when a person is seated on a vehicle seat, the infrared camera 10 includes at least a photographing range and a seat belt 20 (to be described later) appropriately worn and a seat belt 20 (described later) included in the captured image. The shooting direction is set.

  The seat belt 20 is provided so that an occupant seated on the seat can be mounted. The seat belt 20 is provided so that a person sitting on a seat within the photographing range of the infrared camera 10 can be worn. Therefore, an occupant seated on a seat is a subject to be photographed by the infrared camera 10 when a person is seated on the seat.

  The seat belt 20 is formed including a material that reflects infrared rays. The material that reflects infrared light is not limited to a material having a reflectivity of 100%, and is preferably a material having a high reflectivity of infrared light (for example, about 90% or more). Specifically, for example, gold, silver, copper, aluminum and the like correspond. The seat belt 20 is formed including such a material that easily reflects infrared rays. With such a configuration, it is possible to easily recognize the seat belt 20 using the light of the infrared LED mounted on the infrared camera 10. The material having a high infrared reflectance may be a material other than gold, silver, copper, or aluminum.

  The seat belt 20 is provided with a pattern that can be discriminated at the time of photographing by the infrared camera 10 using a material that reflects infrared rays. The distinguishable pattern is a pattern in which the seat belt 20 itself and the extending direction can be recognized in the captured image acquired by the infrared camera 10. Specifically, for example, a pattern provided at a predetermined interval (uniform interval) along a direction orthogonal to the extending direction of the seat belt 20 (width direction of the seat belt 20) is preferable. Of course, it is also possible to apply a pattern at a predetermined interval along the extending direction of the seat belt 20.

  The identifying unit 30 identifies the seat belt 20 included in the captured image acquired by the infrared camera 10. A captured image acquired by the infrared camera 10 is transmitted to the specifying unit 30. As described above, the seat belt 20 is patterned using a material that reflects infrared rays. Therefore, the specifying unit 30 specifies the seat belt 20 based on the pattern form included in the captured image. The pattern form is a pattern form formed of a material that reflects infrared rays. For example, when a pattern is given at a predetermined interval along a direction orthogonal to the extending direction of the seat belt 20, a pattern extending at a predetermined interval included in the captured image corresponds. The identifying unit 30 identifies the seat belt 20 included in the captured image based on such a pattern.

  The estimation unit 40 estimates the wearing state of the occupant's seat belt 20 based on the form of the seat belt 20 specified by the specifying unit 30. The form of the seat belt 20 specified by the specifying unit 30 is a pattern based on the pattern attached to the seat belt 20 included in the captured image. The wearing state of the seat belt 20 of the occupant is a state of the seat belt 20 worn by the occupant.

  The estimation unit 40 estimates the wearing state based on the pattern form in the captured image when the occupant wears the seat belt 20 acquired in advance and the pattern form specified by the specifying unit 30. The form of the pattern in the captured image when the occupant wears the seat belt 20 acquired in advance is taken in advance so that the occupant can properly protect the occupant even in the unlikely case. The pattern is based on the pattern attached to the seat belt 20 included in the captured image. Such a pattern is stored in the storage unit 50 in advance. Therefore, the form of the pattern memorize | stored in the memory | storage part 50 turns into a pattern when a passenger | crew has mounted | worn the seatbelt 20 correctly. Therefore, the estimation unit 40 compares the pattern form stored in the storage unit 50 with the pattern form specified by the specifying unit 30, and estimates the wearing state of the occupant's seat belt 20 by pattern recognition. To do.

  FIG. 2A shows a pattern based on the pattern when the occupant stored in advance in the storage unit 50 correctly wears the seat belt 20. FIG. 2A shows an example of a pattern based on a pattern attached at a predetermined interval along a direction orthogonal to the extending direction of the seat belt 20. Of course, this pattern may be taken by the occupant wearing the seat belt 20.

  On the other hand, (b), (c), and (d) in FIG. 2 are examples of the pattern of the seat belt 20 specified by the specifying unit 30 from the captured image acquired when the infrared camera 10 is taken while facing the seat. is there. (B) is a pattern in the case where the occupant appropriately wears the seat belt 20. (C) is a pattern when the occupant is seated on the seat with the seat belt 20 being passed between the occupant's back and the seat (back of the seat). (D) is a pattern when the occupant seats the seat belt 20 through the armpit.

  The estimation unit 40 compares the pattern (a) stored in advance in the storage unit 50 with the pattern of the seat belt 20 based on the captured image acquired by the infrared camera 10, and performs pattern recognition. For example, when a passenger in the driver's seat properly wears the seat belt 20, the pattern is as shown in (b). The pattern (b) is similar to the pattern (a) because the pattern having a predetermined interval extends along a predetermined direction. Therefore, the estimation unit 40 has the seat belt 20 attached appropriately. Presumed to be in a state.

  On the other hand, in the pattern (c), a pattern having a predetermined interval extends along the predetermined direction at the upper left and the lower right, but there is no pattern at the center. For this reason, the estimation unit 40 estimates that the occupant is seated on the infrared camera 10 side of the seat belt 20 and the seat belt 20 is not properly worn.

  In the pattern (d), the pattern does not continuously extend from the upper left to the center, and the pattern is not arranged in a straight line. For this reason, the estimation unit 40 estimates that the occupant is passing the seat belt 20 under the armpit and is not wearing the seat belt 20 appropriately. In this way, the estimation unit 40 estimates whether or not the seat belt 20 is properly worn. The estimation result of the estimation unit 40 is transmitted to the face recognition unit 60 described later.

  The face recognition unit 60 recognizes the occupant's face from the captured image. As for the face recognition, a known image recognition process can be performed, and the description thereof will be omitted. The recognition result of the face recognition unit 60 is transmitted to a notification unit 70 described later.

  When the seat belt 20 is not properly worn and the face recognition unit 60 recognizes a face (driver's face), the notification unit 70 notifies the user so as to prompt a warning. Further, when the seat belt 20 is properly worn and the face recognition unit 60 recognizes the driver's face and the position of the driver's face is inappropriate, Since it is assumed that there is a possibility of driving aside, the notification unit 70 performs notification.

  On the other hand, when the seat belt 20 is not properly worn on the passenger seat and the face is not recognized by the face recognition unit 60, an object is placed on the seat and the object does not move. Thus, since it is assumed that the seat belt 20 is fixed, the notification unit 70 does not perform notification. Further, even when the seat belt 20 is properly worn and the face is not recognized by the face recognition unit 60, it is assumed that the object is placed on the seat and fixed with the seat belt 20 so that the object does not move. Therefore, the notification unit 70 does not perform notification. Of course, when the seat belt 20 is properly worn and the face is recognized by the face recognition unit 60, if the face is not the driver's face, it is assumed that the passenger is sleeping. Therefore, the notification unit 70 does not perform notification.

  In this way, the occupant monitoring device 1 monitors the occupant's riding state based on the wearing state of the seat belt 20 estimated by the estimation unit 40. Further, with this configuration, it is possible to prevent notification when an object is placed on the seat and fixed by the seat belt 20. Further, by relatively detecting the positions of the seat belt 20 and the occupant, it is possible to determine whether or not the occupant is seated in an appropriate posture and in an appropriate position.

  FIG. 3 is a flowchart showing processing related to occupant monitoring by the occupant monitoring device 1. The occupant monitoring device 1 starts occupant monitoring after the engine is started (step # 01: Yes). When the seating sensor provided on the seat detects a load (step # 02: Yes), the infrared camera 10 starts to acquire a captured image (step # 03).

  When the identifying unit 30 recognizes (identifies) the seat belt 20 in the captured image acquired by the infrared camera 10 (step # 04: Yes), the face recognition unit 60 performs face recognition. When the face is recognized by the face recognition unit 60 (step # 05: Yes), it is determined whether or not the face position is abnormal. When the position of the face is abnormal (step # 06: Yes) and the face is the driver's face (step # 07: Yes), the notification unit 70 gives a notification (warns) ( Step # 08).

  When the face recognition unit 60 cannot recognize a face from the captured image at step # 05 (step # 05: No), when the face position is not abnormal at step # 06 (step # 06: No), or at step # 07 If the face included in the captured image is not the face of the driver (step # 07: No), the process returns to step # 02 and the process is continued.

  In step # 04, even when the specifying unit 30 cannot recognize the seat belt 20, the face recognition unit 60 performs face recognition processing. When the face recognition unit 60 recognizes a face (step # 09: Yes), the notification unit 70 performs notification (step # 08). On the other hand, if the face recognition unit 60 cannot recognize the face (step # 09: No), the process returns to step # 02 and the process is continued. The occupant monitoring apparatus 1 monitors the occupant along such a flow.

[Other Embodiments]
In the above-described embodiment, the estimation unit 40 determines the seat belt 20 based on the pattern form in the captured image when the occupant wears the seat belt 20 acquired in advance and the pattern form specified by the specifying unit 30. It was described as estimating the wearing state. For example, the infrared camera 10 acquires captured images at predetermined time intervals, and the estimating unit 40 is based on time series changes in the form of the seat belt 20 in a plurality of captured images acquired by the infrared camera 10. It is also possible to configure so as to estimate the wearing state. According to such a configuration, it is possible to estimate the wearing state of the seat belt 20 according to the movement of the seat belt 20.

  The seat belt 20 is preferably provided with a biosensor that detects at least one of heartbeat and respiration. When the seat belt 20 is properly worn, the seat belt 20 comes into contact with the chest of the occupant. Therefore, if the heartbeat and breathing can be detected, the seat belt 20 can be properly worn. Therefore, the estimation unit 40 can estimate the wearing state of the seat belt 20 of the occupant and the position and posture of the occupant with high accuracy based on the detection result of the biosensor. It is also possible to detect the physical condition of the occupant.

  In the above-described embodiment, the notification is made based on the seat belt 20 and the face recognition. However, in addition to the notification, in order to prevent the start of operation when the seat belt 20 is not attached, for example, a handle lock or a shift lock is performed. It is also possible to configure.

  Further, when the position of the subject driver changes greatly during driving, or when the seat belt 20 cannot be recognized, the vehicle can be configured to be safely and urgently stopped.

  The present invention can be used in an occupant monitoring device that monitors the boarding state of an occupant of a vehicle.

1: Crew monitoring device 10: Infrared camera 20: Seat belt 30: Specific unit 40: Estimating unit

Claims (4)

An infrared camera that shoots from the front of the seat provided in the vehicle toward the seat;
A seat belt formed so as to be able to be worn by an occupant seated on the seat and including a material that reflects infrared rays;
A specifying unit for specifying the seat belt included in the captured image acquired by the infrared camera;
Based on the form of the seat belt specified by the specifying unit, and an estimation unit that estimates a wearing state of the seat belt of the occupant,
An occupant monitoring apparatus that monitors the occupant's riding state based on the wearing state estimated by the estimating unit. The seat belt is provided with a pattern that can be discriminated at the time of photographing with the infrared camera using the material,
The specifying unit specifies the seat belt based on a form of the pattern included in the captured image,
The estimation unit estimates the wearing state based on the pattern form in the captured image when the occupant wears the seat belt acquired in advance and the pattern form specified by the specifying unit. Item 10. The occupant monitoring device according to item 1. The infrared camera acquires the captured image every preset time,
The occupant monitoring apparatus according to claim 1, wherein the estimation unit estimates the wearing state based on a time-series change in the form of the seat belt in the plurality of captured images acquired by the infrared camera. The seat belt is provided with a biosensor for detecting at least one of heartbeat and respiration,
The occupant monitoring apparatus according to any one of claims 1 to 3, wherein the estimation unit estimates the wearing state based on a detection result of the biological sensor.

Images