TWI741892B - In-car driving monitoring system - Google Patents

Abstract

An in-vehicle driving monitoring system includes an image capturing device and a driving monitoring device. The image capturing device captures a driving image of driving; the driving monitoring device is connected to the image capturing device and includes an image processing unit and a driving monitoring device. A feature point identification unit and a condition monitoring unit. The image processing unit performs an image processing flow on the driving image and generates a face image; the feature point identification unit is connected to the image processing unit to generate a corresponding person based on the face image The complex number feature points of the facial features, and the position of the complex feature points are represented by coordinate values; the state monitoring unit is connected to the feature point identification unit, and the coordinate value of the complex feature points is compared with the internally stored complex preset feature points The coordinate values are compared to determine whether the driving state is abnormal.

Description

In-car driving monitoring system

A monitoring system, especially an in-vehicle driving monitoring system.

Vehicles are one of the most widely used means of transportation in modern society and are closely related to the lives of modern people. For example, daily commuting or cargo transportation involves the application of vehicles. As the number of vehicles increases, the probability of traffic accidents also follows. rising.

The safe driving of a vehicle depends on the driver’s concentration and driving. The driver must always pay attention to the surrounding vehicles and road environment, and maintain a good mental state to respond to various emergencies during driving. When fatigue or mental insufficiency is caused by other factors, the driver is prone to feel drowsy, which makes the driver unable to concentrate, slows down the reaction speed, and even becomes sleepy, which may cause the driver to neglect the driving of the vehicle and cause the vehicle to lose control. Accidents and fatigue driving easily affect human vision, which may result in blurred and diminished vision of the driver, which affects the driver's ability to observe the surrounding environment. The driver cannot respond to emergencies on the road in real time, thereby causing a traffic accident.

In view of this, it is necessary to improve the safety of vehicle driving and reduce the probability of traffic accidents. It is possible to start by detecting the status of the driver to avoid the occurrence of fatigue driving. The present invention provides an in-vehicle driving monitoring system that captures the driver’s Face image, and judge the driver's state according to the feature points corresponding to the facial features in the image, and immediately warn the driver when there is an abnormal state, reminding the driver to pay attention to the condition of the car and their own state, so as to improve the safety of driving.

To achieve the foregoing objective, the in-vehicle driving monitoring system of the present invention includes: An image capture device to capture a driving image of the driver; and A driving monitoring device, connected to the image capturing device, includes: An image processing unit that performs an image processing flow on the driving image captured by the image capturing device, and generates a face image; A feature point identification unit, connected to the image processing unit, generates a complex feature point corresponding to the facial features of the face according to the face image, and expresses the position of the complex feature point as a coordinate value; and A state monitoring unit connected to the feature point identification unit, the state monitoring unit stores therein a plurality of preset feature points corresponding to the plurality of feature points, and the state monitoring unit has the coordinate value of the plurality of feature points and the plurality of preset feature points When the distance between the coordinate value of each feature point and the coordinate value of each preset feature point exceeds a feature point offset reference value, or when the distance between two feature points When the distance between the two corresponding preset characteristic points exceeds a characteristic point change reference value, the state monitoring unit determines that the driving state is abnormal, and outputs a warning signal to warn the user.

The in-vehicle driving monitoring system of the present invention captures the driving image of the driver through the image capture device, the image capture device performs image processing on the driving image, and transmits the image processed face image to the feature Point recognition unit, the feature point recognition unit generates the complex feature points corresponding to different facial features according to the facial features in the face image, and then the state monitoring unit compares the complex feature points to calculate the complex number The coordinate difference between the feature point and the plurality of preset feature points is used to determine whether the current state of the driver deviates from the normal state. When the state monitoring unit determines that the driver's state is abnormal, the warning signal is used to alert the driver to Improve the safety of vehicle driving.

Please refer to Figure 1. The in-vehicle driving monitoring system 1 of the present invention is used to monitor the status of the driver and warn the driver when the driver's status is abnormal. The in-vehicle driving monitoring system 1 includes: an image capture device 10 and a driving monitoring device 20. The image capturing device 10 may be an infrared lens. The image capturing device 10 captures a driving image of the driver, and the image capturing device 10 captures the driving image outward Transmission, the image capturing device 10 can be installed on the dashboard, windshield, air-conditioning outlet, rear mirror, etc. of the vehicle, so that the shooting direction of the image capturing device 10 is toward the driver’s area. When a person rides in a vehicle and drives the vehicle, the image capturing device 10 can be aimed at the driver's face to capture the driving image when the driver is driving and operating the vehicle, wherein the image capturing device The setting position of 10 is based on the position where the driver can capture the driving image from the front of the driver when the driver is driving the vehicle in a normal state, and the image capture device 10 can encode the driving image with H.264 video encoding (and The technology called MPEG-4 performs image compression to reduce the size of the image capacity during the transmission of the driving image, and the image capture device 10 can perform Wi-Fi Peer-to-Peer (P2P) protocol for the driving image transmission.

The driving monitoring device 20 is connected to the image capturing device 10, and includes an image processing unit 21, a feature point identification unit 22, and a condition monitoring unit 23. The driving monitoring device 20 can be a mobile phone or a tablet, etc. The driving monitoring device 20 executes a driving monitoring program. The driving monitoring device 20 can receive the driving image transmitted by the image capturing device 10 through the Wi-Fi Peer-to-Peer (P2P) protocol, and the image processing unit 21 uses H.264 video encoding (also known as MPEG-4) technology to decode the driving image, and the image processing unit 21 performs an image processing procedure on the driving image to generate a face image.

The feature point identification unit 22 is connected to the image processing unit 21. The feature point identification unit 22 uses a boundary point of the face image as a coordinate starting point according to the pixels of the face image after the image processing flow, and sets the A plane coordinate axis is established on the face image, and a complex feature point corresponding to the facial features of the face is generated on the face image, and the position of the complex feature point is expressed as a coordinate value, where the complex feature point may include the eye Feature points, mouth feature points, nose feature points, etc.

As shown in FIG. 2, taking a face image of the frontal face of the driver in a normal state as an example, if the pixel value of the face image is 320*240, the feature point identification unit 22 can be the lower left of the face image The angle is the starting point (0, 0) to set the coordinate system. If the first axis is the X axis and the second axis is the Y axis, the feature point identification unit 22 sets the first axis range of the face image as 0~320. The second axial range of the face image is set to 0~240, and the complex feature points corresponding to the five senses in the face image are generated. The complex feature points may include corresponding points on the two eyebrows of the driver. A first feature point P1 and a second feature point P2 corresponding to the upper and lower edges of one of the driver’s eyes, a third feature point P3 and a fourth feature point P4 corresponding to the corners of the driver’s mouth on both sides Fifth feature point P5 and a sixth feature point P6, etc., but the coordinate starting point set by the feature point identification unit 22 can also be the upper right corner, upper left corner of the face image and other boundary points of the face image, The coordinate starting point and axis of the face image are not limited to this embodiment.

The state monitoring unit 23 is connected to the feature point identification unit 22. The state monitoring unit 23 stores a plurality of preset feature points corresponding to the plurality of feature points. Each preset feature point corresponds to a feature point, and each preset feature point corresponds to each other. Assuming that the feature point and each feature point represent the position of the same part of the facial features, the state monitoring unit 23 has the coordinate value of the complex feature point generated by the feature point identification unit 22 and the coordinate value of the complex preset feature point For comparison, when the distance between the coordinate value of each feature point and the coordinate value of the corresponding preset feature point exceeds a feature point offset reference value, or when the distance between the two feature points and the corresponding When the distance between the two preset feature points exceeds a feature point change reference value, the state monitoring unit 23 determines that the driver's state is abnormal, and outputs a warning signal to warn the user, wherein the plurality of preset feature points It can be set before the driver uses the in-vehicle driving monitoring system 1 to monitor the driving state. The image capturing device 10 first captures a driving image of the driver in a normal state, and the image capturing device 10 captures the driving image After being transmitted to the driving monitoring device 20, the image processing unit 21 of the driving monitoring device 20 executes the image processing procedure on the driving image, and then the feature point identification unit 22 according to the face image after the image processing procedure The state of the five sense organs of the image generates the plurality of preset feature points, and finally the state monitoring unit 23 stores the plurality of preset feature points to complete the process of setting the plurality of preset feature points.

Please refer to Figure 3, the process of the in-vehicle driving monitoring system 1 performing driving state monitoring includes:

S10: The image capturing device 10 captures a driving image of the driver.

S11: The image processing unit 21 performs an image processing procedure on the driving image. Further referring to FIG. 4, the image processing procedure includes:

S111: The image processing unit 21 captures a face image of the driver from the driving image.

S112: The image processing unit 21 performs brightness and contrast processing on the face image, increases the brightness of the face image, and increases the contrast of the face image, so as to highlight the outline of the face and emphasize the main body of the five senses.

S113: The image processing unit 21 gray-scales the face image, and each pixel in the face image has a different gray-scale value.

S114: The image processing unit 21 black-and-white converts the gray-scaled face image, and the image processing unit 21 compares the gray-scale value of each pixel in the face image with a threshold When the grayscale value of a pixel in the face image exceeds the threshold, the image processing unit 21 converts the pixel into a black point, and the grayscale value of a pixel in the face image When the threshold is not exceeded, the image processing unit 21 converts the pixel points into white points, which means that the face image is converted into a binary image with only black and white for subsequent image analysis.

S115: The image processing unit 21 recognizes and confirms the facial features of the face according to the black-and-white face image.

S12: The feature point recognition unit 22 generates a complex number of feature points corresponding to the facial features of the face on the face image, and expresses the position of the complex number of feature points in coordinates, where the technology for recognizing facial feature points is an image The existing technology in the field of identification will not be detailed here.

S13: The state monitoring unit 23 compares the coordinate values of the plurality of feature points with the coordinate values of the plurality of preset feature points, when the difference between the coordinate values of each of the feature points and the corresponding preset feature points is within a certain deviation When shifting within the allowable value, it means that the coordinates of each feature point and each preset feature point are slightly different from each other. The difference between the coordinates of each feature point and each preset feature point may be caused by the vibration of the vehicle or the driver The small action means that the driver is still in a normal state, the in-vehicle driving monitoring system 1 re-executes step S10; and when the difference between the coordinate values of each of the feature points and the corresponding preset feature points exceeds an allowable offset value At this time, the state monitoring unit 23 determines that the driving state has changed, and executes the determination in step S14.

S14: The state monitoring unit 23 performs the coordinate value of the axis in the straight direction of the driver's face in each of the feature points and the coordinate value of the axis in the straight direction of the driver's face in each of the preset feature points. Compare, determine whether it is necessary to adjust the coordinate values of the plurality of preset feature points. When the characteristic point identification unit 22 uses the lower left or lower right boundary point of the face image as the starting point of the coordinate axis, when there is a characteristic point, the coordinate value of the axis in the vertical direction of the driver's face is greater than the corresponding prediction. When the coordinate value of the axis in the vertical direction of the driver's face in the feature points is set, step S15 is executed; and when there is no feature point, the coordinate value of the axis in the vertical direction of the driver's face is greater than the corresponding preset feature When the coordinate value of the axis in the straight direction of the driver’s face is selected, step S16 is executed. Taking Fig. 2 as an example, the axis in the straight direction of the driver’s face is the second axis (Y-axis). When the image capturing device 10 captures the driving image of the driver, if the driver is far away from the image capturing device 10, or the driver’s face is closer to the bottom of the capturing range of the image capturing device 10 When the boundary is reached, the coordinate value of the second axis of each of the preset feature points generated by the feature point identification unit 22 is likely to be small. When the driver approaches the image capturing device 10 or readjusts the sitting posture, When the face is closer to the upper boundary of the capturing range of the image capturing device 10, the new plural feature points obtained from the new driving image have a larger second axis coordinate value, so the preset feature When the point is compared to the new complex feature point, the problem of inaccurate state recognition is likely to occur, and the new complex feature point needs to replace the previously preset complex preset feature point for subsequent driving state monitoring.

Similarly, when the feature point identification unit 22 uses the upper left boundary point or the upper right boundary point of the face image as the starting point of the coordinate axis, when there is a feature point, the coordinate value of the axis in the vertical direction of the driver's face is smaller than the corresponding When the coordinate value of the axis in the straight direction of the driver’s face in the preset feature points is performed, step S15 is executed; and when there is no feature point, the coordinate value of the axis in the straight direction of the driver’s face is smaller than the corresponding preset feature point. When the coordinate value of the axial direction of the driver’s face in the feature point is set, step S16 is executed. When the image capturing device 10 captures the driving image of the driver, if the driver is away from the image capturing device 10 is far away, or when the driver’s face is closer to the upper boundary of the capturing range of the image capturing device 10, it is easy to cause the second axis of each of the preset feature points generated by the feature point identification unit 22 When the driver brings his face closer to the image capturing device 10 or readjusts his sitting posture so that his face is closer to the lower boundary of the capturing range of the image capturing device 10, the new driving image The obtained new complex feature point has a larger second axis coordinate value. Therefore, when the preset feature point is compared with the new complex feature point, the problem of inaccuracy of state identification is likely to occur, and the new complex number must be used. The feature point replaces the previously preset plural preset feature points for subsequent driving state monitoring.

S15: The state monitoring unit 23 determines that the coordinate value of each preset feature point needs to be adjusted, and the state monitoring unit 23 sets each feature point corresponding to each preset feature point that needs to be adjusted as a new preset feature Click and execute step S16.

S16: The state monitoring unit 23 compares the coordinate value of the plurality of feature points with the coordinate value of the plurality of preset feature points, and when the coordinate value of each feature point corresponds to the coordinate value of each preset feature point When the distance difference exceeds a feature point offset reference value, or when the distance between two feature points and the corresponding two preset feature points are greater than a feature point change reference value, it means that the driver may appear Head down, eyes closed, head turning, etc., or abnormal conditions that increase the risk of driving the vehicle, the state monitoring unit 23 determines that the driver's state is abnormal, and outputs a warning signal to warn the user.

The following uses the plural feature points and the plural preset feature points corresponding to different facial features to describe in detail how the state monitoring unit 23 performs step S16.

As shown in FIG. 5A and FIG. 5B, in the normal state of FIG. 5A, a first preset feature point P1' and a second preset feature point corresponding to the upper edge and the lower edge of the driver's eyes in the face image respectively P2', and a first feature point P1 and a second feature point P2 corresponding to the upper and lower edges of the driving eyes in the face image when the eyes are closed in FIG. 5B are taken as examples. , Making the coordinate position of the first feature point P1 deviate from the preset first feature point P1', the state monitoring unit 23 can compare the first feature point P1 with the preset first feature point P1', and the state monitoring On the one hand, the unit 23 can determine whether the distance between the coordinate value of the first feature point P1 and the coordinate value of the preset first feature point P1' exceeds the feature point offset reference value, and on the other hand, it can determine whether the distance between the coordinate value of the first feature point P1 and the coordinate value of the first feature point P1' Whether the distance between the feature point P1 and the second feature point P2 and the corresponding distance between the first preset feature point P1' and the second preset feature point P2' exceed the feature point change reference value , To determine whether the driving state is abnormal.

As shown in FIG. 6A and FIG. 6B, in the normal state in FIG. 6A, a first preset feature point P1' and a second preset feature point corresponding to the upper and lower edges of the driver's eyes in the face image respectively P2', a third preset feature point P3' corresponding to the upper edge of the driver's eyebrows, a fourth preset feature point P4' corresponding to the edge of the corner of the driver's mouth, and the face image in the lowered state in FIG. 6B, respectively A first feature point P1 and a second feature point P2 corresponding to the upper and lower edges of the driver’s eyes, a third feature point P3 corresponding to the upper edge of the driver’s eyebrows, and a fourth feature point P4 corresponding to the edge of the driver’s mouth As an example, it can be seen from Fig. 6 that the position of the first feature point P1, the second feature point P2, the third feature point P3, and the fourth feature point P4 are all offset from the corresponding first prediction point due to the driver's head down. Suppose the feature point P1', the second preset feature point P2', the third preset feature point P3', and the fourth preset feature point P4', the state monitoring unit 23 can determine the first feature point on the one hand The coordinate values of the second feature point P2, the third feature point P3, and the fourth feature point P4 correspond to the first preset feature point P1', the second preset feature point P2', the Whether the distance difference between the coordinate values of the third preset feature point P3' and the fourth preset feature point P4' exceeds the offset reference value of the feature point, on the other hand, the distance between two different feature points can be judged, Whether the distance between the two corresponding preset feature points exceeds the reference value of the feature point change is used to determine whether the driving state is abnormal.

As shown in FIGS. 7A and 7B, in addition to each of the feature points corresponding to the five sense organs and each of the preset feature points in FIGS. 5A to 6B, when the driver wears glasses, the feature point identification unit 22 can also generate differences. The plurality of preset feature points and the plurality of feature points corresponding to the position of the frame of the glasses are a fifth preset feature point P5′ corresponding to the upper and lower edges of the driver's glasses in the face image in the normal state in FIG. 7A. And a preset sixth feature point P6', and a fifth feature point P5 and a sixth feature point P6 corresponding to the upper and lower edges of the driver's glasses in the face image in the lowered state in FIG. 7B as an example It can be seen from FIG. 7 that the coordinate positions of the fifth feature point P2 and the sixth feature point P6 are offset due to the driver's head down, corresponding to the preset fifth feature point P5' and the preset sixth feature point P6' On the one hand, the state monitoring unit 23 can determine that the coordinate values of the fifth feature point P5 and the sixth feature point P6 correspond to the coordinate values of the preset fifth feature point P5' and the sixth feature point P6', respectively Whether the distance difference between the feature points exceeds the reference value of the feature point offset, on the other hand, it can be judged that the distance between the fifth feature point P5 and the sixth feature point P6 corresponds to the fifth preset feature point P5' and the Whether the distance between the sixth preset feature point P6' exceeds the reference value of the feature point change is used to determine whether the driver's state is abnormal.

As shown in FIG. 8A and FIG. 8B, in the normal state in FIG. 8A, a third preset feature point P3' corresponding to the upper edge of the driver’s eyebrows and a fourth preset feature point corresponding to the edge of the driver’s mouth The feature point P4', a fifth preset feature point P5' and a preset sixth feature point P6' respectively corresponding to the upper and lower edges of the driver’s glasses, and the face image in the head-turned state in FIG. 8B A third feature point P3 corresponding to the upper edge of the driver’s eyebrows, a fourth feature point P4 corresponding to the edge of the corner of the driver’s mouth, a fifth feature point P5 and a sixth feature point corresponding to the upper and lower edges of the driver’s glasses, respectively P6 is an example. It can be seen from Fig. 8 that the coordinate positions of the third feature point P3, the fourth feature point P4, the fifth feature point P2, and the sixth feature point P6 are all offset due to the driver turning his head. For the third preset feature point P3', the fourth preset feature point P4', the preset fifth feature point P5', and the preset sixth feature point P6', the state monitoring unit 23 can determine the The coordinate values of the third feature point P3, the fourth feature point P4, the fifth feature point P5, and the sixth feature point P6 correspond to the third preset feature point P3' and the fourth preset feature point, respectively P4', whether the distance difference between the coordinate values of the preset fifth feature point P5' and the sixth feature point P6' exceeds the feature point offset reference value, on the other hand, it can be judged whether the difference between two different feature points is The distance, whether the difference between the length and the distance between the corresponding two preset feature points exceeds the reference value of the feature point change, is used to determine whether the driver's state is abnormal.

As shown in FIGS. 9A and 9B, if the driver wears sunglasses, so that the image capture cannot obtain the information of the eye state, the state monitoring unit 23 can still use the characteristic points corresponding to the upper and lower edges of the sunglasses. Each of the feature points of the eyebrows or the mouth determines the state of the driver. Taking a third preset feature point P3′ corresponding to the upper edge of the driver’s eyebrows, a fourth preset feature point P4′ corresponding to the edge of the corner of the driver’s mouth in the face image in the normal state in FIG. 9A, respectively corresponding to the driver A fifth preset feature point P5' and a preset sixth feature point P6' on the upper and lower edges of the glasses, and a third feature point corresponding to the upper edge of the driver’s eyebrows in the face image in FIG. 9B when the head is lowered. Feature point P3, a fourth feature point P4 corresponding to the edge of the corner of the driver’s mouth, a fifth feature point P5 and a sixth feature point P6 respectively corresponding to the upper and lower edges of the driver’s glasses are taken as examples. The person lowers his head, so that the coordinate positions of the third feature point P3, the fourth feature point P4, the fifth feature point P2, and the sixth feature point P6 are all offset from the corresponding third preset feature point P3', the The fourth preset feature point P4', the preset fifth feature point P5', and the preset sixth feature point P6', the state monitoring unit 23 can determine the third feature point P3 and the fourth feature point on the one hand P4, the coordinate values of the fifth feature point P5 and the sixth feature point P6 correspond to the third preset feature point P3', the fourth preset feature point P4', and the preset fifth feature point P5, respectively Whether the distance difference between the coordinate values of'and the sixth feature point P6' exceeds the feature point offset reference value, on the other hand, it can determine the distance between two different feature points, and the corresponding two preset feature points Whether the difference between the length and the length of the distance exceeds the reference value of the feature point change, it is judged whether the driver's state is abnormal.

In addition to judging the state of the driver from a single face image, the in-vehicle driving monitoring system 1 of the present invention can also perform a shake detection from multiple face images acquired within a preset time to determine whether the driver’s head is shaken or shaken. In a jitter state, the state monitoring unit 23 calculates the average value of a landmark on the same axis of a feature point corresponding to the same facial features in the complex face image, and compares the average value of the coordinate with the preset feature corresponding to the feature point The coordinate value of the point is compared, and when the difference between the average value of the coordinate and the coordinate value of the preset feature point exceeds a jitter threshold, the state monitoring unit 23 determines that the driver's state is abnormal, and outputs the warning signal to warn driving people.

As shown in Figure 2, five face images as shown in Figure 2 are captured, and the coordinate values of the first feature point of each face image are (152,210), (155,207), (152,225), (150,201). ), (153,211) as an example, if the coordinate value of the second axis (Y axis) is used for judgment, the state monitoring unit 23 calculates the coordinates on the second axis of each of the first feature points in each face image The average value is 210+207+225+201+211=210.8, and the coordinate average value is compared with the corresponding coordinate value of the first preset feature point on the second axis. When the coordinate average value is When the numerical difference between the coordinate values of the preset feature points exceeds the jitter threshold, the state monitoring unit 23 determines that the driver has an abnormal state such as jitter or shaking.

Further, in order to improve the accuracy of the jitter detection, the state monitoring unit 23 may first remove the two feature points with the maximum coordinate and the minimum coordinate among the plurality of feature points, and calculate the average value of the coordinates of the remaining feature points. Take five face images as shown in Figure 2, and the coordinate values of the first feature point of each face image are (152,210), (155,207), (152,225), (150,201), (153,211) as an example If the judgment is made with the coordinate value of the second axis (Y-axis), the state monitoring unit 23 judges that the first feature point with a coordinate value of (150,201) has the minimum value of the coordinate on the second axis, and the coordinate value is (152,225). The first feature point of) has the maximum value of the coordinates in the second axis. Therefore, the state monitoring unit 23 eliminates the two first feature points with coordinate values (150,201) and (152,225), and the state monitoring unit 23 calculates the remaining The average value of the coordinates on the second axis of each first feature point, that is, 210+207+211=209.3, and the average value of the coordinates and the corresponding coordinate value of the first preset feature point on the second axis are performed By comparison, when the numerical difference between the coordinate average value and the coordinate value of the preset feature point exceeds the jitter threshold, the state monitoring unit 23 determines that the driver has an abnormal state such as jitter or shaking.

Wherein, the warning signal generated by the state monitoring unit 23 can control the electronic device that executes the driving monitoring device 20, and the electronic device displays a warning message on the display screen according to the warning signal, or emits warning sounds and lights, by This warns the driver.

In summary, the in-vehicle driving monitoring system 1 of the present invention captures the driving image when the driver is driving the vehicle through the image capturing device 10, and the image capturing device 10 transmits the driving image to the driving monitoring device 20 The image capturing device 10 executes the image processing flow on the driving image, and transmits the image-processed face image to the feature point identification unit 22, and the feature point identification unit 22 performs the image processing according to the face image. The facial features in the image generate the complex feature points corresponding to different facial features, and then the state monitoring unit 23 compares the complex feature points with the complex preset feature points when the driver is in a normal state to calculate the complex feature The coordinate difference between the point and the plurality of preset feature points is used to determine whether the current state of the driver deviates from the normal state. When the state monitoring unit 23 determines that the state of the driver is abnormal, the warning signal is used to alert the driver to improve The safety of the driver in the car reduces the possibility of traffic accidents caused by the driver's mental insufficiency and fatigue driving.

1: In-car driving monitoring system

10: Image capture device

20: Driving monitoring device

21: Image processing unit

22: Feature point identification unit

23: Condition monitoring unit

P1: The first feature point

P1’: The first preset feature point

P2: The second feature point

P2’: The second preset feature point

P3: The third feature point

P3’: The third preset feature point

P4: Fourth feature point

P4’: The fourth preset feature point

P5: Fifth feature point

P5’: Fifth preset feature point

P6: The sixth feature point

P6’: The sixth preset feature point

Figure 1: A block diagram of the in-vehicle driving monitoring system of the present invention. Figure 2: Schematic diagram of the face image of the frontal face when driving in a normal state. Figure 3: A flow chart of the steps of the in-vehicle driving monitoring system of the present invention for monitoring the driving state. Figure 4: A flow chart of the steps of the image processing process performed by the image processing unit. Fig. 5A: A schematic diagram of a face image of a driver in a normal state. Fig. 5B: A schematic diagram of a face image of a driver with his eyes closed. Fig. 6A: A schematic diagram of a face image of a driver in a normal state. Fig. 6B: A schematic diagram of a driver's face image in a state where the driver's head is lowered. Fig. 7A: A schematic diagram of a face image of a driver in a normal state while wearing glasses. Fig. 7B: A schematic diagram of a face image of a driver with his head down when wearing glasses. Fig. 8A: A schematic diagram of a face image of a driver in a normal state while wearing glasses. Fig. 8B: A schematic diagram of a face image of a driver turning his head while wearing glasses. Fig. 9A: A schematic diagram of a face image of a driver in a normal state while wearing sunglasses. Fig. 9B: A schematic diagram of a face image of a driver with his head down while wearing sunglasses.

1: In-car driving monitoring system

10: Image capture device

20: Driving monitoring device

21: Image processing unit

22: Feature point identification unit

23: Condition monitoring unit

Claims (9)

An in-vehicle driving monitoring system includes: an image capturing device that captures a driving image of a driver; and a driving monitoring device connected to the image capturing device, including: an image processing unit that captures the image The driving image captured by the capturing device performs an image processing flow to generate a face image; a feature point identification unit connected to the image processing unit to generate plural feature points corresponding to the facial features according to the face image, and The position of the plurality of feature points is represented by coordinate values; and a state monitoring unit is connected to the feature point identification unit, the state monitoring unit stores a plurality of preset feature points corresponding to the plurality of feature points, and the state monitoring unit The coordinate values of the plural feature points are compared with the coordinate values of the plural preset feature points, when the distance between the coordinate value of each feature point and the corresponding coordinate value of each preset feature point exceeds a feature point offset reference Value, or when the difference between the distance between the two feature points and the distance between the corresponding two preset feature points exceeds a feature point change reference value, the state monitoring unit determines that the driving state is abnormal, and outputs a A warning signal warns the user; the image processing unit obtains a plurality of facial images within a predetermined time, the state monitoring unit performs a shake detection based on the plurality of facial images, and the state monitoring unit calculates the plurality of facial images In the image, the average value of a feature point corresponding to the same facial features is compared, and the average value of the coordinate is compared with the coordinate value of the preset feature point corresponding to the feature point. When the average value of the coordinate and the preset feature point are When the numerical difference between the coordinate values exceeds a jitter threshold, the state monitoring unit judges that the driving state is abnormal and outputs a warning signal. For the in-vehicle driving monitoring system according to claim 1, the image processing flow includes: the image processing unit captures a face image from the driving image; The image processing unit performs brightness and contrast processing on the face image; the image processing unit performs grayscale processing on the face image; the image processing unit performs black and white grayscale on the face image Processing; and the image processing unit recognizes the position of the facial features of the face based on the black-and-white face image. For the in-vehicle driving monitoring system described in claim 1, the feature point identification unit uses a boundary point of the face image as a coordinate starting point, and establishes a plane coordinate axis on the face image, and then sets the face image The plural feature points corresponding to the facial features of the human face are generated on the image. For the in-vehicle driving monitoring system described in claim 1, the image capturing device captures a driving image of the driver when the driver is in a normal state, and the image processing unit of the driving monitoring device captures the driving image Performing the image processing flow to generate a face image, the feature point recognition unit generates the plurality of preset feature points corresponding to the facial features of the driver in a normal state according to the face image, and stores the plurality of preset feature points In the condition monitoring unit. According to the in-vehicle driving monitoring system according to claim 1, the plural feature points include feature points corresponding to the positions of the upper and lower edges of the eyes, the positions of the corners of the mouth on both sides, or the positions of the eyebrows. For the in-vehicle driving monitoring system according to claim 1, the plurality of feature points include feature points corresponding to the position of the frame of the glasses. For the in-vehicle driving monitoring system described in claim 1, the image capturing device and the driving monitoring device use H.264 video coding technology for image processing. For the in-vehicle driving monitoring system described in claim 1, the image capturing device and the driving monitoring device use Wi-Fi Peer-to-Peer (P2P) protocol to transmit the driving image. According to the in-vehicle driving monitoring system of claim 1, the image capturing device is an infrared lens.

Images