TW201819226A - Method for detecting driving behaviors and system thereof capable of alerting drivers to concentrate on driving before an abnormal driving behavior occurs, thereby effectively improving driving safety - Google Patents

Abstract

The present invention relates to a method for detecting driving behaviors and a system thereof, which comprises capturing an image of a user with an arm image and a head behavior image; capturing and comparing an arm image with an arm sample image in a database with a processor, wherein the arm image determines whether the driving behavior is normal or not, so when the arm image conforms to the arm sample image, the driving behavior is considered normal, and vice versa; capturing and comparing a head behavior image with a head violation sample image in the database, wherein when the head behavior image does not conform to the head violation sample image, it indicates that the driving behavior is normal, and vice versa, and a second-level warning signal is then sent to alert the driver about the abnormal driving behavior. The present invention can alert drivers to concentrate on driving before an abnormal driving behavior occurs, which effectively improves driving safety.

Description

Driving behavior detection method and system

The present invention relates to a behavior detection technology, in particular to a driving behavior detection method and system capable of reminding a driver to focus before driving distracting behavior.

With the advancement of technology, cars have gradually become a necessary means of transportation in common families, but the number of cars has gradually increased, making annual traffic accidents still high. According to statistical reports, the main causes of car accidents are mostly from For the driver, the cause may be the driver ’s poor driving behavior, fatigue, or illness, which ca n’t drive the vehicle. Among them, the bad driving behavior, such as using a mobile phone, taking things, taking food, etc. The distracted behavior of the driver's hand away from the steering wheel. Therefore, the current road traffic regulations stipulate that those who use handheld mobile devices or hold and smoke cigarettes that affect the driving safety of others will be punished to reduce the driver ’s bad behavior. Traffic accidents.

However, a monitoring device capable of detecting bad driving behavior has been produced to remind the driver to concentrate on driving when the driver has a distracted behavior. However, most current monitoring devices focus on using a camera to capture a face to perform facial characteristics. Warning judgment to judge whether the eye sight or the face is deflected. Or use other sensors to identify abnormal driving behaviors and determine whether to be distracted, such as using a smoke detection device to determine whether the driver is distracted to smoke or using a mobile phone signal detector to determine whether the driver is using a mobile phone. However, since these monitoring platforms are relatively expensive to set up, and the current technology can only detect when the driver has performed a distracting action, the monitoring device can only alert the driver after the driver has performed a distracting action. The driver's distraction has already been generated. Even if the driver is reminded by the monitoring device, the driver still influences the traffic safety of himself and other passers-by.

Furthermore, in recent years, the Internet of Things system has emerged. Through the Internet of Things system, various vehicle status information can be applied to implement fleet management. The information collected in the background can include the driving behavior status detected by the monitor on the driver's vehicle. When the monitor detects the driver's distraction, this information is transmitted to the background. The background can store the driving behavior information for the background management personnel to monitor, or for the performance management of commercial fleet drivers, but it also has The same shortcomings as the above monitoring device can only send distracting information to the background after the driver has performed distraction. Even if the background staff notices the driver immediately after discovery, the driver's distraction has still occurred, and the driver It still affects the traffic safety of itself and other passers-by.

In view of this, the present invention addresses the lack of the above-mentioned conventional technology, and proposes a method that can effectively detect the front-end actions of the driver to generate abnormal behaviors, so as to remind the driver to concentrate on driving before the phenomenon of unconscious behavior occurs. Behavior detection method and system to effectively overcome these problems.

The main object of the present invention is to provide a driving behavior detection method and a system thereof, which can effectively detect the front-end movement of the driver to produce abnormal behavior, and remind the driver to concentrate on driving before the driver may generate unfocused behavior, so as to effectively Improve driving safety.

Another object of the present invention is to provide a driving behavior detection method and system, which can transmit the driver's current behavior to a remote server, provide background management personnel for monitoring, and serve as a performance management for commercial fleet drivers. , Can effectively reduce the problem of unfocused driving, and has obvious benefits to driving safety.

In order to achieve the above object, the present invention provides a driving behavior detection method. The method includes the following steps: firstly obtaining a user image including an arm image and a head behavior image; and then performing an arm image and a plurality of arm sample images in a database. Compare to determine whether the arm image matches at least one arm sample image. If yes, the arm image matches the arm sample image, indicating that the driving behavior is normal, but if not, the arm image does not match the arm sample image, indicating that the driving behavior may be abnormal, then enter the next One step; generating a first-level warning signal and transmitting the first-level warning signal; comparing the head behavior image with the plurality of head violation sample images in the database, and determining whether the head behavior image meets at least one head If the head behavior image does not match the head violation sample image, it indicates that the driving behavior is normal; if it does, the head behavior image matches the head violation sample image, indicating that the driving behavior is abnormal, and a second-level warning is issued Signal to relay the second level warning signal.

In addition, the present invention also provides a driving behavior detection system, which includes a photographic device capturing at least one user image, including an arm image and a head behavior image, a database storing a plurality of arm sample images and a plurality of head violations. A sample image. A processor is electrically connected to the photography device and the database. The processor captures the arm image and compares it with a plurality of arm sample images in the database. When the arm image matches at least one arm sample image, it indicates that the driving behavior is normal; When the arm image does not match at least one arm sample image, it indicates that the driving behavior may be abnormal, that is, a first-level warning signal is generated, and the first-level warning signal is transmitted, and the processor retrieves the head behavior image and the database. The multiple head violation sample images are compared. When the head behavior image does not match at least one head violation sample image, the driving behavior is normal; when the head behavior image matches at least one head violation sample image, the driving behavior is abnormal, and a first Secondary warning signal; a transmission device is electrically connected to the processor to receive Controller will signal the first stage or the second stage alert warning signal to pass out.

When the processor captures an arm image that does not match at least one arm sample image, it indicates that the driving behavior may be abnormal, and the first-level warning signal is generated after the driving abnormal behavior continues for a predetermined time.

The processor can also use a head deflection algorithm to determine whether the user's head is deflected in the head behavior image. If the user's head is deflected, a second-level warning signal is issued to remind the driving behavior abnormally. ; If not, the user's head does not yaw, indicating that the driving behavior is normal.

Detailed descriptions will be provided below through specific embodiments to make it easier to understand the purpose, technical content, features and effects of the present invention.

First, please refer to the first figure to explain the structure of the driving behavior detection system 1 of the present invention. The driving behavior detection system 1 includes a photographing device 10, which can be installed in front of the driver's seat, so that the lens of the photographing device 10 is aligned. The driver in the driver's seat can obtain a user image of the driver's front, the user image includes an arm image and a head behavior image; a database 12 is used to store a plurality of arm sample images and head violation sample images, among which the arm The sample image is the image of the driver holding the steering wheel with both hands. The image contains the general image of the driver in various postures and angles when holding the steering wheel with both hands, such as holding the steering wheel with two hands in parallel, crossing his right hand on top, and crossing his left hand. The image of the state of the driver holding the steering wheel in a variety of high-end general. The head violation sample image is the driver ’s mobile phone or the driver ’s smoking, eating and other violations; a processor 14 is electrically connected to the camera 10 and the database 12, and the processor 14 may obtain the user image from the camera 10 And obtain the arm sample image and the head violation sample image from the database 12 to compare the arm image and the arm sample image of the user image, and the head behavior image and the head violation sample image to determine the driving behavior Whether it is normal and generates a judgment result; a transmission device 16 is electrically connected to the processor 14 to transmit the judgment result generated by the processor 14 to a computer host 20, wherein the computer host 20 may be a computer host used in a car or a remote end; Background, etc., in this embodiment, there are two computer hosts 20, which are a vehicle host 22 and a remote server 24. The vehicle host 22 can store the received judgment results and display the judgment results. The remote server 24 Can store the received judgment results, and display the judgment results to the monitoring staff of the remote server, and provide the monitoring staff to determine whether it should issue a warning to remind the driver Note.

After explaining the various roles in the system architecture of the present invention, the present invention continues the flowchart of the driving behavior detection method with the system device to explain the technical content of the present invention in more detail. Please refer to the first and second figures together First, before performing the driving behavior detection method, a plurality of arm sample images and a plurality of head violation sample images must be established in the database 12 so that the processor 14 can use the data in the database 12 for comparison. Next, the steps of the driving behavior detection method of the present invention are described. First, the method proceeds to step S10, and at least one user image is obtained through the photographing device 10. The user image includes an arm image and a head behavior image. Next, referring to step S12, the processor 14 captures the arm image and the plurality of arm sample images in the database 12 to start the comparison. During the comparison process, a Gaussian mixture model equation (GMM) is used to first compare the arms. The image is converted into an image that can be compared with the image of the complex arm sample, and compared by the processor 14, the Gaussian mixture model equation is as follows: among them Is a Gaussian distribution density function; A matrix formed by successive frames of arm positions in the arm image; for Average vector for Transposed conjugate matrix of consecutive frames; , versus For weight. The arm image is compared with the plural arm samples in the database 12 after being transformed by the Gaussian mixture model equation, thereby determining whether the arm image matches at least one arm sample image. If the processor 14 determines that the arm image matches at least one arm sample image, then Go to step S14, indicating that the driving behavior is normal, and the processor 14 can generate a normal driving behavior signal, and transmit the normal driving behavior signal to the remote server 24 through the transmission device 16 to store the normal driving behavior signal, and can inform the background monitoring The current driving behavior of the personnel is normal; however, if the processor 14 judges that the arm image does not match at least one arm sample image, it indicates that the driving behavior may be abnormal, and at this time, it proceeds to the next step S16. As shown in step S16, the processor 14 can further determine whether the abnormal driving behavior continues to occur for a predetermined time. If otherwise, the process proceeds to step S14 to indicate that the driving behavior is normal. At the same time, the processor 14 can generate a normal driving behavior signal and transmit the driving signal through the transmission device 16. The normal behavior signal is transmitted to the remote server 24 to inform the monitoring personnel that the current driving behavior is normal; but if it is, the processor 14 judges that the abnormal driving behavior continues to generate a predetermined time. In this embodiment, the predetermined time is 5 seconds, so when driving If the abnormal behavior lasts for 5 seconds, go to step S18 to generate a first-level warning signal, and transmit the first-level warning signal to the remote server 24 through the transmission device 16 to store the first-level warning signal and inform the background monitoring Personnel, provide monitoring personnel as the monitoring judgment, and at the same time proceed to the next step S19. Of course, the first-level warning signal can also be directly transmitted to the vehicle host 22 to directly remind the driver to concentrate on driving, and is not limited to transmitting to the remote server 24.

Please refer to step S19. The processor 14 then determines whether the head behavior image meets at least one head violation sample image. When determining the head behavior image, a Gaussian mixture model equation can also be used to convert the head behavior image into a head image The image of the illegal sample image comparison is then compared with the head violation sample image, but the comparison head behavior image in this embodiment is compared by means of feature points, and the processor 14 will capture the head behavior The image features in the image are used to compare the contrast features in the multiple head violation sample images. The contrast feature is a mobile phone, cigarette, or other items. The processor 14 determines whether the head behavior image has a mobile phone, cigarette, or If there are no image features of other items in the head behavior image, there is no contrast feature point that matches the head violation sample image, then the process proceeds to step S14 to indicate that the driving behavior is normal; but if the head behavior image has the contrast feature that matches the head violation sample image Point, indicating that the driving behavior is abnormal. The driver may have actions such as smoking or using a mobile phone. At this time, the process proceeds to step S20. The device 14 generates a second-level warning signal and transmits the second-level warning signal to the remote server 24 through the transmission device 16 to remind the background monitoring staff that the driver has been distracted. The monitoring staff can issue a warning alert The driver records the current abnormal behavior of the driver. At the same time, after the processor 14 generates the second-level warning signal, it can also be directly transmitted to the vehicle host 22 on the driver's car, so that the vehicle host 22 issues the second-level warning signal. An audible or visual alert reminds you that driving behavior is abnormal and you must concentrate on driving.

The above-mentioned head behavior image can be used to provide the processor 14 to compare the head violation sample images in the database 12 to determine whether the driver is in a normal driving behavior. The processor 14 can also be used before step S18 or Then, a head deflection algorithm is used to determine whether the driver's head is deflected in the head behavior image. When the processor 14 judges that the head is deviated, a second-level warning message is issued, and the first The second-level warning signal is transmitted to the remote server 24 to remind the background monitoring personnel, or the processor 14 can also control the transmission device 16 to directly transmit the second-level warning signal to the on-vehicle host 22 for direct reminder. The driver concentrates on driving; however, if the head does not have an offset, it indicates that the driving behavior is normal. At the same time, the processor 14 can generate a normal driving behavior signal and transmit the normal driving behavior signal to the remote server 24 through the transmission device 16, To inform the background monitoring staff that the current driving behavior is normal.

However, how to determine whether the head is deflected is determined by a head deflection algorithm. Please refer to Figures 3 and 4 to explain the head deflection algorithm, as shown in Figure 4. When judging the head deflection, you must first preset the area as a reasonable face area, and consider the reasonable face area as the face area, and the nose coordinates of the face area will be located in the middle of the screen. Next, please refer to step S22. The processor 14 detects the driver's nose in the current user image to detect the driver's nose in the user image. Then it proceeds to step S24 to obtain the coordinates of the position of the nose, which is about 50% of the X axis, and the estimated value of the X axis is about 140. Then proceed to step S26, after comparing the position coordinates of the nose with a reasonable face area, obtain the driver's head swing angle. When the driver's head slightly deviates from -15 to -30 degrees, the position of the nose will shift to the screen At 35% of the X axis, the nose will appear between the X-coordinates of 100 and 130. Conversely, if the X-axis result of nose detection falls between 100 and 130, the driver's head swing angle can be reversed between -30 and -15 degrees, so it can be judged by nose offset. Driver swings his head. Finally, the process proceeds to step S28. When the head swing angle is higher than a preset threshold, for example, when the preset threshold value is higher than 15 degrees, the head is deflected. In addition, another severe deflection preset threshold may be added. When the head swing angle is 30 degrees higher than the preset threshold for severe swing, it is a serious swing.

In addition to the above method embodiment, please refer to the fifth figure. After determining that the arm image does not meet at least one arm sample image indicating that the driving behavior may be abnormal, the processor 14 may not need to determine whether the abnormal driving behavior continues to generate a preset time. . In detail, as shown in the fifth figure, steps S30 to S34 are the same as steps S10 to S14 and therefore will not be repeated. The difference is that when it is determined that the arm image does not match at least one arm sample image, it is directly entered. Step S36 generates a first-level warning signal, which does not need to be the same as that in step S16 of the above-mentioned embodiment. It must be judged whether the abnormal driving behavior continues to generate a preset time. In this embodiment, the first-level warning signal can be generated directly through the transmission device 16 Pass the first-level warning signal to the remote server 24, so that the remote server 24 stores the first-level warning signal and informs the background monitoring staff, so as to provide the monitoring staff as a monitoring judgment. The following steps S38 to S40 are the same as steps S19 to S20, so the description will not be repeated.

In summary, since the sample images of the multiple arms are images of the driver holding the steering wheel with both hands, if the arm images do not correspond to the sample images of the arms, it means that the driver may leave the steering wheel with his arm. The reason for leaving the steering wheel may be to take a mobile phone, Cigarettes, etc., therefore, when the driver's hand leaves the steering wheel, most of them are unfocused front-end actions. Therefore, the present invention can effectively detect the front-end actions of the driver to produce abnormal behaviors, so that the driver may have unfocused behaviors before they occur. Remind the driver to note that it can effectively improve driving safety, and transfer the driver's current behavior to a remote server, and provide back-office management personnel to monitor, which can be used for the performance management of commercial fleet drivers, which has obvious benefits for driving safety .

The foregoing are merely preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, all equal changes or modifications made according to the features and spirit described in the scope of the application of the present invention shall be included in the scope of patent application of the present invention.

1‧‧‧Driving Behavior Detection System

10‧‧‧ Camera

12‧‧‧Database

14‧‧‧ processor

16‧‧‧Transmission device

20‧‧‧Computer host

22‧‧‧Host

24‧‧‧ remote server

The first figure is a block diagram of the detection system of the present invention. The second figure is a flowchart of the detection method steps of the present invention. The third figure is a flowchart of the steps of the head deflection algorithm used in the present invention. The fourth figure is a schematic diagram of the image used for judging the head swing of the present invention. The fifth figure is a flowchart of the detection method steps according to another embodiment of the present invention.

Claims (19)

A driving behavior detection method includes the following steps: obtaining at least one user image including an arm image and a head behavior image; comparing the arm image with a plurality of arm sample images in a database to determine the arm Whether the image meets at least one sample image of the arm: if yes, it indicates that the driving behavior is normal; and if not, it indicates that the driving behavior is abnormal and proceed to the next step; generate a first-level warning signal and pass the first-level warning signal ; And compare the head behavior image with the plurality of head violation sample images in the database, and determine whether the head behavior image meets at least one of the head violation sample images: if not, the driving behavior is normal; and if it is , Indicating that the driving behavior is abnormal, and a second-level warning signal is issued to transmit the second-level warning signal. The driving behavior detection method according to claim 1, wherein before the step of generating the first-level warning signal, the first-level warning signal is generated when the abnormal driving behavior continues to be generated for a predetermined time, and the predetermined time may be For 5 seconds. The driving behavior detection method according to claim 1, wherein when comparing the arm image with the arm sample images, a Gaussian mixture model (GMM) is used to convert the arm image into Images that can be compared with these arm sample images for comparison. The driving behavior detection method according to claim 1, wherein in the step of comparing the head behavior image with the head violation sample images, the image features in the head behavior image are captured to compare the Contrast features in images of head violation samples. The driving behavior detection method according to claim 1, further comprising, after the step of comparing the head behavior image and the head violation sample images, using a head deflection algorithm to determine the head behavior image Whether the user ’s head is tilted: If yes, issue the second-level warning signal to remind the driving behavior is abnormal; and if not, it indicates that the driving behavior is normal. The driving behavior detection method according to claim 5, wherein the steps of the head deflection algorithm include: detecting the user's nose on the current user image; obtaining the position coordinates of the nose; comparing the After the position coordinates of the nose and a reasonable face area, the driver ’s head swing angle is obtained; and when the head swing angle is higher than a preset threshold value, it indicates that the head is deflected, and the preset threshold value may be 15 degrees. The driving behavior detection method according to claim 6, wherein the reasonable face area is a preset area or a face area facing the face. The driving behavior detection method according to claim 1, wherein the arm sample images and the head violation sample images in the database are established before the step of capturing the user image. The driving behavior detection method according to claim 1, wherein the first-level warning signal and the second-level warning signal are transmitted to at least one computer host, and the computer host may be a vehicle host or a remote server. A driving behavior detection system includes: a photographing device that captures at least one user image including an arm image and a head behavior image; a database storing a plurality of arm sample images and a plurality of head violation sample images; And a processor, which is electrically connected to the photographing device and the database, the processor captures the arm image and compares them with the arm sample images in the database, and when the arm image matches at least one of the arm sample images Indicates that the driving behavior is normal; when the arm image does not match at least one sample image of the arm, it indicates that the driving behavior is abnormal, that is, a first-level warning signal is generated, and the first-level warning signal is transmitted, and the processor then Capture the head behavior image and compare the head violation sample images in the database. When the head behavior image does not meet at least one of the head violation sample images, it indicates that the driving behavior is normal; when the head behavior The image matches at least one sample of the violation of the head, indicating that the driving behavior is abnormal, and a second-level warning signal is issued to Level warning signal to pass out. The driving behavior detection system according to claim 10, wherein the processor generates the first-level warning signal after the driving abnormal behavior continues for a predetermined time, and the predetermined time may be 5 seconds. The driving behavior detection system according to claim 10, further comprising a transmission device electrically connected to the processor, and receiving control of the processor to transmit the first-level warning signal or the second-level warning signal. The driving behavior detection system according to claim 12, further comprising at least one computer host receiving the first-level warning signal and the second-level warning signal transmitted by the transmission device, so as to receive the first-level warning signal or the first-level warning signal. The secondary warning signal sends out an audible warning signal or a visual warning signal. The driving behavior detection system according to claim 13, wherein the computer host is a vehicle host or a remote server. The driving behavior detection system according to claim 10, wherein the processor uses a Gaussian mixture model (GMM) to convert the arm image into an image that can be compared with the arm sample images for processing. Comparison of the arm image with the arm sample images. The driving behavior detection system according to claim 10, wherein the processor compares the contrast features in the sample images of the head violations by capturing the image features in the behavior images of the head to compare the features in the sample images of the violations of the head. Behavior images and sample images of these head violations. The driving behavior detection system according to claim 11, wherein the processor can further use a head deflection algorithm to determine whether the user's head is deflected in the head behavior image: if yes, the processor issues The second-level warning signal to remind the driving behavior is abnormal; if not, it indicates that the driving behavior is normal. The driving behavior detection system according to claim 17, wherein the head deflection algorithm is to detect the nose of the user from the current user image, obtain the position coordinates of the nose, and then compare the nose position After the position coordinates and a reasonable face area, the driver's head swing angle is obtained. When the head swing angle is higher than a preset threshold value, it indicates that the head is deflected, wherein the preset threshold value can be 15 degrees . The driving behavior detection system according to claim 18, wherein the reasonable face area is a preset area or a face area facing the face.