CN1680779A - Driver fatigue monitoring method and device - Google Patents

Abstract

A method for monitoring fatigue strength of driver includes shining driver eye with infrared ray to obtain multiimage of different retina image at the same time, carrying out difference processing for collected images to obtain pupil images, using kalmen filter to trace pupil in real - time to obtain pupil characteristic parameter, processing the parameter to obtain maximum value of pupil size and real - time coroclisis percentage of pupil, calculating out PERCLOS value f and using BP network sorter to judge fatigue strength of driver based on obtained value f. The monitoring device is also disclosed.

Description

Driver fatigue monitoring method and device

technical field

The invention relates to transportation engineering, in particular to a driver fatigue monitoring method and device.

Background technique

At present, the recognition and monitoring technology for driver's eye fatigue characteristics is mainly based on monitoring the driver's mouth state to understand his behavior status and provide necessary auxiliary information for safe driving. Related documents include Shi Shuming, Jin Bensheng, Wang Rongben, Tong Bingliang, Journal of Jilin University (Engineering Edition), Vol. There is a certain difference in the degree of mouth opening of drivers in the three states of normal driving, talking and yawning (sleeping). According to this feature, the author uses the Fisher classifier to extract the contour and position of the lips, and then uses the geometric features of the lip area as the feature value to form a feature vector, which is used as the input of the three-layer BP neural network to combine normal driving, speaking and yawning ( Drowsiness) three different mental states as output.

However, due to the limitations of the method itself, firstly, it cannot meet the all-weather requirements due to the influence of light brightness, etc., and secondly, in the background technology, the single-frame image of the mouth is recognized. Other actions are confused, and the recognition rate is not high.

Contents of the invention

In view of the above-mentioned deficiencies, the present invention proposes a driver fatigue monitoring method and device aimed at the driver's eye fatigue characteristics. The use of infrared rays to irradiate the driver's eyes meets the all-weather requirements without affecting the driver's normal driving; the PERCLOS (Percentage of Eyelid Closure Over the Pupil Over Time) index is used as the judgment standard, and the BP network is adopted As an aid to the evaluation criteria, the classifier further improves the accuracy of the evaluation and makes up for the limitations of the experimental data, so that it can better adapt to different groups of people.

The technical solution to achieve the above goals is based on the method of infrared light source, differential image, and KALMAN filter, and a system prototype is designed. The test shows that it can fully meet the requirements of real-time, all-weather, and high recognition rate.

Main technical solutions:

①Image acquisition:

Using infrared rays to irradiate the driver's eyes, a plurality of CMOS cameras are used to obtain multiple images with different retinal images at the same time.

The device for realizing the above method is mainly composed of an infrared light source, a CMOS camera, a control board and corresponding software parts. It utilizes 2 separate cameras, crossed at 90°. When the image passes through a beam splitter, it is divided into two beams and enters the lenses of the two cameras respectively. Then, the two lenses respectively use filters with wavelengths of 850nm and 950nm to obtain corresponding infrared images. The result is 2 images that differ only in the retinal image at the same time.

②Image processing, real-time tracking:

The collected image signal is differentially processed by the built-in image processing program in the control board to obtain the pupil image. At the same time, the neural network-assisted Kalman filter is used to track and predict the pupil in real time.

③Calculation matching:

The obtained characteristic parameters of the pupil are processed by the control unit, and the maximum pupil size and the real-time pupil closure percentage are obtained through statistical processing, and the PERCLOS value f is calculated, and then the fatigue degree of the driver is judged.

PERCLOS refers to the percentage of eye closure time in a certain time, and the P80 of PERCLOS (percentage of time with eyes closed more than 80% in the total time per unit time) has the best correlation with driving fatigue.

As long as the values of t ₁ ~ t ₄ are measured, f can be calculated:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="A20051003777100081.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}$

Among them, f is the percentage of the time that the eyes are closed more than 80% in a certain time.

④ Fatigue evaluation:

The evaluation standard adopted by the system is the P80 standard of PERCLOS (Percentage of Eyelid Closure Over the Pupil OverTime). At the same time, the BP network classifier is used as an auxiliary evaluation standard. The BP network based on the regional geometric feature neural network algorithm has a 3-layer structure, and the input layer has 4 neurons, which respectively represent the eigenvalues t ₁ ~ t ₄ in PERCLOS. There are 10 neurons in the hidden layer and 3 neurons in the output layer, which represent 3 different states of the eigenvalue f in PERCLOS, and the transfer function of the hidden layer is the Sigmoid function. The output vectors of the network are Y ₁ =[1,0,0], Y ₂ =[0,1,0], Y ₃ =[0,0,1]. Among them, X ₁ ~ X ₄ represent t ₁ ~ t ₄ , Y ₁ represents that the f value is too small, Y ₂ represents that the f value is appropriate, and Y ₃ represents that the f value is too large.

The beneficial effects of the present invention are:

① Machine vision is used to track and monitor the driver's eyes, avoiding direct physical contact with the driver;

② In the current research on monitoring fatigue by detecting pupils, the PERCLOS method with the best correlation with Pearson is used;

③The use of infrared imaging greatly improves the applicability of the device and meets the monitoring requirements of the driver's state in any driving situation;

④Using the neural network to assist the Kalman filter to process the collected eye feature parameters can well realize the tracking and prediction of the driver's eyes, and effectively solve the problem of identifying the eyes when the driver's head shakes ;

⑤The system adopts a highly integrated CMOS camera sensor and a control board based on a DSP processor, which is easy to integrate with the original circuit in the car;

⑥The BP network classifier, as an aid to the evaluation criteria, can further improve the accuracy of the evaluation and make up for the limitations of the experimental data, so that it can better adapt to different groups of people.

Description of drawings

Figure 1 Device composition and detection flow diagram

Figure 2 Structural diagram of the PERCLOS camera

Figure 3 Structure diagram of BP network classifier

Fig.4 Schematic diagram of the measurement principle of PERCLOS value f

1-fan; 2-control board; 3-950nm filter; 4-950nm filter; 5-beam splitter

Detailed ways

As shown in the figure, this embodiment is mainly composed of an infrared light source, a CMOS camera, a control board and corresponding software parts. The camera is installed in front of the driver so as not to affect the driver's field of vision. The camera uses CMOS Complementary Metal-Oxide-Semiconductor (Complementary Metal-Oxide-Semiconductor) as the sensor.

Considering the applicability, the basic physiological characteristics of the human eye are used, that is, the amount of reflection of the retina to different wavelengths of infrared light is different. At 850nm wavelength, it can reflect 90% of the incident light, and at 950nm the retina can only reflect 40% of the incident light. Under the same illuminance, two cameras measure the images of the human eye at the same time, one is an image of 850nm wavelength, the other is an image of 950nm, and the result of subtraction of the two images leaves only the image of the retinal position, and then Then analyze the size and position of the retina.

In order to obtain two identical images of light sources with different wavelengths, two separate cameras are used to intersect at 90°. When the image passes through a beam splitter, it is divided into two beams and enters the lenses of the two cameras respectively. Then, the two lenses respectively use filters with wavelengths of 850nm and 950nm to obtain corresponding infrared images. The result is 2 images that differ only in the retinal image at the same time. The structure of the camera is shown in Figure 2.

Due to the use of infrared light sources, on the one hand, it will not affect the driver's driving operation, on the other hand, it can effectively meet the requirements of all-weather.

The collected image signal is differentially processed by the built-in image processing program in the control board to obtain the pupil image. At the same time, the neural network-assisted Kalman filter is used to track and predict the pupil in real time.

The obtained characteristic parameters of the pupil are processed by the control unit, and the maximum pupil size and the real-time pupil closure percentage are obtained through statistical processing, and the PERCLOS value f is calculated, and then the fatigue degree of the driver is judged. The measurement principle of the PERCLOS value f is shown in Figure 4.

As long as the values of t ₁ ~ t ₄ are measured, f can be calculated:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="A20051003777100081.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}$

Among them, f is the percentage of the time that the eyes are closed more than 80% in a certain time.

The evaluation standard adopted by the system is the P80 standard of PERCLOS (Percentage of Eyelid Closure Over the Pupil OverTime). At the same time, the BP network classifier is used as an auxiliary evaluation standard. The BP network based on the regional geometric feature neural network algorithm has a 3-layer structure, and the input layer has 4 neurons, which respectively represent the eigenvalues t ₁ ~ t ₄ in PERCLOS. There are 10 neurons in the hidden layer and 3 neurons in the output layer, which represent 3 different states of the eigenvalue f in PERCLOS, and the transfer function of the hidden layer is the Sigmoid function. The output vectors of the network are Y ₁ =[1,0,0], Y ₂ =[0,1,0], Y ₃ =[0,0,1]. Among them, X ₁ ~ X ₄ represent t ₁ ~ t ₄ , Y ₁ represents that the f value is too small, Y ₂ represents that the f value is appropriate, and Y ₃ represents that the f value is too large. The structure of the neural network is shown in Figure 3.

The system collects infrared images with a wavelength of 850nm/950nm and differential images of the eye.

Claims (2)

1. a driver fatigue monitoring method is characterized in that: utilize the irradiation of Infrared to driver's eye, obtain several by a plurality of CMOS cameras and have only the different image of retinal images at synchronization; The picture signal that collects is carried out difference processing by image processing program built-in in the control main board, obtains pupil image; Utilize the auxiliary Kalman wave filter of neural network that pupil is carried out the real-time follow-up prediction simultaneously; The characteristic parameter of the pupil that obtains is handled by control module, obtain the maximal value of pupil size and real-time coreclisis number percent by statistical treatment, calculate PERCLOS value f, and utilize the supplementary means of BP network classifier simultaneously as evaluation criterion, judge driver's degree of fatigue.

2. realize the device of the described driver fatigue monitoring method of claim 1, it is characterized in that: device is made of the CMOS camera of 2 separation, 90 ° of intersections infrared light supply, CMOS camera, control main board, be installed in the place ahead of driver, the CMOS camera through signal wire links to each other with mainboard.

Images