CN103440476A - A pupil location method in face video - Google Patents

Abstract

The invention discloses a method for positioning pupils in a face video, and belongs to the technical field of signal processing. A positioning method of pupils in a face video comprises a face detection module, an image preprocessing module, a human eye coarse positioning module and a pupil fine positioning module; the input video image is processed by a face detection module, an image preprocessing module, a human eye coarse positioning module and a pupil fine positioning module to finally obtain the position of the pupil center.

Description

A pupil location method in face video

technical field

The invention relates to a pupil positioning method in a human face video, belonging to the technical field of signal processing.

Background technique

The pupil localization of facial images has important application prospects in the fields of facial image processing, eye movement human-computer interaction and so on. Pupil location methods mainly include region segmentation method, edge extraction method, gray projection method, template matching method, Adaboost method and so on. The region segmentation method is easily interfered by the glasses, and the effect is rough; the feature extraction method actually uses the Hough transform to find an eye template, but this requires a lot of preprocessing, and it and some of its improved algorithms have to face the problems caused by lenses and eyelashes, etc. The interference caused by it; the grayscale projection method is a fast algorithm that projects the image to two coordinate axes and locates the eye position according to the peak and valley values of the projection, but it only relies on two-dimensional projection, for black-rimmed glasses , eyebrows, hair, etc. are difficult to distinguish; the template matching method needs to normalize the scale and direction of the face image, and the template needs to be obtained through training, so the calculation is relatively large; the Adaboost algorithm based on sample training has certain advantages in eye positioning , it strictly requires abundant training samples, but eyebrows with high gray values may actually be determined as eyeballs. In addition, the size of the candidate search window limits the size of training samples, which will also lead to low-resolution images. The recognition rate is reduced.

Contents of the invention

In order to overcome the above-mentioned shortcomings, the object of the present invention is to provide a method for locating pupils in human face videos.

The technical scheme that the present invention takes is as follows:

A method for locating pupils in a face video includes a face detection module, an image preprocessing module, a coarse human eye positioning module, and a fine pupil positioning module; The positioning module and the pupil fine positioning module finally obtain the position of the pupil center.

Principles and beneficial effects of the present invention: The method mentioned in the background technology does not utilize the characteristics of the pupil, so its performance is restricted by the image quality of the eye. If the image quality is not good, the positioning accuracy will decrease. When positioning the pupil of the human eye, glasses, eyebrows, eyelashes, hair, etc. will interfere with it. In order to improve the performance of pupil location, it is necessary to make full use of the characteristics of the pupil image, such as the shape of the pupil is round, the color is darker, the corresponding gray value is lower, and the pupil is located in the upper half of the face. The invention utilizes the radial symmetry characteristic of the pupil, and proposes a pupil positioning method based on integral projection and radial symmetric transformation, so as to improve the pupil positioning performance.

Description of drawings

Fig.1 Block diagram of radial symmetric transformation method.

Figure 2 Mapping relationship diagram of pixels.

Fig. 3 is a flow chart of "Iris and pupil positioning method based on human eye structure classification" used by Huang Xinyu and Yang Ruigang in the Chinese patent publication number 201210393147.2.

Figure 4 Yefei Chen and Jianbo Su in the paper "Fast eye localization based on a new haar-like feature" (10th World Congress on Intelligent Control and Automation, Beijing, China. 2012, 4825-4830) based on the new haar-like feature Flow chart of rapid human eye localization.

Fig. 5 is a functional block diagram of the technical solution of the present invention.

Figure 6 uses Mikael Nilsson, J.Nordberg, Ingvar Claesson in the paper "Face detection using local SMQT features and split up SNOW classifier" (IEEE International Conference on Acoustics, Speech, and Signal Processing, Honolulu, USA.2007, 589-592) The face detection result graph of the face detection method.

Figure 7. The face graph in Figure 6.

Fig. 8 is the image after median filtering of the face image in Fig. 7 .

Figure 9 is the image after performing histogram equalization on the image in Figure 8 .

Figure 10 The vertical projection curve of the image in Figure 9.

Figure 11 The left and right eye areas in Figure 9.

Figure 12 Image of a single human eye area.

Fig. 13 Results of pupil positioning for a single human eye.

Fig. 14 Results of pupil positioning for both eyes.

Detailed ways

The present invention will be further described below in conjunction with accompanying drawing:

(1) Radial symmetric transformation

Radial symmetric transformation is developed on the basis of generalized symmetric transformation. It is a gradient-based target detection operator. It can simply and quickly detect pixels with radial symmetry characteristics, and realize the detection of circular targets. effective detection.

Generally speaking, given the radius n of a circular target (n∈N, N is the set of radii of the radially symmetric features to be detected), the corresponding result of the radially symmetric transformation can be obtained. The value of this result at point P represents the degree of radial symmetry in the image at this point, that is, how likely the image is to contain a circle with point P as the center and n as the radius. With the increase of the detection radius n, the region with high symmetry can quickly accumulate a larger radial symmetric intensity value S with the radial symmetry characteristic, and realize the detection of the circular region. The block diagram of the radial symmetric transformation method is shown in Figure 1.

Convolute the image I with the Sobel horizontal operator and vertical operator to calculate the edge gradient image

g(p)＝[g _x (p), g _y (p)]＝|[I*Sobel _hor ,I*Sobel _ver ]|,

Among them, '*' means convolution, Sobel _hor is the Sobel horizontal operator, and Sobel _ver is the Sobel vertical operator.

${\mathrm{<span\; class="notranslate">\; Sobel</span>}}_{\mathrm{<span\; class="notranslate">\; hor</span>}}<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 2</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; Sobel</span>}}_{\mathrm{<span\; class="notranslate">\; ver</span>}}<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; ,</span>$

For each radius n, the corresponding direction projection image O _n and magnitude projection image M _n can be calculated. It can be seen from Figure 2 that for a given point P, O _n and M _n are calculated by the positive and negative influence pixels P _+ve and P _-ve , and P _+ve and P _-ve are functions of the gradient g(p) . The definitions of positive and negative impact pixels are different. A positive impact pixel is the coordinate of a point pointed by the gradient vector g(p) and the length of the point P is n; a negative impact pixel is the distance point P pointed by the gradient vector g(p) in the negative direction. The coordinates of a point of length n.

From the gradient vector g(p), the positively affected pixel P _+ve and the negatively affected pixel P _-ve can be calculated, namely

${\mathrm{<span\; class="notranslate">\; P</span>}}_{<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; ve</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; round</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; P</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; ve</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; round</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

Among them, "round" means to round all the elements in the vector to the integer closest to them. |·| means vector modulo.

Both the direction projection image O _n and the gradient projection image M _n are initialized to 0. For each pair of affected pixels, the value of the direction projection image O _n at the point P _+ve is increased by 1, and the value of the magnitude projection image M _n at the point P _+ve is increased by |g(p)|; correspondingly, the direction The value of the projection image O _n at the point P _-ve minus 1, and the value of the magnitude projection image M _n at the point P _-ve minus |g(p)|, namely

_On [P _+ve (p)] = _On [P _+ve (p)] + 1,

_On [P _-ve (p)]= _On [P _-ve (p)]-1,

M _n [P _+ve (p)]=M _n [P _+ve (p)]+|g(p)|,

M _n [P _-ve (p)] = M _n [P _-ve (p)] -|g(p)|,

It can be seen that _On reflects the number of pixels around the point P mapped to the point along its gradient direction, and M _n reflects the superposition of the gradient of the points around the point P on this point.

When the detection radius is n, the radially symmetric intensity value _Sn is defined as

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}{<span\; class="notranslate">\; [</span>\frac{{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; o</span>}}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$

S _n =F _n *A _n ,

Among them, k _n is _{used to normalize the scale factors of On and M n obtained} at different radii. α is the radial control parameter. '*' means convolution, and _An is a two-dimensional Gaussian matrix.

The final radially symmetric transformation S is the mean of the radially symmetric results S _n corresponding to all detection radii n

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; |</span>}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

The present invention will use the radial symmetry transform good circle detection characteristics for precise pupil location.

Prior art relevant to the present invention one

Technical solution of prior art one

Huang Xinyu and Yang Ruigang proposed an iris and pupil positioning method based on human eye structure classification in the Chinese invention patent "Iris and pupil positioning method based on human eye structure classification" with the publication number 201210393147.2. The flow chart of the method is shown in the figure 3. The invention uses unsupervised learning technology to automatically classify the structure of human eye images before accurately locating the iris and pupil boundaries. This classification can roughly locate the iris and pupil, estimate the size of the effective iris region, effectively remove outlier data that are not borders of the iris and pupil, and reduce the search space for iris and pupil locations and sizes. In the narrowed search space, the invention further performs constrained optimization according to the inherent characteristics of the iris and pupil to search for the optimal iris and pupil boundary. The invention can increase the stability and accuracy of iris and pupil positioning, and is especially suitable for long-distance non-invasive iris acquisition systems.

The shortcoming of prior art one

This method requires training and strictly requires abundant training samples, but eyebrows with high gray values may actually be identified as eyeballs, which will lead to a decrease in recognition rate.

Related prior art 2 of the present invention

Technical scheme of prior art 2

Yefei Chen and Jianbo Su proposed a fast pupil localization method in the paper "Fast eye localization based on a new haar-like feature" (10th World Congress on Intelligent Control and Automation, Beijing, China. 2012, 4825-4830). According to the prior proportional relationship of facial features, the method first selects a suitable candidate window in the detected face area; then uses histogram equalization technology in the candidate area to remove the lighting effect; finally, the method also proposes a A new Haar-like feature is used to quickly and accurately locate the pupil of the candidate region. The method is simple, requires no training, and is robust to distractions caused by eyebrows, hair, glasses, etc. The flow chart of the method is shown in Fig. 4 .

The shortcoming of prior art two

The main problems of the new haar-like feature-based rapid human eye positioning method are: (1) the error rate of positioning is high in the case of reflection of glasses or thick black eyebrows; (2) the method is not sensitive to light The stickiness is still low; (3) The method is only effective for face images with pose changes of ±20 degrees.

Detailed elaboration of the technical solution of the present invention

Technical problem to be solved by the present invention

The invention processes the face image, removes the influence of glasses, eyebrows, hair, light and other interference, and automatically and robustly locates the pupil position of the target human eye image, thereby providing an accurate basis for follow-up research such as face recognition and eye tracking. data.

Complete technical solution provided by the present invention

The present invention first obtains the face area in the image through the face detection method; then, according to the prior knowledge such as the pupil position, the integral projection method is used to obtain the approximate eyebrow area, that is, the rough positioning of the human eye; finally, the radial symmetric transformation is used Circular object detection method to precisely locate human eyes. The block diagram of the technical solution of the present invention is shown in Figure 5, and the solution mainly includes a face detection module, an image preprocessing module, a human eye coarse positioning module and a pupil fine positioning module.

Face detection module; the processing method of the face detection module is: the input of this module is the target image Iorig, using Mikael Nilsson, J.Nordberg, Ingvar Claesson in the paper "Face detection using local SMQT features and split up SNOW classifier" (IEEE InternationalConference on Acoustics, Speech, and Signal Processing, Honolulu, USA.2007, 589-592) based on local continuous mean quantization transform (Successive Mean Quantization Transform, SMQT) and sparse screening network (Sparse Network of Winnows, SNoW) classifier The face detection method, detects the face image Iface containing eyes, and takes it as the output of the module.

Image preprocessing module, the processing method of image preprocessing module is: the input of this module is face detection result Iface, after carrying out preprocessing such as median filtering and histogram equalization to Iface, the image I after output preprocessing.

(1) Median filtering

A 3×3 window is used to perform median filtering on the face image, so that each point in the image corresponds to a 3×3 window, and the point is located in the center of the window. The 8 points around the center point of the window are sorted by the gray value from large to small, and finally the median value of the gray value of each point in the window, that is, the average value of the gray value of the middle two points after sorting, is used to replace the center of the window The gray value of the point, that is, the median filter is completed, and the obtained image is I _temp .

(2) Histogram equalization

Perform histogram equalization on the median-filtered image I _temp . Histogram equalization is to map an image to another with a balanced histogram through grayscale transformation, and the mapping function is a cumulative distribution function. The specific steps are as follows:

1) First calculate the normalized gray histogram H of I _temp .

i=0,1,…,255

i=0,1,...,255

Among them, n _i is the number of pixels with gray level i in the image, i=0,1,...,255, and n is the total number of pixels in the image.

2) Then calculate the histogram integral H',

${\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>$

3) Finally calculate the image I after histogram equalization.

I(x,y)=H'[I _temp (x,y)]

Human eye rough positioning module, the processing method of the human eye rough positioning module is: the input of this module is the preprocessed image I, vertical projection is performed on I, and the coordinates of the peak at 1/3～1/2 of the vertical projection curve are obtained, It is the middle part of the human nose. Using the coordinates to intercept the Iface in the vertical direction, the approximate human eye area of the human eye image can be obtained. Finally, the human eye area is segmented with the center of the width of the human eye area as the boundary, and the two eyes can be separated to obtain the output left eye area Ileft and right eye area Iright of the module.

The specific process of human eye coarse positioning is as follows:

(1) Use formula (20) to calculate the vertical grayscale projection curve P _y (x) of I.

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 20</span>}<span\; class="notranslate">\; )</span>$

Among them, w is the width of the image.

(2) The coordinates of the peak value of P _y (x) at 1/3 to 1/2, which is the middle of the nose of the human face. Use this coordinate to intercept the I _face in the vertical direction to obtain a rough human eye area.

(3) Taking the center of the width of the human eye region as the boundary, the human eye region is divided into half in the vertical direction, and the two eyes can be separated to obtain the left eye region I _left and the right eye region I _right .

Pupil Location Module

The input of this module is the eye regions I _left and I _right obtained by the rough positioning of the human eye, and the pupils of I _left and I _right are precisely positioned respectively by radial symmetric transformation, and the final pupil positioning result is obtained. Radial symmetric transformation is an effective operator in the detection of radially symmetric regions. It can detect bright and dark points respectively by adjusting the direction of the gray gradient. The gradient direction has positive and negative directions, and the positive direction is from the dark point to the bright point, and the negative direction is from the bright point to the dark point, which is in line with the characteristics of the pupil. In order to be able to detect the pupil region effectively, it is necessary to make full use of the circular symmetry property of the pupil to calculate for each pixel the pixels it can affect in the negative direction, because they point to the pupil in the negative direction. As the detection radius grows, all pixels at the edge of the pupil converge toward the center of the pupil in the negative direction. However, the pixels at the edge of the bright spot will not converge to the center in the negative direction, because they are away from the center in the negative direction. In addition, eyelashes, hair, and spectacle frames do not have radial symmetry features, and their influence in the negative direction will not be detected. This detection technology can effectively avoid the above noise interference. The specific calculation steps are as follows:

(1) Convolute the eye image I _left or I _right obtained by the human eye coarse positioning module with the Sobel horizontal operator and vertical operator respectively, and calculate the edge gradient image g(p)=[g _x (p),g _y (p)].

(2) Initialize the initial search radius N=5, S _{n_max} =0, S _max =0.

(3) If S _max ≥ S _{n_max} is satisfied, perform the following steps:

(a) S _max = S _{n_max} ;

(b) For n=2: N

First calculate P _-ve (p) according to formula (5), and can get O _n (P _-ve (p)) and M _n (P _-ve (p)) according to formula (7) and formula (9); then Calculate F _n (p) according to formula (11), where parameter α=2, k _n =9.9; finally get S _n according to formula (10), where A _n is a two-dimensional Gaussian matrix with a size of n×n, Variance σ=0.25n;

(c) Calculate S according to formula (13), and select the largest value as the center of the pupil;

(d) N=N+2.

(4) If S _max <S _{n_max} , the iterative search ends, and the position of the pupil center is obtained.

Beneficial effects brought by the technical solution of the present invention

The CAS-PEAL face image database was created by the computer of the Chinese Academy of Sciences. It contains a total of 99,450 head and shoulder images of 1,040 Chinese people. All images are collected in a special collection environment, covering four types of posture, expression, accessories and lighting. The main changing conditions. The IMM face database was created by the Technical University of Denmark, which contains 240 face images with different poses, expressions, and lighting. Select 500 images from the IMM face bank and the CAS-PEAL face bank to evaluate the present invention. These 500 images include from frontal posture to side posture, from long hair to hat, from no glasses to different Various human faces such as glasses.

Using Mikael Nilsson, J.Nordberg, Ingvar Claesson in the document "Face detection using local SMQT features and split up SNOW classifier" (IEEE International Conference on Acoustics, Speech, and Signal Processing, Honolulu, USA.2007, 589-592) The face detection method detects human faces, and the results are shown in Figure 6. Perform preprocessing such as median filtering and histogram equalization on the detected face image, wherein, Figure 7 is the face image before processing, Figure 8 is the image after median filtering, and Figure 9 is the image after histogram equalization .

Calculate the vertical projection curve of the image after histogram equalization, as shown in Figure 10, due to the low gray value of the eye image and the high gray value of the nose image, an obvious "valley-peak-valley" will be formed on the vertical projection curve ", corresponding to the human eye area; the abscissa corresponding to the "peak" is the position of the middle of the nose; the two "valleys" correspond to the left and right eye areas. As shown in Figure 11.

The pupil location of the single human eye area shown in Figure 12 is performed using radial symmetric transformation, and the processing results are shown in Figure 13. After positioning the two eyes respectively, the final positioning result obtained is shown in FIG. 14 . The green cross mark in the figure is the final pupil positioning result.

Table 1 shows the performance of pupil location. It can be seen from Table 1 that the method proposed by the present invention can detect at least one pupil in almost all pictures, and only two pupils in 8 pictures are located incorrectly. In addition, the rate of accurate positioning of both pupils was 95.4%. Table 2 shows the distribution law of the pupil positions detected by the present invention compared with the accurate pupil positions. Here, we no longer consider the whole picture, but consider the left and right eyes separately, then the number of all eyes is 500*2=1000. It can be seen from Table 2 that the number of pupils whose positioning error is within 5 pixels accounts for 95.5%, while the number of pupils whose positioning error exceeds 10 pixels only accounts for 2.6%.

Table 1 Pupil positioning performance

eye positioning Positioning accuracy Correct positioning of at least one pupil 98.2% (491/500) Both pupils are positioned accurately 95.4% (477/500) Both pupil positioning is inaccurate 1.8% (9/500)

Table 2 Detection rate of different positioning errors

Distance from exact pupil position quantity within 5 pixels 955 (95.5%) 5 to 10 pixels 19 (1.9%) 10 to 15 pixels 8(0.8%) 15 to 20 pixels 6(0.6%) 20 pixels or more 12 (1.2%)

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any person familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention Any equivalent replacement or change of the inventive concepts thereof shall fall within the protection scope of the present invention.

Abbreviations and key terms involved in the present invention are defined as follows:

RST: Radial Symmetry Transform, radial symmetric transformation.

SMQT: Successive Mean Quantization Transform, continuous mean quantization transform.

SNoW: Sparse Network of Winnows, sparse screening network.

Claims (5)

1. a positioning method of pupil in a human face video, is characterized in that: comprise human face detection module, image preprocessing module, human eye rough positioning module and pupil fine positioning module; The video image of input is through human face detection module, image The preprocessing module, the human eye coarse positioning module and the pupil fine positioning module finally obtain the position of the pupil center. 2. a positioning method of pupil in a human face video, is characterized in that: the processing method of human face detection module is: The input of this module is the target image I _orig , using the face detection method based on local continuous mean quantization transformation and sparse screening network classifier, the face image I _face including eyes is detected, and it is taken as the output of this module. 3. a positioning method for pupils in a face video, characterized in that: the processing method of the image preprocessing module is: (1) Median filtering Use a 3×3 window to perform median filtering on the face image, so that each point in the image will correspond to a 3×3 window, and the point is located in the center of the window; the 8 points around the center point of the window are grayscale The values are sorted from large to small, and finally the median value of the gray value of each point in the window, that is, the average value of the gray value of the middle two points after sorting, is used to replace the gray value of the center point of the window, that is, the median filter is completed to obtain Image is I _temp ; (2) Histogram equalization Carry out histogram equalization to the image I _temp after the median filter; Histogram equalization is to map an image to another with a balanced histogram through grayscale transformation, and the mapping function is a cumulative distribution function; its specific steps are as follows: 1) First calculate the normalized grayscale histogram H of I _temp ;

i=0,1,...,255, Among them, n _i is the number of pixels with gray level i in the image, i=0,1,...,255, n is the total number of pixels in the image; 2) Then calculate the histogram integral H', ${\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>$ 3) finally calculate the image I after histogram equalization; I(x,y)=H'[I _temp (x,y)]. 4. a positioning method for pupils in a human face video, characterized in that: the processing method of the coarse positioning module of human eyes is: (1) Calculate the vertical grayscale projection curve P _y (x) of I, ${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ Among them, w is the width of the image; (2) The coordinates of the peak value of P _y (x) at 1/3 to 1/2, which is the middle of the nose of the human face, use this coordinate to intercept the I _face in the vertical direction to obtain a rough human eye area; (3) Taking the center of the width of the human eye region as the boundary, the human eye region is divided into half in the vertical direction, and the two eyes can be separated to obtain the left eye region I _left and the right eye region I _right . 5. A positioning method for pupils in a human face video, characterized in that: the processing method of the coarse positioning module of human eyes is: (1) Convolute the eye image I _left or I _right obtained by the human eye coarse positioning module with the Sobel horizontal operator and vertical operator respectively, and calculate the edge gradient image g(p)=[g _x (p),g _y (p)]; g(p)＝[g _x (p), g _y (p)]＝|[I*Sobel _hor ,I*Sobel _ver ]|, Among them, '*' means convolution, Sobel _hor is the Sobel level operator ${\mathrm{Sobel}}_{\mathrm{hor}}=\left(\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right),$ Sobel _ver is the Sobel vertical operator ${\mathrm{Sobel}}_{\mathrm{ver}}=\left(\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right);$ (2) Initialize the initial search radius N=5, S _{n_max} =0, S _max =0; (3) If S _max ≥ S _{n_max} is satisfied, perform the following steps: (a) S _max = S _{n_max} ; (b) For n=2,3,...,N, the corresponding direction projection image O _n and amplitude projection image M _n can be calculated; for a given point P, On and _{M n are} determined by the positive and negative influence pixels P _+ve Calculated with P _-ve , and P _+ve and P _-ve are functions of the gradient g(p); the definitions of positive and negative impact pixels are different, and the positive impact pixels are pointed by the gradient vector g(p) and are far from point P The coordinates of a point with a length of n; the negatively affected pixel is the coordinate of a point with a length of n from the point P that the gradient vector g(p) negatively points to; From the gradient vector g(p), the negatively affected pixels P _-ve and _On [P _-ve (p)], M _n [P _-ve (p)] can be calculated, namely ${\mathrm{<span\; class="notranslate">\; P</span>}}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; ve</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; round</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$ _On [P _-ve (p)] = _On [P _-ve (p)] - 1, (7) _Mn [P _-ve (p)]= _Mn [P _-ve (p)]-|g(p)|, (9) Among them, "round" means to take all the elements in the vector to the integer closest to them; |·| means to take the modulus of the vector; Then calculate F _n (p), ${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}{<span\; class="notranslate">\; [</span>\frac{{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; o</span>}}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}<span\; class="notranslate">\; ,</span>$ Among them, α is the radial control parameter; parameter α=2, k _n =9.9;

Among them, k _n is used to normalize the scale factors of O _n and M _n obtained at different radii; When the detection radius is n, the radial symmetric intensity value S _n is S _n =F _n *A _n , (10) Among them, '*' means convolution, A _n is a two-dimensional Gaussian matrix, its size is n×n, variance σ=0.25n; (c) The final radial symmetry transformation S is the mean value of the radial symmetry results S _n corresponding to all detection radii n, and the largest value is selected as the pupil center; $\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; |</span>}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$ (d) N=N+2; (4) If S _max <S _{n_max} , the iterative search ends, and the position of the pupil center is obtained.

Images