CN105205480A - Complex scene human eye locating method and system - Google Patents

Abstract

The present invention relates to the technical field of face recognition, and provides a human eye positioning method and system in a complex scene. The method includes: performing face image processing and detection on an acquired image, and generating a face image containing a pure face area; Contrast enhancement processing is performed on the face image containing the pure face area to obtain a face image that highlights the grayscale features of the eyes of the face; preliminary positioning of the human eye is performed on the face image that highlights the grayscale features of the eyes of the face Processing, to obtain the human eye image of the candidate eye area; according to the position of the eye center in the acquired candidate eye area, calculate the coordinates of the eye center in the collected image, and mark it, so as to realize the fast and accurate face recognition in large scenes. accurate positioning.

Description

Human eye positioning method and system in a complex scene

technical field

The invention belongs to the technical field of face recognition, and in particular relates to a human eye positioning method and system in complex scenes.

Background technique

Face recognition is an important topic in the field of pattern recognition research, and has a good application prospect in information security, entrance and exit access control, and smart cards. Among them, a very important process in two-dimensional and three-dimensional face recognition methods is the detection and positioning of human eyes. This is because the eye area contains a wealth of important information that can be used to distinguish individuals, which can not only improve the speed of recognition and detection, Moreover, the complexity of the recognition algorithm can be reduced. At the same time, since the position and distance of the eyes are least affected by changes in illumination and expression, the positioning of the eyes is the prerequisite for normalizing the position, size and angle of the face image, as well as the detection and extraction of other parts of the face such as eyebrows, nose, and mouth. Foundation. Therefore, the automatic positioning of human eyes has become a basic and very important topic in face recognition research.

At present, there are many methods for human eye positioning, mainly including methods based on template matching, methods based on grayscale projection, and methods based on classifier design. Among them, these human eye positioning methods have defects, specifically:

In the method based on template matching, it is necessary to use the left-eye template and the right-eye template to match in the image, which does not require a lot of prior knowledge, but it has requirements for the initial position and a large amount of calculation;

Based on the gray-scale projection method, due to its small amount of calculation, it is often used as the positioning of the human eye, but this method requires a lot of image preprocessing, and is greatly affected by illumination and occlusion, and the occlusion of hair will make the algorithm failure;

Classifier-based design methods mainly include support vector machine (Support Vector Machine, SVM), neural network, iterative algorithm AdaBoost, etc. These methods regard human eye positioning as a classification problem, and their positioning accuracy is high. There are faint targets, and using the classifier to search the image globally multiple times is computationally intensive and the process is cumbersome.

Contents of the invention

The purpose of the present invention is to provide a human eye positioning method in a complex scene that can quickly and accurately locate a human face in a complex large scene.

The present invention is achieved in this way, a human eye positioning method in a complex scene, said method comprising the following steps:

Perform face image processing and detection on the acquired image to generate a face image containing a pure face area;

Contrast enhancement processing is performed on the face image containing the pure face area to obtain a face image that highlights the gray features of the eyes of the face;

Perform preliminary positioning processing of the human eye on the human face image that highlights the grayscale features of the human face's eyes, and obtain the human eye image of the candidate eye area;

According to the obtained position of the eye center in the candidate eye area, the coordinates of the eye center in the collected image are calculated and marked.

As an improved solution, the step of performing face image processing and detection on the collected images to generate a face image containing a pure face area specifically includes the following steps:

Convert the acquired RGB image to a color space image;

Carrying out model analysis based on skin color and arithmetic processing based on morphology on the color space image to obtain a face image based on grayscale;

Perform area screening on the face image processed based on the morphological operation, and obtain the image of the face candidate area based on the gray scale;

According to the obtained image of the face candidate area, obtain a screenshot of the face candidate;

The human face candidate screenshot is converted into a grayscale image, and a human face region is detected on the grayscale image to generate a human face image including a pure human face region.

As an improved solution, the step of performing contrast enhancement processing on the face image containing the pure face region to obtain a face image highlighting the grayscale features of the eyes of the face specifically includes the following steps:

Perform high-hat transformation processing on face images containing pure face regions;

Perform low-hat transformation processing on the face image;

Perform contrast enhancement calculations on face images;

Binarize the face image after the contrast enhancement calculation;

Filtering is performed on the binarized face image to obtain a face image that highlights the gray features of the eyes of the face.

As an improved solution, the preliminary positioning process of the human eye is performed on the human face image highlighting the grayscale features of the eyes of the human face, and the step of obtaining the human eye image of the candidate eye area specifically includes the following steps:

Crop the face image that highlights the grayscale features of the eyes of the face, and remove the edge hair area;

Screen the face images that remove the edge hair area, and select two areas as candidate eye areas;

Carry out frame marking and filling on the selected two candidate eye regions to form a masked binary image;

Carry out matting process with described mask binary image and the face image of the gray-scale feature after cropping, obtain thick left-eye image and thick right-eye image;

The coarse left-eye image and the coarse right-eye image are sent to a support vector machine classifier for detection and verification, and human eye images of candidate eye regions conforming to human eye characteristics are obtained and output.

As an improved solution, the preliminary positioning process of the human eye is performed on the human face image highlighting the grayscale features of the human face and eyes, and after the step of obtaining the human eye image of the candidate eye area, the obtained The position of the eye center in the candidate eye area, the step of calculating the coordinates of the eye center in the collected image also includes the following steps:

For the acquired human eye image of the candidate eye area, the pupil center of the candidate eye area is located, and the position of the eye center in the candidate eye area is determined.

Another object of the present invention is to provide a human eye positioning system in complex scenes, said system comprising:

A human face image generation module, which is used to process and detect the acquired image to generate a human face image containing a pure human face area;

Contrast enhancement processing module, for carrying out contrast enhancement processing to the face image that contains pure face area, obtains the face image that highlights the gray scale feature of people's face eyes;

The candidate eye region acquisition module is used to perform preliminary positioning processing of the human eye on the human face image highlighting the gray-scale features of the human face and eyes, and obtain the human eye image of the candidate eye region;

The eye center calculation and marking module is configured to calculate and mark the coordinates of the eye center in the collected image according to the acquired position of the eye center in the candidate eye area.

As an improved solution, the face image generation module specifically includes:

A color conversion module is used to convert the obtained RGB image into a color space image;

The skin color model analysis module is used to perform model analysis based on skin color to the color space image;

The morphological operation module is used for morphological-based operation processing to obtain a grayscale-based face image;

The area screening module is used to perform area screening on the face image processed based on the morphological operation, and obtain an image of a grayscale-based face candidate area;

The human face candidate screenshot acquisition module is used to obtain the human face candidate screenshot according to the acquired image of the human face candidate region;

The conversion detection module is used to convert the candidate screenshot of the human face into a grayscale image, and detect the human face area on the grayscale image to generate a human face image containing a pure human face area.

As an improved solution, the contrast enhancement processing module specifically includes:

A high-hat transformation processing module, which is used to perform high-hat transformation processing on a face image containing a pure face region;

A low-hat transformation processing module for performing low-hat transformation processing on the face image;

Contrast enhancement calculation module, for carrying out contrast enhancement calculation to face image;

Binarization processing module, for carrying out binarization processing to the face image after contrast enhancement calculation;

The filtering processing module is used to perform filtering processing on the binarized human face image to obtain the human face image highlighting the gray features of the eyes of the human face.

As an improved solution, the candidate eye region acquisition module specifically includes:

The screenshot processing module is used to crop the face image that highlights the grayscale features of the eyes of the face, and remove the edge hair area;

The screening module is used to screen the face images that remove the edge hair area, and select two areas as candidate eye areas;

The mask binary image forming module is used to mark and fill the selected two candidate eye regions to form a mask binary image;

A matting processing module, which is used to perform matting processing on the mask binary image and the face image of the gray-scale feature after cropping, to obtain a thick left-eye image and a thick right-eye image;

The classification, detection and verification module is used to send the coarse left-eye image and the coarse right-eye image to the support vector machine classifier for detection and verification, obtain human eye images of candidate eye regions conforming to human eye characteristics, and output them.

As an improved solution, the system also includes:

The pupil center positioning module is configured to locate the pupil center of the candidate eye area on the acquired human eye image of the candidate eye area, and determine the position of the eye center in the candidate eye area.

In the embodiment of the present invention, face image processing and detection are performed on the acquired image to generate a face image containing a pure face area; contrast enhancement processing is performed on a face image containing a pure face area to obtain a highlighted face The face image of the grayscale features of the eyes; the preliminary positioning of the human eye is performed on the face image highlighting the grayscale features of the eyes of the human face, and the human eye image of the candidate eye area is obtained; according to the obtained candidate eye area The position of the center of the eye in the middle, calculate the coordinates of the center of the eye in the collected image, and mark it, so as to realize the fast and accurate positioning of the face in a large scene.

Description of drawings

Fig. 1 is the implementation flowchart of the human eye positioning method in the complex scene provided by the present invention;

Fig. 2 is the specific implementation flow chart of carrying out face image processing and detection to the collected image provided by the present invention to generate a face image comprising a pure face area;

Fig. 3 is the implementation flow chart of the present invention for carrying out contrast enhancement processing on a face image containing a pure face region to obtain a face image highlighting the grayscale features of the eyes of a face;

Fig. 4 is the concrete implementation flow chart of obtaining the human eye image of the candidate eye area on the human face image highlighting the gray scale feature of human face and eyes for the preliminary positioning processing of the human eye provided by the present invention;

Fig. 5 is a structural block diagram of a human eye positioning system in a complex scene provided by the present invention;

Fig. 6 is the structural block diagram of the human face image generation module provided by the present invention;

Fig. 7 is a structural block diagram of a contrast enhancement processing module provided by the present invention;

Fig. 8 is a structural block diagram of a candidate eye region acquisition module provided by the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Fig. 1 shows the implementation process of the human eye positioning method in the complex scene provided by the present invention, and its specific steps are as follows:

In step S101, face image processing and detection are performed on the acquired image, and a face image including a pure face area is generated.

In step S102, contrast enhancement processing is performed on the face image including the pure face area to obtain a face image highlighting the gray features of the eyes of the face.

In step S103, a preliminary positioning process of the human eye is performed on the human face image highlighting the grayscale features of the eyes of the human face, and the human eye image of the candidate eye area is obtained.

In step S104 , for the acquired human eye image of the candidate eye region, the pupil center of the candidate eye region is located, and the position of the eye center in the candidate eye region is determined.

In step S105, according to the obtained position of the eye center in the candidate eye area, the coordinates of the eye center in the collected image are calculated and marked.

Wherein, the above step S104 is a preferred solution, the step of locating the center of the eye may not be performed, and the above step S105 may be directly performed, which will not be repeated here.

Wherein, Fig. 2 shows the specific implementation process of performing face image processing and detection on the collected images provided by the present invention to generate a face image containing a pure face area, which specifically includes the following steps:

In step S201, the acquired RGB image is converted into a color space image.

Wherein, the RGB image can be obtained in various ways, such as collecting on-site by a camera or reading a color picture from a database;

The above color space conversion is mainly to convert the RGB image into the YCbCr color space, which is mainly used to effectively separate the brightness and chroma, and the model mode of the conversion can be used as follows:

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; r</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; 0.2990</span>}& \mathrm{<span\; class="notranslate">\; 0.5870</span>}& \mathrm{<span\; class="notranslate">\; 0.1140</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.1687</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.3313</span>}& \mathrm{<span\; class="notranslate">\; 0.5000</span>}& \mathrm{<span\; class="notranslate">\; 128</span>}\\ \mathrm{<span\; class="notranslate">\; 0.5000</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.4187</span>}& \mathrm{<span\; class="notranslate">\; 0.0813</span>}& \mathrm{<span\; class="notranslate">\; 128</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; G</span>}\\ \mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

Among them, Y represents the brightness of the color, Cb represents the blue component, Cr represents the red component, Cr and Cb together represent the chromaticity information of the color, and Cr and Cb are two-dimensionally independent.

In step S202, model analysis based on skin color and arithmetic processing based on morphology are performed on the color space image to obtain a face image based on grayscale.

Among them, since the face skin color in the YCbCr space shows better clustering characteristics, and has a better distinguishing effect from the background color, the image segmented by calculating the skin color similarity of a single pixel, if 1 is used to represent the skin color area, 0 indicates a non-skinned area, then the discriminant function of the skinned area can be obtained as follows:

Since the skin color area is only processed by the Cb and Cr components of the YCbCr space, it is necessary to apply a morphological operator to remove the isolated background area in the face image. After calculation, the erosion operation is used to remove isolated noise, and the expansion operation is used to fill the face. Therefore, the closing operation is "·", which is firstly expanded and then corroded. A is closed with B and recorded as A B. Its definition as follows:

$\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; \&CirclePlus;</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&Theta;</span>}\mathrm{<span\; class="notranslate">\; B</span>}$

Among them, the symbol of corrosion operation is "Θ", and the symbol of expansion operation is " ".

In step S203, area screening is performed on the face image processed based on the morphological operation, and images of face candidate areas based on gray scale are obtained.

Among them, after processing by the filtering method based on mathematical morphology, most of the small block noise in the face image is removed, but due to the complex background and many influencing factors, there may be non-face areas such as bare arms or legs that are mistakenly detected. It is detected as a face candidate area. In order to delete the area of the non-face image as much as possible, it is verified based on prior geometric knowledge such as shape size, aspect ratio, ellipse approximation of the long and short axis ratio, and pixel occupancy. region, retaining skin color image patches containing faces.

For the face area, since there are non-skin color areas such as eyes, mouth, eyebrows, etc., there will be one or more "holes" (non-face areas) in the face area. Based on this, those that do not contain holes can be removed. For the skin color area, the number of holes in the candidate face area is calculated. The calculation method uses the Euler number. The Euler number is defined as the number of connected components minus the number of holes, expressed by the formula:

E=C-H

Among them, E, C and H are Euler number, connected component fraction and hole number, respectively, and can be obtained from the above formula:

H=C-E

Considering the growth area of skin color, C=1, so H=1-E.

Calculate the Euler number of each block, reflecting how many holes each block has. Because the eyes, nose and lips of the human face will be displayed as black holes after the above steps, so a threshold is set through the calculated Euler number, and when the Euler number of the block is >0, the block is regarded as a face area, enter the next round of face area candidates, otherwise it is regarded as a non-face area.

In addition, the length and width of the circumscribed rectangle of the above-mentioned face area are also used in the method of area screening, which will not be repeated here.

In step S204, according to the obtained image of the face candidate area, a screenshot of the face candidate is obtained.

Among them, according to the obtained face candidate area, the upper, lower, left, and right ranges of the face candidate area are respectively expanded by 20 pixels to form an extended rectangle, so as to ensure that all the face areas fall into the extended rectangle to the greatest extent, and then the extended rectangle is saved as Image format, get a screenshot of the face candidate.

In step S205, the candidate screenshot of the face is converted into a grayscale image, and the face area is detected on the grayscale image to generate a face image including a pure face area.

Among them, the conversion of the face candidate screenshot into a grayscale image can be combined with the principle of the sensitivity of the human eye to the color value, and the weighted average method is adopted, namely:

Y＝ω _k *R+ω _G *G+ω _B *B

Among them, W _R , W _G , and W _B are the weights corresponding to the color components R, G, and B respectively, Y is the pixel value of the corresponding point in the grayscale image, and the parameters used are set as W _R =0.30, W _G =0.59, W _B =0.11, the pixel value of the grayscale image is 256 levels.

The detection of the face area on the grayscale image is mainly based on the iterative algorithm AdaBoost, and its specific implementation is as follows:

First, use the Haar-like rectangular feature to describe the face, and use the "integral map" to realize the fast calculation of the feature vector; then, based on the AdaBoost algorithm, select some rectangular features that best represent the face to form a weak classifier, and use weighted voting. The weak classifier is constructed as a strong classifier; finally, several strong classifiers trained are connected in series to form a cascaded classifier, which can effectively improve the detection speed.

If no face is detected in this step, the original image needs to be read in as a whole, and the entire image is searched to obtain a face after grayscale transformation. For the case where no face is detected by using the Adaboost classifier search for the entire image , directly prompting "No face detected".

In this embodiment, in the above step S205, the face image containing the pure face area needs to be screenshot based on the grayscale image, and the face image containing the pure face area after the screenshot is a square with an aspect ratio of 1 :1.

At the same time, it is necessary to normalize the size of the square face image containing the pure face area, that is, to scale the image size to 100 pixels*100 pixels.

Fig. 3 shows the implementation process of performing contrast enhancement processing on a face image containing a pure face region to obtain a face image highlighting the gray features of the eyes of the face provided by the present invention, and its specific steps are as follows:

In step S301, a high-hat transformation process is performed on the face image including the pure face area.

Among them, in order to highlight the gray features of the eyes in the face image, it is necessary to use the method of morphological filtering for contrast enhancement processing. The fast splicing mapping algorithm (Top-Hat) has some characteristics of high-pass filtering, that is, the Top-Hat algorithm The operator can detect the peak of the gray value in the image, and the closed Top-Hat operator can detect the valley of the gray value in the image, and use the Top-hat transformation in mathematical morphology to preprocess the face image to weaken The impact of external light changes on the face recognition effect. Find clusters of bright pixels from the background.

In morphology, erosion and dilation are the basis of mathematical morphology, which are the maximum and minimum operations in the domain of definition, and other transformations are defined by the combination of these two transformations.

Suppose f(x) and b(x) are two discrete functions defined on the two-dimensional discrete space F and B, where f(x) represents the grayscale image to be processed, and b(x) represents the selected structural element , then the expansion and erosion of f(x) with respect to b(x) are defined as:

$<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&CirclePlus;</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; B</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$

After the dilation operation, the gray value in the result is the maximum value of the sum of the gray values of the point in a local range and the corresponding point in the structural element. It is a process of expanding the boundary points to the outside, which can expand the boundary points of the object, so as to merge all the background points in contact with the object into the object.

$<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; \&Theta;</span>}\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; B</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$

The result of the erosion operation is the minimum value of the difference between the gray value of the point and the corresponding point in the structure element within a local range. It can remove objects that are smaller than structural elements, and can eliminate object boundary points. It is a process of shrinking the boundary to the inside.

Therefore, the specific process of the top-hat transformation in the step S301 is as follows:

The difference between the original image f(x) and the image after the opening operation is subtracted to detect the peak in the image, thereby extracting the foreground information of the image. It is 8*8 size.

In step S302, low-hat transformation processing is performed on the face image.

Among them, the low-hat transformation process is to use the difference between the original image obtained after the closed operation of the original image f(x) and the original image to detect valleys in the image and extract the background information of the image, that is, to expand the grayscale image first and then Corrosion operation, its operator is 8*8 in size.

In step S303, a contrast enhancement calculation is performed on the face image.

Wherein, the process of the contrast enhancement calculation is to add the image processed by the high-hat transformation in step S301 to the original image, and then subtract the image processed by the low-hat transformation in step S302 to obtain a contrast-enhanced face image.

In step S304, binarization processing is performed on the face image after the contrast enhancement calculation.

Wherein, assuming that the image processed by the above step S303 is f(x, y), the image after binarization processing is g(x, y), and the threshold is set to T, then:

$\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; T</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; T</span>}\end{array}\right.$

Among them, the part with a value of 1 represents the target area, and the part with a value of 0 represents the background.

In step S305, filtering is performed on the binarized human face image to obtain a human face image highlighting the grayscale features of the eyes of the human face.

Among them, the opening operation processing based on morphology is performed on the binarized face image, that is, the opening operation processing is performed using the morphological operator [0, 1, 1, 1, 0], and some vertically distributed connected regions are excluded. , to reduce the connection of eyes and eyebrows caused by hair or other disturbances.

Fig. 4 shows the specific implementation process of performing the preliminary positioning processing of the human eye on the human face image highlighting the gray-scale features of the human face and eyes, and obtaining the human eye image of the candidate eye area provided by the present invention, which specifically includes the following steps :

In step S401, the face image highlighting the gray features of the eyes is cropped, and the edge hair area is removed.

Wherein, a screenshot of the face image highlighting the grayscale features of the eyes of the face is taken, and the upper half of the face image is taken for analysis, for example, the height of the intercepted image is halved, and the width remains unchanged.

In face images, the presence of hair will appear in the image bordering the edge area, so it needs to be culled.

First, the target area in the captured face image is marked with the 8-connected area marking method to distinguish each independent white area.

Then, find the area with edge coordinates in each area (because the image is 100×50 pixels in size, as long as 1 or 100 appears in the abscissa, or 1 or 50 appears in the ordinate, the area is considered to be an edge border area).

Finally, search the bordering areas of each edge to find out whether there is a coordinate point located within the rectangular area with the upper left corner coordinates [26,16] and the lower right corner coordinates [40,85], if so, use other areas outside the rectangular area with 0 is filled with black; otherwise, the edge border area is filled with 0 and black.

In step S402, the face images with the edge hair regions removed are screened, and two of the regions are selected as candidate eye regions.

Among them, the conditions for filtering face images are:

the height of the region is greater than its width;

The area width is less than 8 pixels;

The area area is less than 15 pixels;

That is: if the following conditions are not met, the area will be filtered out, and the pixels in the area will be replaced with 0.

In this step, the specific implementation of selecting two regions as candidate eye regions is as follows:

Separation of eyebrows and eyes, extracting candidate eye regions;

The number of filtered regions is mostly 4, but there may be exceptions that need to be processed.

First, count the number of regions, and calculate the center coordinates of each region;

Then determine the number of regions, and process them separately according to the number of regions;

a. If the number of regions is 4, select the 2 regions with the smallest vertical coordinates as eye candidate regions;

b. If the number of areas is 2-3, it is necessary to perform a symmetrical filling operation on the face image. The specific method is: perform a left-right mirror operation on image A to obtain a mirror image B, and then perform an XOR operation on images A and B , obtain image C, and then select the two regions with the smallest vertical coordinates as eye candidate regions.

c. If the number of regions is 0-1 or greater than 4, then directly perform cutout operation on the current image, and the cutout region is a rectangle of 10×20 pixels.

In step S403, the selected two candidate eye regions are frame marked and filled to form a masked binary image.

Among them, the two candidate eye regions after the screening operation are frame-marked, that is, the two candidate eye regions are made the smallest rectangular area, and marked with a frame;

Then the minimum rectangular area is filled, that is, the two minimum rectangular areas are filled with a pixel value of 1, and the other parts are filled with 0, and finally the mask binary image is formed.

In step S404, a matting process is performed on the masked binary image and the face image with gray features after cropping to obtain a rough left-eye image and a rough right-eye image.

In step S405, the coarse left-eye image and the coarse right-eye image are sent to the support vector machine classifier for detection and verification, and human eye images of candidate eye regions conforming to human eye characteristics are obtained and output.

Send the above two images to the support vector machine classifier for detection and verification. If the human eye condition standard is met, proceed to the next step, otherwise, the system will change the parameters again. (For the design of the support vector machine classifier, it mainly uses human eye samples and non-human eye samples to complete its training, the technology and steps are mature, and it is not an innovative point of the present invention, so it will not be described in detail).

In this embodiment, the support vector machine classifier can not only verify the human eyes, but also avoid the global search of the entire image, reduce the amount of calculation, and increase the accuracy of rough positioning.

In the embodiment of the present invention, for the acquired human eye image of the candidate eye area, the pupil center of the candidate eye area is positioned, and the specific steps for determining the position of the eye center in the candidate eye area are as follows:

Due to the influence of interference from eyelashes, etc., the obtained candidate eye regions need to be precisely positioned. For the obtained eye window, it is mainly composed of pupils and whites of the eyes. Utilizing the characteristic that the pupil area changes greatly relative to the surrounding gray scale, the position of the pupil can be roughly positioned first, and then the center of the pupil can be precisely positioned, thereby realizing the positioning of the center of the eye. The specific steps are as follows:

(1) Use the obtained binary image (that is, the human eye image of the candidate eye area) as a mask, and multiply it with the corresponding grayscale image to obtain a cutout image IM1 containing only the eye area, in which the background is 0, and the cut out The eye area is saved as a new image;

(2) Find the background point where the pixel in the IM is 0, replace it with a grayscale of 255, and obtain an eye image with a white background;

(3) image adopts threshold segmentation algorithm to obtain threshold, then obtains candidate pupil region by performing threshold segmentation;

(4) Perform an open operation on the candidate pupil area, the operator is:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$

And filter out redundant noise points;

(5) Select the pupil area;

a. Mark the white area in the binary image;

b. Count the area of each region;

c. Then sort the area;

d. Filter out the first two areas with the largest area, and replace the pixel values of other areas with 0.

(6) Fill the hole in the pupil area;

The operators are:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccccc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$

This operation makes it possible to fill in gaps caused by reflections in the pupil area.

(7) Calculate the center of the pupil using the center of gravity method. The calculation formula is: use the center point of the image as the starting point for boundary tracking.

In the embodiment of the present invention, in the above step S105, the absolute coordinates of the eye center in the original image are calculated, and then the eye center is marked by "+", and the eye area is marked by a rectangle, so as to realize Human eye recognition.

FIG. 5 shows a structural block diagram of the human eye positioning system in a complex scene provided by the present invention. For the convenience of description, only the parts related to the present invention are shown in the figure.

Face image generation module 11 is used to carry out face image processing and detection to the image obtained, generates the face image that comprises pure human face area; Contrast enhancement processing module 12 is used for comprising the human face image of pure human face area Contrast enhancement processing, to obtain a human face image highlighting the grayscale features of the eyes of the human face; the candidate eye region acquisition module 13 is used to perform preliminary positioning processing of the human eye on the human face image highlighting the grayscale features of the eyes of the human face, Acquire the human eye image of the candidate eye area; the eye center calculation and marking module 14 is used to calculate and mark the coordinates of the eye center in the collected image according to the acquired position of the eye center in the candidate eye area.

Wherein, the pupil center positioning module 15 is used for locating the pupil center of the candidate eye area on the acquired human eye image of the candidate eye area, and determining the position of the eye center in the candidate eye area.

As shown in Figure 6, the specific structure of face image generating module 11 is as follows:

The color conversion module 21 is used to convert the acquired RGB image into a color space image; the skin color model analysis module 22 is used to perform model analysis based on skin color to the color space image; the morphology operation module 23 is used for the operation based on morphology Process to obtain a face image based on gray scale; the area screening module 24 is used to perform area screening on the face image processed based on morphological operations, and obtain the image of a face candidate area based on gray scale; the human face candidate screenshot acquisition module 25 is used for obtaining the human face candidate screenshot according to the obtained image of the human face candidate area; the conversion detection module 26 is used for converting the human face candidate screenshot into a grayscale image, and performing detection of the human face area on the grayscale image to generate Face images containing pure face regions.

As shown in Figure 7, the specific structure of the contrast enhancement processing module 12 is as follows:

High-hat transformation processing module 31 is used for carrying out high-hat transformation processing to the face image that contains pure human face region; Low-hat transformation processing module 32 is used for carrying out low-hat transformation processing to human face image; Contrast enhancement calculation module 33 is used for Face image carries out contrast enhancement calculation; Binarization processing module 34 is used for carrying out binarization processing to the face image after contrast enhancement calculation; Filtering out processing module 35 is used for filtering the face image after binarization processing In addition to processing, the face image that highlights the gray features of the eyes of the face is obtained.

As shown in Figure 8, the specific structure of the candidate eye region acquisition module 13 is as follows:

The screenshot processing module 41 is used to crop the face image highlighting the grayscale features of the eyes of the human face, and removes the edge hair region; the screening module 42 is used to screen the human face image removing the edge hair region, and selects 2 of them region as the candidate eye region; the mask binary image forming module 43 is used to frame mark and fill the two selected candidate eye regions to form a mask binary image; the matting processing module 44 is used to use the mask The film binary image and the face image of the gray-scale feature after the cropping are subjected to matting processing to obtain a thick left-eye image and a thick right-eye image; the classification detection verification module 45 is used to combine the thick left-eye image and thick right-eye image The image is sent to the support vector machine classifier for detection and verification, and the human eye image of the candidate eye area that conforms to the characteristics of the human eye is obtained and output.

Wherein, the specific implementation of each module shown in the above-mentioned FIG. 5 to FIG. 8 is as described in the above-mentioned corresponding method embodiment, and will not be repeated here, but it is not used to limit the present invention.

In the embodiment of the present invention, face image processing and detection are performed on the acquired image to generate a face image containing a pure face area; contrast enhancement processing is performed on a face image containing a pure face area to obtain a highlighted face The face image of the grayscale features of the eyes; the preliminary positioning of the human eye is performed on the face image highlighting the grayscale features of the eyes of the human face, and the human eye image of the candidate eye area is obtained; according to the obtained candidate eye area The position of the center of the eye in the middle, calculate the coordinates of the center of the eye in the collected image, and mark it, so as to realize the fast and accurate positioning of the face in a large scene.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (10)

1. a human eye location method in a complicated scene, is characterized in that, described method comprises the steps: Perform face image processing and detection on the acquired image to generate a face image containing a pure face area; Contrast enhancement processing is performed on the face image containing the pure face area to obtain a face image that highlights the gray features of the eyes of the face; Perform preliminary positioning processing of the human eye on the human face image that highlights the grayscale features of the human face's eyes, and obtain the human eye image of the candidate eye area; According to the obtained position of the eye center in the candidate eye area, the coordinates of the eye center in the collected image are calculated and marked. 2. the human eye location method in the complex scene according to claim 1, is characterized in that, described face image processing and detection are carried out to the image that gathers, and the step of generating the face image that comprises pure human face area specifically comprises Follow the steps below: Convert the acquired RGB image to a color space image; Carrying out model analysis based on skin color and arithmetic processing based on morphology on the color space image to obtain a face image based on grayscale; Perform area screening on the face image processed based on the morphological operation, and obtain the image of the face candidate area based on the gray scale; According to the obtained image of the face candidate area, obtain a screenshot of the face candidate; The human face candidate screenshot is converted into a grayscale image, and a human face region is detected on the grayscale image to generate a human face image including a pure human face region. 3. the human eye positioning method in the complex scene according to claim 1, is characterized in that, the described human face image that comprises pure human face area is carried out contrast enhancement processing, obtains the people who highlights the grayscale feature of human face eyes. The steps of the face image specifically include the following steps: Perform high-hat transformation processing on face images containing pure face regions; Perform low-hat transformation processing on the face image; Perform contrast enhancement calculations on face images; Binarize the face image after the contrast enhancement calculation; Filtering is performed on the binarized face image to obtain a face image that highlights the gray features of the eyes of the face. 4. the human eye positioning method in the complex scene according to claim 1, is characterized in that, the preliminary positioning processing of human eyes is carried out on the human face image highlighting the gray scale features of human face eyes, and the candidate eye area is obtained The steps of the human eye image specifically include the following steps: Crop the face image that highlights the grayscale features of the eyes of the face, and remove the edge hair area; Screen the face images that remove the edge hair area, and select two areas as candidate eye areas; Carry out frame marking and filling on the selected two candidate eye regions to form a masked binary image; Carry out matting process with described mask binary image and the face image of the gray-scale feature after cropping, obtain thick left-eye image and thick right-eye image; The coarse left-eye image and the coarse right-eye image are sent to a support vector machine classifier for detection and verification, and human eye images of candidate eye regions conforming to human eye characteristics are obtained and output. 5. the human eye positioning method in the complex scene according to claim 1, is characterized in that, the preliminary positioning processing of human eyes is carried out on the human face image highlighting the gray scale features of human face eyes, and the candidate eye area is obtained After the step of the human eye image, the step of calculating the coordinates of the eye center in the collected image according to the position of the eye center in the obtained candidate eye area also includes the following steps: For the acquired human eye image of the candidate eye area, the pupil center of the candidate eye area is located, and the position of the eye center in the candidate eye area is determined. 6. A human eye positioning system in a complex scene, characterized in that the system comprises: A human face image generation module, which is used to process and detect the acquired image to generate a human face image containing a pure human face area; Contrast enhancement processing module, for carrying out contrast enhancement processing to the face image that contains pure face area, obtains the face image that highlights the gray scale feature of people's face eyes; The candidate eye region acquisition module is used to perform preliminary positioning processing of the human eye on the human face image highlighting the gray-scale features of the human face and eyes, and obtain the human eye image of the candidate eye region; The eye center calculation and marking module is configured to calculate and mark the coordinates of the eye center in the collected image according to the acquired position of the eye center in the candidate eye area. 7. human eye positioning system in complex scene according to claim 6, is characterized in that, described human face image generation module specifically comprises: A color conversion module is used to convert the obtained RGB image into a color space image; A skin color model analysis module, used for performing skin color model analysis on the color space image; The morphological operation module is used for morphological-based operation processing to obtain a grayscale-based face image; The area screening module is used to perform area screening on the face image processed based on the morphological operation, and obtain an image of a grayscale-based face candidate area; The human face candidate screenshot acquisition module is used to obtain the human face candidate screenshot according to the acquired image of the human face candidate region; The conversion detection module is used to convert the candidate screenshot of the human face into a grayscale image, and detect the human face area on the grayscale image to generate a human face image containing a pure human face area. 8. The human eye positioning system in a complex scene according to claim 6, wherein the contrast enhancement processing module specifically includes: A high-hat transformation processing module, which is used to perform high-hat transformation processing on a face image containing a pure face region; A low-hat transformation processing module for performing low-hat transformation processing on the face image; Contrast enhancement calculation module, for carrying out contrast enhancement calculation to face image; Binarization processing module, for carrying out binarization processing to the face image after contrast enhancement calculation; The filtering processing module is used to perform filtering processing on the binarized human face image to obtain the human face image highlighting the gray features of the eyes of the human face. 9. The human eye positioning system in a complex scene according to claim 6, wherein the candidate eye region acquisition module specifically includes: The screenshot processing module is used to crop the face image that highlights the grayscale features of the eyes of the face, and remove the edge hair area; The screening module is used to screen the face images that remove the edge hair area, and select two areas as candidate eye areas; The mask binary image forming module is used to mark and fill the selected two candidate eye regions to form a mask binary image; A matting processing module, which is used to perform matting processing on the mask binary image and the face image of the gray-scale feature after cropping, to obtain a thick left-eye image and a thick right-eye image; The classification, detection and verification module is used to send the coarse left-eye image and the coarse right-eye image to a support vector machine classifier for detection and verification, obtain human eye images of candidate eye regions conforming to human eye characteristics, and output them. 10. human eye positioning system in complex scene according to claim 6, is characterized in that, described system also comprises: The pupil center positioning module is configured to locate the pupil center of the candidate eye area on the acquired human eye image of the candidate eye area, and determine the position of the eye center in the candidate eye area.