CN111369496A - Pupil center positioning method based on star ray - Google Patents

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a pupil center positioning method based on a star ray. The method comprises the steps of firstly preprocessing a picture, then detecting a pupil area through a star ray algorithm, obtaining a binarization threshold value in the ROI area through an iteration method, optimizing the pupil area after binarization, and removing star ray edge point errors. The pupil center under the shielding condition can be accurately detected by the method. The invention better meets the requirements of real-time performance and robustness.

Description

A pupil center localization method based on star rays

technical field

The invention belongs to the technical field of image processing, and in particular relates to a pupil center positioning method based on star rays.

Background technique

Pupil localization is the first and most important step in many computer vision applications. For example, face recognition, facial feature tracking, facial expression analysis, and iris detection and positioning are also inseparable from pupil positioning. The accuracy of pupil center positioning will directly and profoundly affect the next processing and analysis. The first step in gaze tracking is to locate the center of the pupil. Gaze tracking systems provide a powerful analytical tool for real-time cognitive processing and information transfer. There are roughly two important application areas for gaze tracking: diagnostic analysis and human-computer interaction. Eye-tracking systems for diagnosis provide an objective, quantitative, and powerful way to record a reader's gaze. This information provided by eye tracking systems has significant application value in many fields.

At present, pupil center positioning methods are mainly divided into feature-based methods and statistical learning-based methods; feature-based methods are further divided into integral projection-based methods and hough circle detection; shape-based methods mainly use a large amount of data for model training. In order to achieve the effect of centering the pupil. The integral projection method only calculates the gray value of the image, and the calculation amount is small, and it is easily affected by illumination, eyelashes, eyelid occlusion, etc., so the detection error is large. Hough circle detection requires a large amount of calculation and is difficult to meet the requirements of real-time performance. The method based on statistical learning requires a large amount of data to train the model, and the workload is large. Its learning model is relatively complex and cannot meet the requirements of accurately locating the pupil center.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention provides a method for locating pupil center based on star rays.

In order to achieve the above object, the present invention has adopted the following technical solutions:

A pupil center positioning method based on star rays, first through image preprocessing, and then through star ray algorithm, the pupil area is detected, and the binarized threshold is obtained through an iterative method in the ROI area, and the binarized pupil area is optimized. , to eliminate the error of star-ray edge points.

Further, the specific operations of the picture preprocessing are: collecting data through an infrared camera, converting the collected video data into a grayscale image, and performing median filtering on the grayscale image. Median filtering can reduce the influence of eyelashes, which is the basis for effectively extracting the binarized pupil area.

Further, the specific operation of detecting the pupil area through the star ray algorithm is: taking the center point of the pupil area detected by yolov3 as the point inside the pupil area; Yolov3 has a fast speed and a simple model, and can detect small objects well. A real-time object detection model. The pupil area can be detected in real time through yolov3.

Further, the binarized threshold is obtained by an iterative method in the ROI area, and the algorithm steps are as follows:

Step 1, find the average value of the gray level in the pupil area, denoted as a, let the initial threshold be a;

Step 2, the average value of the pixels greater than the threshold value a is recorded as b, and the average value of the pixels less than the threshold value a is recorded as c;

Step 3, find out the new threshold a=(b+c)/2;

Step 4, if a(k)=a(k+1), then the obtained value is the threshold; otherwise, the operation of step 2 is continued to perform iterative calculation. The gray value of the pupil area is lower than that of other areas, so the binarization threshold of the pupil area can be adaptively obtained by the iterative method, and the pupil can be effectively extracted by this method.

Further, the specific operation of optimizing the binarized pupil area is as follows: median filtering is performed on the binarized pupil area to remove the roughness of the edge and make the edge as smooth as possible; fill the reflective area and scan the The line algorithm effectively fills the holes in the pupil area from top to bottom and from left to right. Then take the center point of the pupil area as the center, and send out rays every 20° to get 18 pupil edge points. The edge of the binarized image can be effectively smoothed by median filtering, and more accurate pupil edges can be extracted by the star ray model; the scan line algorithm can effectively fill the holes in the pupil area, which can prevent the extraction of pupil edge points. is the point inside the pupil. The pupil edge points extracted through this step are coarsely extracted pupil edge points.

Further, the edge points with errors in the star rays are eliminated, and the error points are respectively: edge points of eyelid occlusion, edge points of reflective areas, and outlier points of the binarized image.

Still further, the eyelid occlusion edge point is the upper eyelid occlusion edge point, and the method for culling the upper eyelid occlusion edge point is: calculating the slopes between the initial pupil edge points h ₁ , h ₂ , h ₃ ...... ; Calculate the slope of edge points above the pupil center; cull pupil edge points at occlusions. The slope of the pupil edge point where the eyelid is occluded is lower, and the slope of the pupil edge point at the non-occlusion point close to 0° is higher, which is obviously different from the edge point at the occlusion point.

Further, the method for removing outliers from the binarized image is as follows: the pupil is an approximate ellipse, and reasonable pupil edge points should conform to the distribution of the ellipse; the slope of the ray emitted from the center of the ellipse should be the same as the point at the pupil edge. The tangential direction is vertical; the points that are not perpendicular to the slope of the rays emitted by the pupil center point are excluded; and the points with a large error are excluded. Through this step, the edge points of eyelid occlusion and the outlier points of the binarized image can be effectively eliminated, and the calculation amount is small and the speed is fast.

Further, the method for removing edge points in the reflective area is as follows: first, the edge points of the pupil with the error points removed are subjected to ellipse fitting; because the points in the reflective area are closer to the center of the pupil after fitting; , and then culling the edge points of the reflective area. Through this step, the edge points of the reflective area can be effectively eliminated, so that the fitted ellipse is closer to the pupil, and then the center point of the pupil can be effectively located.

Compared with the prior art, the present invention has the following advantages:

By adopting a coarse-to-fine strategy, the pupil center point is refined by gradually eliminating the pupil edge points with larger errors. The present invention can accurately locate the center point of the pupil under the condition of occlusion.

Yolov3 has fast speed, simple model, and can detect small objects very well. It is a real-time target detection model; the iterative method can adaptively obtain the binarization threshold of the pupil area, and can effectively extract the pupil; through median filtering It can effectively smooth the edge of the binarized image, and then use the star ray model to extract more accurate pupil edges; the scan line algorithm can effectively fill the holes in the pupil area, which can prevent the extracted pupil edge points from being inside the pupil. point.

The invention has small calculation amount and high speed, can better satisfy real-time performance, has high pupil center positioning accuracy and high robustness, and can be applied to different individuals.

Description of drawings

Fig. 1 is the flow chart of the present invention;

Fig. 2 is the detection diagram of pupil area by yolov3;

Fig. 3 is the binarization graph by iterative method;

Fig. 4 is the pupil area map after optimized binarization;

Fig. 5 is an effective filling diagram for the cavity in the pupil area;

Fig. 6 is the error map of star ray edge point;

Fig. 7 is the image of culling the pupil edge point of occlusion;

Fig. 8 is the elimination diagram of outliers of the binarized image;

Fig. 9 is the effect of the first fitting and the effect diagram of the second fitting;

FIG. 10 is a diagram showing the accurate detection of the pupil center point of the present invention.

Detailed ways

Example 1

The present invention is a method for locating pupil center based on star rays, and the present invention adopts a 1*7 median filter template.

1 Image preprocessing

2 Detection of pupil area

The pupil area is detected by yolov3, and the detection effect is shown in Figure 2: the center point of the pupil area detected by yolov3 is taken as the point inside the pupil area; taking the center point of the pupil area as the center, rays are emitted every 20°, and 18 pupil edge point. The pupil area can be detected accurately and efficiently by yolov3.

3 Pupil area binarization

The binarization threshold is obtained by an iterative method in the ROI area. The algorithm steps are as follows:

1). Find the average value of the gray level in the area, denoted as a, and set the initial threshold to be a;

2). The average value of the pixels greater than the threshold value a is recorded as b, and the average value of the pixels less than the threshold value a is recorded as c;

3). Find the new threshold a=(b+c)/2;

4). If a(k)=a(k+1), the result is the threshold; otherwise, go to 2 and iterative calculation.

The binarized image through the iterative method is shown in Figure 3:

3.1 Optimize the pupil area after binarization

Median filtering is performed on the binarized pupil area to remove the roughness of the edge and make the edge as smooth as possible. The comparison is shown in Figure 4; it can be seen from Figure 4 that the edge of the pupil becomes smooth after median filtering, which reduces the error of subsequent extraction of pupil edge points. The reflective area is filled, and the hole in the pupil area is effectively filled by the scan line algorithm (from top to bottom, from left to right). The effect is shown in Figure 5; it can be seen from Figure 5 that the area inside the pupil is effectively filled.

4 star ray edge point error:

Points at the edge of eyelid occlusion; edge points in reflective areas; outliers obtained from binarized images;

As shown in Figure 6, it can be seen from Figure 6 that in order to improve the accuracy of the pupil center point, these three pupil edge points must be eliminated.

4.1 Culling of edge points of upper eyelid occlusion

Calculate the slopes h ₁ , h ₂ , h ₃ ...... In order to reduce the amount of calculation, only the slope of the edge point above the pupil center is calculated; the slope of the pupil edge point where the eyelid is occluded is lower and approaches 0. The slope of the pupil edge point in the non-occlusion place is higher, which is obviously different from the edge point in the occlusion place. Figure 7 shows the effect of culling the pupil edge points at the occluded location; it can be seen from Figure 7 that the occlusion points of the upper eyelid can be better culled.

4.2 Removal of outliers in binarized images

The pupil is an approximate ellipse, and a reasonable pupil edge point should conform to the distribution of the ellipse; the slope of the ray emitted from the center of the ellipse should be perpendicular to the tangential direction of the point at the pupil edge; the slope of the ray that is not emitted from the pupil center point is excluded. Vertical points; and points with larger errors; the effect is shown in Figure 8; it can be seen from Figure 8 that this type of error point is better proposed.

4.3 Culling of edge points in reflective areas

It can be seen from Figure 6 that the edge points of the reflective area cannot be eliminated by the slope. First, the edge points of the pupil with the error points are eliminated, and ellipse fitting is performed; because the points in the reflective area are closer to the center of the pupil after fitting; the elimination distance The closest point to the center of the pupil, which is then culled.

The effect of the first fitting and the effect of the second fitting are shown in Figure 9; it can be seen from Figure 9 that the positioning accuracy of the pupil center point has been improved; the pupil center before culling is [471.203,148.868], The pupil center is [470.726,147.615].

Through the culling of the above three steps, the present invention can accurately detect the pupil center point under the occlusion condition, as shown in Figure 10; it can be seen from Figure 10 that the method of this subject is efficient and accurate, and has been tested, it can be better Meet the requirements of real-time and robustness.

Contents that are not described in detail in the specification of the present invention belong to the prior art known to those skilled in the art. Although the illustrative specific embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, As long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the inventive concept are included in the protection list.

Claims (9)

1. a pupil center positioning method based on star ray, it is characterized in that: first through image preprocessing, then through star ray algorithm, detect pupil area, in ROI area through iterative method, obtain the threshold value of binarization, optimize two. Valued pupil area, eliminating star ray edge point errors. 2. a kind of pupil center positioning method based on star rays according to claim 1, is characterized in that: the concrete operation of described picture preprocessing is: collect data by infrared camera, convert the video data collected into grayscale images , do median filtering on grayscale images. 3. a kind of pupil center positioning method based on star ray according to claim 1, is characterized in that: described through star ray algorithm, the concrete operation of detecting pupil area is: the center point of the pupil area that yolov3 detects as Points inside the pupil area; take the center point of the pupil area as the center, and emit rays every 20° to get 18 pupil edge points. 4. a kind of pupil center positioning method based on star rays according to claim 1, is characterized in that: described in ROI area by iterative method, obtains the threshold value of binarization, and algorithm steps are as follows: Step 1, find the average value of the gray level in the pupil area, denoted as a, let the initial threshold be a; Step 2, the average value of the pixels greater than the threshold value a is recorded as b, and the average value of the pixels less than the threshold value a is recorded as c; Step 3, find out the new threshold a=(b+c)/2; Step 4, if a(k)=a(k+1), then the obtained value is the threshold; otherwise, the operation of step 2 is continued to perform iterative calculation. 5. A star-ray-based pupil center positioning method according to claim 1, wherein the specific operation of optimizing the binarized pupil area is: performing a median of the binarized pupil area Filter, remove the roughness of the edge, and make the edge as smooth as possible; fill the reflective area, and effectively fill the hole in the pupil area from top to bottom and from left to right through the scan line algorithm. 6. a kind of pupil center positioning method based on star ray according to claim 1, is characterized in that: described rejecting star ray edge point error, error is respectively: the edge point of eyelid occlusion edge point, reflective area and binary value image outliers. 7. a kind of pupil center positioning method based on star rays according to claim 6, is characterized in that: described eyelid occlusion edge point is upper eyelid occlusion edge point, and the culling method of upper eyelid occlusion edge point is: calculate initial pupil Slopes between edge points h ₁ , h ₂ , h ₃ .......; calculate the slope of edge points above the pupil center; culling pupil edge points at occlusions. 8. a kind of pupil center positioning method based on star rays according to claim 6, is characterized in that: the method for eliminating outliers of described binarized image is: pupil is approximate ellipse, and reasonable pupil edge point should meet The distribution of the ellipse; the slope of the ray emitted from the center of the ellipse should be perpendicular to the tangential direction of the point at the edge of the pupil; the points that are not perpendicular to the slope of the ray emitted from the center of the pupil are eliminated; and the point with a larger error. 9. A star-ray-based pupil center positioning method according to claim 6, characterized in that: the method for removing edge points of the reflective area is: first, ellipse fitting is performed on the edge points of the pupil with the error points removed. ; because the points in the reflective area are closer to the center of the pupil after fitting; the points closest to the center of the pupil are eliminated, and then the edge points in the reflective area are eliminated.

Images