CN112580464A - Method and device for judging iris occlusion of upper eyelid - Google Patents

Abstract

The invention provides a method and a device for judging the iris sheltered by an upper eyelid, wherein the method comprises the following steps: acquiring an upper eyelid boundary and an iris area in continuous face video frames; determining a standard point of an iris area according to a boundary line of the iris area of each frame of the human face video frame; under the condition that continuous shielding video frames exceeding a preset frame number appear, judging that the upper eyelid shields the iris; the continuous occlusion video frame refers to a continuous face video frame of which the upper eyelid boundary passes through the standard point. The standard points of the iris area are determined through the boundary line of the iris area, so that the standard points used for judging whether the occlusion occurs can be correspondingly changed in the moving process of the iris area, the judgment accuracy is guaranteed, meanwhile, the influence of blinking on the judgment accuracy is avoided through the condition that video frames are continuously occluded by preset frames, and the accuracy and the reliability of occlusion judgment are effectively improved.

Description

Method and device for judging that upper eyelid covers iris

technical field

The invention relates to the technical field of data processing, in particular to a method and device for judging that the upper eyelid blocks the iris.

Background technique

The judgment that the upper eyelid covers the iris has extensive application significance in many fields. For example, when detecting driver fatigue, in the prior art, the driver’s fatigue condition is usually judged by whether the upper eyelid covers the iris. When detecting the concentration level of students in listening to lectures in class, the concentration level of students can also be judged by detecting whether the upper eyelids of the students cover the iris.

However, the judgment of the upper eyelid covering the iris in the prior art is determined by judging the relationship between the upper eyelid boundary line and the midline of the eye. However, in many cases, such a detection method is not accurate. For example, when students are in class, they will Staring at the blackboard, and at this time, the position of the iris is often tested, and at this time, if the relationship between the upper eyelid boundary line and the midline of the eye is still used to judge whether there is occlusion, the accuracy will be lost, so how to better achieve the above The judgment that the eyelid covers the iris has become an urgent problem to be solved in the industry.

SUMMARY OF THE INVENTION

The present invention provides a method and a device for judging that the upper eyelid covers the iris, so as to solve the problem that how to better realize the judgment of the upper eyelid covering the iris has become an urgent problem to be solved in the industry.

The present invention provides a method for judging that the upper eyelid covers the iris, comprising:

Obtain the upper eyelid boundary and iris area in consecutive face video frames;

According to the boundary line of the iris area of each frame of face video frame, determine the standard point of the iris area;

In the case of continuous occlusion video frames exceeding a preset number of frames, it is determined that the upper eyelid occludes the iris;

The continuous occlusion video frames refer to the continuous face video frames in which the upper eyelid boundary passes through the standard point.

According to a method for judging that the upper eyelid covers the iris provided by the present invention, the step of determining the standard point of the iris area according to the boundary line of the iris area of each frame of the human face video frame is specifically:

The boundary lines of the iris region are equally divided into multiple sub-boundary lines, and the boundary points of each sub-boundary line are determined;

Determine the intersection between the normals of the boundary points, and perform regression operation through the intersection in the iris area to obtain a standard point in the iris area;

Wherein, the normal line of the boundary point is the normal line of the tangent line of each boundary point.

According to a method for judging that the upper eyelid covers the iris provided by the present invention, it specifically includes:

Through the target detection algorithm, the human eye area in the continuous face video frame is detected;

Through the semantic segmentation model, human eye semantic segmentation is performed on the human eye region in the continuous face video frames, and the iris region, sclera region and upper eyelid boundary of each face video frame are obtained.

According to a method for judging that the upper eyelid blocks the iris provided by the present invention, after the step of obtaining the iris area, the sclera area and the upper eyelid boundary of each frame of the human face video frame, the method further includes:

The iris area and the sclera area are analyzed by the Sobel operator to obtain the boundary line of the iris area.

According to a method for judging that the upper eyelid covers the iris provided by the present invention, before the step of acquiring the upper eyelid boundary and the iris area in the continuous face video frames, the method further includes:

Through the preset image processing algorithm, image optimization processing is performed on the original continuous face video frames to obtain continuous face video frames;

The preset image processing algorithms are: Gaussian smoothing filter denoising method, mean filtering denoising method, median filtering denoising method and optimal filtering denoising method.

According to a method for judging that the upper eyelid covers the iris provided by the present invention,

After the step of determining that the upper eyelid blocks the iris, the method further includes:

A driving fatigue warning signal is generated according to the occlusion.

The present invention also provides a judging device for covering the iris with the upper eyelid, comprising:

The acquisition module is used to acquire the upper eyelid boundary and iris area in continuous face video frames;

A determination module for determining the standard point of the iris region according to the boundary line of the iris region of each frame of human face video frame;

a determination module, configured to determine that the upper eyelid has blocked the iris in the case of continuous occlusion video frames exceeding a preset number of frames;

The continuous occlusion video frames refer to the continuous face video frames in which the upper eyelid boundary passes through the standard point.

The determining module is specifically used for:

The boundary lines of the iris region are equally divided into multiple sub-boundary lines, and the boundary points of each sub-boundary line are determined;

Determine the intersection between the normals of the boundary points, and perform regression operation through the intersection in the iris area to obtain a standard point in the iris area;

Wherein, the normal line of the boundary point is the normal line of the tangent line of each boundary point.

The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, when the processor executes the program, the upper eyelid occlusion as described above is implemented The steps of the iris judgment method.

The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any of the above-mentioned methods for judging that the upper eyelid covers the iris.

The present invention provides a method and device for judging that the iris is blocked by the upper eyelid. By distinguishing the iris area and the sclera area in the human eye area, the standard point of the iris area is determined by the boundary line of the iris area, so that the iris area is moving During the process, the standard points used to judge whether occlusion occurs can also be changed accordingly, so as to ensure the accuracy of judgment, and at the same time, by the condition of continuous occlusion of video frames by a preset number of frames, the influence of blinking on the judgment accuracy can be avoided. Effectively improve the accuracy and reliability of occlusion judgment.

Description of drawings

In order to explain the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is the schematic flow chart of the judgment method of upper eyelid shielding iris provided by the present invention;

2 is a schematic diagram of determining the intersection between the normals of the boundary points provided by the invention;

3 is a schematic diagram of a semantic segmentation effect provided by the present invention;

4 is a schematic diagram of a judging device for the upper eyelid covering the iris provided by the present invention;

FIG. 5 is a schematic diagram of the physical structure of the electronic device provided by the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

1 is a schematic flowchart of a method for judging an upper eyelid covering an iris provided by the present invention, as shown in FIG. 1 , including:

Step S1, obtaining the upper eyelid boundary and the iris area in the continuous face video frame;

Specifically, the upper eyelid boundary described in the present invention specifically refers to the edge position of the upper eyelid that protects the eyeball, and is also the boundary line where the upper eyelid covers the eyeball in the process of closing the eye.

In the present invention, the position of the upper eyelid boundary is obtained to determine the current positional relationship between the upper eyelid and the iris.

The continuous face video frames described in the present invention refer to the video frames split from the face videos captured in continuous time.

Step S2, according to the boundary line of the iris area of each frame of human face video frame, determine the standard point of the iris area;

Specifically, since the position of the iris will change correspondingly with the change of the gaze target, that is to say, the position of the iris area will change. Therefore, according to the boundary line of the iris area of the face video frame, determine the position of the iris area. The standard point is continuously updated according to the position change of the iris area, so as to ensure the accuracy of the occlusion judgment.

Step S3, in the presence of continuous occlusion video frames exceeding the preset number of frames, it is determined that the upper eyelid has occluded the iris;

The continuous occlusion video frames refer to the continuous face video frames in which the upper eyelid boundary passes through the standard point.

In the process of detection, normal blinking often occurs, and the number of blinks is relatively high. During the blinking process, the upper eyelid will inevitably cover the iris, but the occlusion of blinking is not the present invention. To detect the situation, this application uses the screening criteria of continuous occlusion video frames exceeding the preset number of frames to screen out the situation where the upper eyelid occludes the iris caused by blinking. Normal people blink on average more than a dozen times per minute. , usually 2 to 6 seconds to blink once, and each blink takes 0.2 to 0.4 seconds. Different image frames have different numbers. The data targeted by the present invention is the 30 frames per second video shot by the camera, then the present invention will take the occlusion within 15 frames (0.5) seconds as the blinking phenomenon, and will not record it in the result. , so that the accuracy of occlusion judgment can be further improved.

The invention determines the standard point of the iris area by distinguishing the iris area and the sclera area in the human eye area, and determines the standard point of the iris area through the boundary line of the iris area, so that the standard point used for judging whether occlusion occurs during the movement of the iris area can also be used. Corresponding changes are made to ensure the accuracy of the judgment. At the same time, through the condition of continuous occlusion of video frames with a preset number of frames, the influence of blinking on the judgment accuracy is avoided, and the accuracy and reliability of the occlusion judgment are effectively improved.

Optionally, the step of determining the standard point of the iris area according to the boundary line of the iris area of each frame of the face video frame is specifically:

The boundary lines of the iris region are equally divided into multiple sub-boundary lines, and the boundary points of each sub-boundary line are determined;

Determine the intersection between the normals of the boundary points, and perform regression operation through the intersection in the iris area to obtain a standard point in the iris area;

Wherein, the normal line of the boundary point is the normal line of the tangent line of each boundary point.

More specifically, FIG. 2 is a schematic diagram of determining the intersection between the normals of the boundary points provided by the invention. As shown in FIG. 2 , the boundary line of the iris area in the present invention refers to the dividing line that distinguishes the iris area and the sclera area. The dividing line is usually an arc, and the present invention can divide it into multiple sub-boundary lines according to actual needs, that is, dividing the dividing line into multiple segments. At this time, each sub-boundary line has two boundary points. Find each boundary After the tangent line of the point is determined, the normal line of the tangent line of each boundary point is determined, the intersection point between the normal lines is determined, only the intersection point in the iris area is retained, and the pixel coordinates of each intersection point are determined.

According to the pixel coordinates of all intersection points, the regression operation is performed according to the following formula,

与

是回归运算的预测值，y_i与x_i表示所有可能的法线交点(黑点)，n是法线交点的数量。Among them, L is the loss value of the regression operation,

and

is the predicted value of the regression operation, _yi and _xi represent all possible normal intersections (black dots), and n is the number of normal intersections.

与

的求取方法是：遍历以睑裂长度为半径的圆上所有的点，把这些点作为

与

的值代入公式求出L的结果，其中，眼睑长度是根据巩膜最左边的点和最右边的点的横坐标相减进行计算的，然后最终选择L值最小的那个

与

作为最后的坐标G。in,

and

The calculation method is: traverse all the points on the circle with the palpebral fissure length as the radius, and use these points as

and

The value of L is substituted into the formula to obtain the result of L, where the eyelid length is calculated by subtracting the abscissa of the leftmost point and the rightmost point of the sclera, and then the one with the smallest L value is finally selected.

and

as the final coordinate G.

即虹膜的标准点。Traverse all the intersection points, calculate the L value to make it the smallest, and you can get the final pupil center point G

That is, the standard point of the iris.

The present invention determines the standard point of the iris area through the boundary line of the iris area, so that the standard point used for judging whether occlusion occurs can also change correspondingly during the movement of the iris area, thereby ensuring the accuracy of the judgment.

Optionally, the step of acquiring the upper eyelid boundary and the iris area in the continuous face video frames specifically includes:

Through the target detection algorithm, the human eye area in the continuous face video frame is detected;

Through the semantic segmentation model, human eye semantic segmentation is performed on the human eye region in the continuous face video frames, and the iris region, sclera region and upper eyelid boundary of each face video frame are obtained.

Specifically, the target detection algorithm described in the present invention is used to detect the human eye region in the face video frame, which can be specifically implemented by conventional algorithms such as the AdaBoost algorithm or the target recognition framework of yolov3.

Next, for the human eye area, use the current robust semantic segmentation model to segment the human eye part, such as segnet. Figure 3 is a schematic diagram of the semantic segmentation effect provided by the present invention. As shown in Figure 3, each frame of the face is obtained. The iris area and sclera area of the video frame.

The present invention performs semantic segmentation on the part of the human eye through the semantic segmentation model, which is beneficial to the subsequent determination of standard points in the iris region.

Optionally, after the step of obtaining the iris area, the sclera area and the upper eyelid boundary of each frame of the human face video frame, the method further includes:

The iris area and the sclera area are analyzed by the Sobel operator to obtain the boundary line of the iris area.

Specifically, the present invention calculates the neighborhood gradient values of all pixels through the Sobel operator, and uses the gradient direction found in this way, that is, the direction of the normal line of the boundary point, to determine the boundary line of the iris area, that is, the iris area and the sclera. The dividing line of the area.

Optionally, before the step of obtaining the upper eyelid boundary and the iris area in the continuous face video frames, the method further includes:

Through the preset image processing algorithm, image optimization processing is performed on the original continuous face video frames to obtain continuous face video frames;

The preset image processing algorithms are: Gaussian smoothing filter denoising method, mean filtering denoising method, median filtering denoising method and optimal filtering denoising method.

Optionally, after the step of determining that the upper eyelid blocks the iris, the method further includes:

A driving fatigue warning signal is generated according to the occlusion.

Specifically, when it is detected that the upper eyelid covers the iris in the present invention, it is determined that the driver is currently in a fatigued state, and a driving fatigue warning needs to be issued to the driver, so as to protect the safety of driving.

FIG. 4 is a schematic diagram of the judging device for the upper eyelid covering the iris provided by the present invention. As shown in FIG. 4 , it includes: an acquisition module 410, a determination module 420, and a determination module 430; wherein, the acquisition module 410 is used to acquire continuous face video frames. The upper eyelid boundary and the iris area; wherein, the determination module 420 is used to determine the standard point of the iris area according to the boundary line of the iris area of each frame of the human face video frame; In the case of continuous occlusion video frames of the number of frames, it is determined that the upper eyelid occludes the iris; the continuous occlusion video frame refers to the continuous face video frame in which the upper eyelid boundary passes through the standard point.

The determining module is specifically used for:

The boundary lines of the iris region are equally divided into multiple sub-boundary lines, and the boundary points of each sub-boundary line are determined;

Determine the intersection between the normals of the boundary points, and perform regression operation through the intersection in the iris area to obtain a standard point in the iris area;

Wherein, the normal line of the boundary point is the normal line of the tangent line of each boundary point.

The acquisition module is specifically used for:

Through the target detection algorithm, the human eye area in the continuous face video frame is detected;

Through the semantic segmentation model, human eye semantic segmentation is performed on the human eye region in the continuous face video frames, and the iris region, sclera region and upper eyelid boundary of each face video frame are obtained.

The obtaining module is also used for:

The iris area and the sclera area are analyzed by the Sobel operator to obtain the boundary line of the iris area.

The device also includes a preprocessing module, which is specifically used for:

Through the preset image processing algorithm, image optimization processing is performed on the original continuous face video frames to obtain continuous face video frames;

The preset image processing algorithms are: Gaussian smoothing filter denoising method, mean filtering denoising method, median filtering denoising method and optimal filtering denoising method.

The device also includes an early warning module, which is used for:

A driving fatigue warning signal is generated according to the occlusion.

The invention determines the standard point of the iris area by distinguishing the iris area and the sclera area in the human eye area, and determines the standard point of the iris area through the boundary line of the iris area, so that the standard point used for judging whether occlusion occurs during the movement of the iris area can also be used. Corresponding changes are made to ensure the accuracy of the judgment. At the same time, through the condition of continuous occlusion of video frames with a preset number of frames, the influence of blinking on the judgment accuracy is avoided, and the accuracy and reliability of the occlusion judgment are effectively improved.

FIG. 5 is a schematic diagram of the physical structure of the electronic device provided by the present invention. As shown in FIG. 5 , the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540 , wherein the processor 510 , the communication interface 520 , and the memory 530 communicate with each other through the communication bus 540 . The processor 510 can invoke the logic instructions in the memory 530 to execute a method for judging that the upper eyelid covers the iris, the method comprising: acquiring the upper eyelid boundary and the iris area in consecutive human face video frames; The boundary line of the area determines the standard point of the iris area; in the case of continuous occlusion video frames exceeding the preset number of frames, it is determined that the upper eyelid has occluded the iris; the continuous occlusion video frame refers to the The upper eyelid boundary passes through the continuous face video frames of the standard point.

In addition, the above-mentioned logic instructions in the memory 530 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer When executing, the computer can execute the judgment method of the upper eyelid covering the iris provided by the above-mentioned methods, and the method includes: acquiring the upper eyelid boundary and the iris area in the continuous human face video frames; The boundary line is used to determine the standard point of the iris area; in the case of continuous occlusion video frames exceeding the preset number of frames, it is determined that the upper eyelid has occluded the iris; the continuous occlusion video frame refers to the upper eyelid. The eyelid boundary passes through the continuous face video frames of the standard point.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, is implemented to execute the method for judging that the upper eyelid covers the iris provided by the above embodiments , the method comprises: obtaining the upper eyelid boundary and the iris area in the continuous face video frames; determining the standard point of the iris area according to the boundary line of the iris area of each frame of the face video frame; In the case of continuous occlusion video frames, it is determined that the upper eyelid occludes the iris; the continuous occlusion video frame refers to the continuous face video frame in which the upper eyelid boundary passes through the standard point.

The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for judging the iris occlusion of an upper eyelid is characterized by comprising the following steps:

acquiring an upper eyelid boundary and an iris area in continuous face video frames;

determining a standard point of an iris area according to a boundary line of the iris area of each frame of the human face video frame;

under the condition that continuous shielding video frames exceeding a preset frame number appear, judging that the upper eyelid shields the iris;

the continuous occlusion video frame refers to a continuous face video frame of which the upper eyelid boundary passes through the standard point.

2. The method for judging the iris occlusion by the upper eyelid according to claim 1, wherein the step of determining the standard point of the iris area according to the boundary line of the iris area of each frame of the face video frame comprises the following specific steps:

equally dividing the boundary line of the iris area into a plurality of sections of sub-boundary lines, and determining the boundary point of each section of sub-boundary line;

determining intersection points between the normal lines of the boundary points, and performing regression operation through the intersection points in the iris area to obtain iris area standard points;

wherein the boundary point normal is a normal of each boundary point tangent.

3. The method for determining the occlusion of the iris by the upper eyelid according to claim 1, wherein the step of obtaining the upper eyelid boundary and the iris region in the continuous human face video frames specifically comprises:

detecting human eye regions in the continuous human face video frames through a target detection algorithm;

and performing human eye semantic segmentation on human eye regions in the continuous human face video frames through a semantic segmentation model to obtain an iris region, a scleral region and an upper eyelid boundary of each frame of human face video frames.

4. The method for determining the occlusion of the iris by the upper eyelid according to claim 3, wherein after the step of obtaining the iris region, sclera region and upper eyelid boundary of each frame of the human face video, the method further comprises:

and analyzing the iris region and the sclera region through a Sobel operator to obtain a boundary line of the iris region.

5. The method for determining occlusion of an iris by an upper eyelid according to claim 3, wherein before the step of obtaining the upper eyelid boundary and the iris region in consecutive video frames of human faces, the method further comprises:

carrying out image optimization processing on the original continuous face video frames through a preset image processing algorithm to obtain continuous face video frames;

wherein, the preset image processing algorithm is as follows: a Gaussian smoothing filter denoising method, a mean filter denoising method, a median filter denoising method and an optimal filter denoising method.

6. The method for determining occlusion of an iris by an upper eyelid according to claim 3, wherein after the step of determining occlusion of the iris by the upper eyelid, the method further comprises:

and generating a driving fatigue early warning signal according to the shielding.

7. An apparatus for determining occlusion of an iris by an upper eyelid, comprising:

the acquisition module is used for acquiring the upper eyelid boundary and the iris area in the continuous human face video frames;

the determining module is used for determining a standard point of an iris area according to a boundary line of the iris area of each frame of the face video frame;

the judging module is used for judging that the upper eyelid shields the iris under the condition that continuous shielding video frames exceeding a preset frame number appear;

the continuous occlusion video frame refers to a continuous face video frame of which the upper eyelid boundary passes through the standard point.

8. The apparatus for determining occlusion of an iris by an upper eyelid according to claim 7, wherein the determining module is specifically configured to:

equally dividing the boundary line of the iris area into a plurality of sections of sub-boundary lines, and determining the boundary point of each section of sub-boundary line;

determining intersection points between the normal lines of the boundary points, and performing regression operation through the intersection points in the iris area to obtain iris area standard points;

wherein the boundary point normal is a normal of each boundary point tangent.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for determining occlusion of an iris by an upper eyelid according to any one of claims 1 to 6 when executing the program.

10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the method for determining that an upper eyelid occludes an iris according to any one of claims 1 to 6.

Images