CN111178151A - Method and device for realizing human face micro-expression change recognition based on AI technology - Google Patents
Method and device for realizing human face micro-expression change recognition based on AI technology Download PDFInfo
- Publication number
- CN111178151A CN111178151A CN201911251927.1A CN201911251927A CN111178151A CN 111178151 A CN111178151 A CN 111178151A CN 201911251927 A CN201911251927 A CN 201911251927A CN 111178151 A CN111178151 A CN 111178151A
- Authority
- CN
- China
- Prior art keywords
- target object
- time range
- preset time
- video data
- expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method and a device for realizing facial micro-expression change recognition based on an AI technology, wherein the method comprises the following steps: acquiring video data of a target object, and extracting the target video data within a preset time range from the video data; obtaining characteristic values corresponding to the faces of the target objects in each frame of images in the video frames contained in the target video data; and determining the change condition of the internal emotion of the target object in the preset time range according to the size change information of the characteristic value in the preset time range. By adopting the method for realizing the face micro-expression change recognition based on the AI technology, the change situation of the internal emotion of the target object in the preset time range can be quickly determined by detecting the micro-expression change of the target object in the preset time range, and the efficiency and the accuracy of the internal emotion recognition of the target object are improved.
Description
Technical Field
The embodiment of the invention relates to the field of artificial intelligence recognition, in particular to a method and a device for realizing facial micro-expression change recognition based on an AI technology, and further relates to a server and a computer readable storage medium.
Background
The micro expression is a very short facial expression which is revealed when people try to suppress or hide real internal emotion and cannot be controlled autonomously, and compared with a common facial expression, the micro expression has the characteristics of small action amplitude and short dwell time, so that the micro expression is not easy to be perceived, and therefore, the detection and recognition difficulty is high. However, the micro-expression, which is one of the key factors for people to convey true internal mood, usually occurs in the case that people are careless and unable to control and suppress, can explore the true intention and idea of the target object in the state that the target object is unconscious. Therefore, the method for determining the true internal emotion of the target object by identifying the change information of the target object facial micro expression in the preset time range has very important significance for the professional fields of education, safety, criminal investigation, medicine, and the like.
With the rapid development of artificial intelligence technology, how to design a scheme for realizing human face micro-expression change recognition based on an AI technology becomes a technical problem which needs to be solved urgently in the field.
Disclosure of Invention
Therefore, the embodiment of the invention provides a method for realizing face micro-expression change recognition based on an AI technology, so as to solve the problem that the change situation of the mood of a target object cannot be accurately determined due to the fact that the operation steps of the face micro-expression recognition process are complicated and are not intuitive enough in the prior art.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
in a first aspect, an embodiment of the present invention provides a method for implementing facial micro-expression change recognition based on an AI technology, including:
the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data;
obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
Further, the characteristic value is a probability value for representing the mood of the target object.
Further, the characteristic values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
Further, the determining, according to the size change information of the feature value within the preset time range, a change condition of the mood of the target object within the preset time range specifically includes:
determining range thresholds of different types of characteristic values respectively corresponding to the target object according to the age characteristic information, the occupation characteristic information and the gender information of the target object;
and determining the change condition of the internal mood of the target object in the preset time range according to the size change information of the characteristic value in the preset time range and the range threshold value.
Further, the method for realizing facial micro-expression change recognition based on the AI technology further comprises:
and outputting a dynamic change curve graph for displaying the change condition of the internal emotion of the target object according to the change condition of the internal emotion of the target object within the preset time range.
In a second aspect, an embodiment of the present invention further provides a device for implementing facial micro-expression change recognition based on an AI technology, including:
the target video data extraction unit is used for acquiring video data of a target object through a preset image acquisition device and extracting target video data within a preset time range from the video data;
the characteristic value obtaining unit is used for obtaining video frames contained in the target video data and obtaining characteristic values corresponding to the faces of the target objects in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and the internal emotion determining unit is used for obtaining the size change information of the characteristic value within the preset time range and determining the change condition of the internal emotion of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
Further, the characteristic value is a probability value for representing the mood of the target object;
the characteristic values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
Further, the apparatus for implementing face micro-expression change recognition based on AI technology further includes:
and the output unit is used for outputting a dynamic change curve chart for displaying the change condition of the internal emotion of the target object according to the change condition of the internal emotion of the target object in the preset time range.
Further, the internal emotion determining unit is specifically configured to:
determining range thresholds of different types of characteristic values respectively corresponding to the target object according to the age characteristic information, the occupation characteristic information and the gender information of the target object;
and determining the change condition of the internal mood of the target object in the preset time range according to the size change information of the characteristic value in the preset time range and the range threshold value.
In a third aspect, an embodiment of the present invention further provides a server, including:
a processor; and
the memory is used for storing a program of the method for realizing the facial micro-expression change recognition based on the AI technology, and after the equipment is powered on and runs the program of the method for realizing the facial micro-expression change recognition based on the AI technology through the processor, the following steps are executed:
the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data;
obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium contains one or more program instructions, and the one or more program instructions are configured to be executed by a server to implement a method for recognizing a change in a facial micro-expression based on an AI technique as described in any one of the above.
By adopting the method for realizing the face micro-expression change recognition based on the AI technology, the change situation of the internal emotion of the target object in the preset time range can be quickly determined by detecting the micro-expression change of the target object in the preset time range, the efficiency and the accuracy of the internal emotion recognition of the target object are improved, and the internal emotion of the target object is more intuitively presented.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
Fig. 1 is a flowchart of a method for implementing facial micro-expression change recognition based on an AI technique according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of an apparatus for implementing facial micro-expression change recognition based on AI technology according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a server according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided for illustrative purposes, and other advantages and effects of the present invention will become apparent to those skilled in the art from the present disclosure.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
The following describes an embodiment of the method for recognizing the micro-expression change of the human face based on the AI technology. As shown in fig. 1, which is a flowchart of a method for implementing facial micro-expression change recognition based on AI technology according to an embodiment of the present invention, the specific implementation process includes the following steps:
step S101: the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data.
In the embodiment of the invention, the image acquisition device can be a video recorder, a video camera, a smart phone with a video recording function and other equipment. The target object may refer to a person to be identified in the area to be detected. The video data typically contains within it at least one facial image of the target object.
In the implementation process, the duration of the micro expression is usually very short, only 0.04 second to 0.25 second, most people often can not detect the micro expression, and the micro expression is one of the most key factors reflecting the actual emotion of one person. The micro expression of the target object can be effectively obtained based on the existing Artificial Intelligence (Artificial Intelligence) technology. Specifically, the invention can accurately capture the micro-expression of the target object by utilizing the face recognition technology in the existing artificial intelligence technology, and the real internal mood of the target object can be obtained by further analyzing and processing. Therefore, the server side usually extracts video data within a preset time range from the video data as target video data, for example: video data within 0.04 seconds to 0.25 seconds is extracted from the video data as target video data.
Step S102: obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object.
After extracting the target video data within the preset time range from the video data in step S101, data preparation is performed for this step, and further, in this step, the server side may obtain the video frames included in the target video data based on the Python technology, and further obtain feature values respectively corresponding to the faces of the target object in each frame image in the video frames by using a preset recognition model.
In an actual implementation process, the recognition model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object, and the recognition model is a facial micro-expression recognition model based on a deep convolutional neural network, which is obtained by training through a large number of training samples in advance and then continuously adjusting internal parameters. It should be noted that the micro expression may mean that people express the feeling of mind or emotion by making some expressions, and other fine expression change information is "revealed" by the face between different expressions or in a certain expression. The minimum duration of micro expression is 1/25 second, and although a slight change in expression may last only a moment, it sometimes just expresses the opposite emotion to the masking expression exhibited by the face. In the process of actually recognizing the face based on the AI technology, compared with expressions consciously made by people, the micro expression can more reflect the real feeling or emotion of people. Micro-expressions tend to refer more in application to those expressions that are suppressed. For example, in the case of apparent sadness, a person exhibits a mostly sad expression, but the corners of the mouth are inhibited from rising unimpaired. At this time, the person apparently showed sad emotions, but rather presented smiling micro-expressions involuntarily. The expression is not obvious or transient due to physiological inhibition. Differences like this are more common in micro-expression analysis.
It should be noted that in the implementation, the happy or pleasant facial micro-expression may include: the mouth corners are raised, the cheeks are raised and wrinkled, the eyelids are contracted, and the tail parts of the eyes can form fishtail lines and the like. Facial micro-expressions at sadness may include: squinting, tightening of eyebrows, pulling down the corners of the mouth, lifting or tightening the chin, etc. The micro-expressions of the face when angry may include: eyebrow sagging, forehead wrinkles, eyelid and lip tension, etc. Facial micro-expressions at panic may include: open mouth and eyes, raise eyebrows, open nostrils, etc. The facial micro-expressions at fidgeting may include: lifting the upper lip, drooping eyebrows, squinting, etc.
For example, if a person naturally expresses a happy expression and does not contain a micro-expression, it can be concluded that the person is a happy emotion or feeling. But if a subtle michler's micro-emotional flash is caught in the meantime, it is generally considered that the "happy" face actually shows a subtle or incredible mood.
Step S103: and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
After the feature values respectively corresponding to the face of the target object in each frame of image in the video frame are obtained in step S102, data preparation is performed for this step, and further, size change information of the feature values in the preset time range may be obtained in this step, so as to determine a change condition of the mood of the target object in the preset time range.
In an embodiment of the present invention, the feature value is a probability value representing a mood of the target object. The characteristic values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
In practical implementation, the server side usually extracts the size change information of the feature value within 0.04 second to 0.25 second from a piece of target video data. The following description will be given by taking the first type of characteristic value as an example, for example: the first-class feature value is 90% at 0.04 seconds, 80% at 0.08 seconds, 60% at 0.12 seconds, 30% at 0.16 seconds, 75% at 0.20 seconds, and 90% at 0.25 seconds, because the first-class feature value is used to represent the degree of mental pleasure of the target object, and the first-class feature value is obtained by the recognition model according to the facial micro-expression of the target object. Therefore, it can be concluded that the target object undergoes a process of re-rising from high to low in a very short time to a pleasant degree.
Of course, the embodiment of the present invention is not limited to the first type of feature values listed above, and a plurality of feature values corresponding to the face of the target object in each frame of image in the video frame and size change information of the plurality of feature values in a preset time range may be obtained simultaneously by using a preset recognition model.
In addition, the server side can also determine range thresholds of different types of characteristic values corresponding to the target objects respectively according to set age characteristic information, occupation characteristic information and gender information of the target objects in advance, and determine the change condition of the mood of the target objects in the preset time range according to the size change information of the characteristic values in the preset time range and the range thresholds.
Further, the server side can also output a dynamic change curve graph for displaying the change condition of the mood of the target object according to the change condition of the mood of the target object within the preset time range. For example, the X-axis is time, the Y-axis is the magnitude of the first class eigenvalue, the first class eigenvalue is 90% at 0.04 seconds, the first class eigenvalue is 80% at 0.08 seconds, the first class eigenvalue is 60% at 0.12 seconds, the first class eigenvalue is 30% at 0.16 seconds, the first class eigenvalue is 75% at 0.20 seconds, and the first class eigenvalue is 90% at 0.25 seconds.
By adopting the method for realizing the face micro-expression change recognition based on the AI technology, the change situation of the internal emotion of the target object in the preset time range can be quickly determined by detecting the micro-expression change of the target object in the preset time range, the efficiency and the accuracy of the internal emotion recognition of the target object are improved, and the internal emotion of the target object is more intuitively presented.
Corresponding to the method for realizing the facial micro expression change recognition based on the AI technology, the invention also provides a device for realizing the facial micro expression change recognition based on the AI technology. Since the embodiment of the device is similar to the embodiment of the method described above, the description is relatively simple, and for the relevant points, reference may be made to the description of the embodiment of the method described above, and the embodiment of the device for implementing face micro-expression change recognition based on the AI technology described below is only illustrative. Fig. 2 is a schematic diagram of an apparatus for implementing facial micro-expression change recognition based on AI technology according to an embodiment of the present invention.
The invention relates to a device for realizing face micro-expression change recognition based on AI technology, which comprises the following parts:
a target video data extraction unit 201, configured to obtain video data of a target object through a preset image capture device, and extract target video data within a preset time range from the video data;
in the embodiment of the invention, the image acquisition device can be a video recorder, a video camera, a smart phone with a video recording function and other equipment. The target object may refer to a person to be identified in the area to be detected. The video data typically contains within it at least one facial image of the target object.
In the implementation process, the duration of the micro expression is usually very short, only 0.04 second to 0.25 second, most people often can not detect the micro expression, and the micro expression is one of the most key factors reflecting the actual emotion of one person. The micro expression of the target object can be effectively obtained based on the existing Artificial Intelligence (Artificial Intelligence) technology. Specifically, the invention can accurately capture the micro-expression of the target object by utilizing the face recognition technology in the existing artificial intelligence technology, and the real internal mood of the target object can be obtained by further analyzing and processing. Therefore, the server side usually extracts video data within a preset time range from the video data as target video data, for example: video data within 0.04 seconds to 0.25 seconds is extracted from the video data as target video data.
A feature value obtaining unit 202, configured to obtain video frames included in the target video data, and obtain feature values corresponding to the faces of the target object in each frame of image in the video frames by using a preset recognition model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
in an actual implementation process, the recognition model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object, and the recognition model can be obtained by training in advance through a large number of training samples and further continuously adjusting internal parameters. It should be noted that the micro expression may mean that people express the feeling of mind or emotion by making some expressions, and other fine expression change information is "revealed" by the face between different expressions or in a certain expression. The minimum duration of micro expression is 1/25 second, and although a slight change in expression may last only a moment, it sometimes just expresses the opposite emotion to the masking expression exhibited by the face. In the process of actually recognizing the face based on the AI technology, compared with expressions consciously made by people, the micro expression can more reflect the real feeling or emotion of people. Micro-expressions tend to refer more in application to those expressions that are suppressed. For example, in the case of apparent sadness, a person exhibits a mostly sad expression, but the corners of the mouth are inhibited from rising unimpaired. At this time, the person apparently showed sad emotions, but rather presented smiling micro-expressions involuntarily. The expression is not obvious or transient due to physiological inhibition. Differences like this are more common in micro-expression analysis.
It should be noted that in the implementation, the happy or pleasant facial micro-expression may include: the mouth corners are raised, the cheeks are raised and wrinkled, the eyelids are contracted, and the tail parts of the eyes can form fishtail lines and the like. Facial micro-expressions at sadness may include: squinting, tightening of eyebrows, pulling down the corners of the mouth, lifting or tightening the chin, etc. The micro-expressions of the face when angry may include: eyebrow sagging, forehead wrinkles, eyelid and lip tension, etc. Facial micro-expressions at panic may include: open mouth and eyes, raise eyebrows, open nostrils, etc. The facial micro-expressions at fidgeting may include: lifting the upper lip, drooping eyebrows, squinting, etc.
For example, if a person naturally expresses a happy expression and does not contain a micro-expression, it can be concluded that the person is a happy emotion or feeling. But if a subtle michler's micro-emotional flash is caught in the meantime, it is generally considered that the "happy" face actually shows a subtle or incredible mood.
And the internal emotion determining unit 203 is configured to obtain size change information of the feature value within the preset time range, and determine a change condition of the internal emotion of the target object within the preset time range according to the size change information of the feature value within the preset time range.
In an embodiment of the present invention, the feature value is a probability value representing a mood of the target object. The characteristic values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
In practical implementation, the server side usually extracts the size change information of the feature value within 0.04 second to 0.25 second from a piece of target video data. The following description will be given by taking the first type of characteristic value as an example, for example: the first-class feature value is 90% at 0.04 seconds, 80% at 0.08 seconds, 60% at 0.12 seconds, 30% at 0.16 seconds, 75% at 0.20 seconds, and 90% at 0.25 seconds, because the first-class feature value is used to represent the degree of mental pleasure of the target object, and the first-class feature value is obtained by the recognition model according to the facial micro-expression of the target object. Therefore, it can be concluded that the target object undergoes a process of re-rising from high to low in a very short time to a pleasant degree.
Of course, the embodiment of the present invention is not limited to the first type of feature values listed above, and a plurality of feature values corresponding to the face of the target object in each frame of image in the video frame and size change information of the plurality of feature values in a preset time range may be obtained simultaneously by using a preset recognition model.
In addition, the server side can also determine range thresholds of different types of characteristic values corresponding to the target objects respectively according to set age characteristic information, occupation characteristic information and gender information of the target objects in advance, and determine the change condition of the mood of the target objects in the preset time range according to the size change information of the characteristic values in the preset time range and the range thresholds.
Further, the server side can also output a dynamic change curve graph for displaying the change condition of the mood of the target object according to the change condition of the mood of the target object within the preset time range. For example, the X-axis is time, the Y-axis is the magnitude of the first class eigenvalue, the first class eigenvalue is 90% at 0.04 seconds, the first class eigenvalue is 80% at 0.08 seconds, the first class eigenvalue is 60% at 0.12 seconds, the first class eigenvalue is 30% at 0.16 seconds, the first class eigenvalue is 75% at 0.20 seconds, and the first class eigenvalue is 90% at 0.25 seconds.
By adopting the device for realizing the face micro-expression change recognition based on the AI technology, the change situation of the internal emotion of the target object in the preset time range can be quickly determined by detecting the micro-expression change of the target object in the preset time range, the efficiency and the accuracy of the internal emotion recognition of the target object are improved, and the internal emotion of the target object is more intuitively presented.
Corresponding to the method for realizing the facial micro-expression change recognition based on the AI technology, the invention also provides a computer readable storage medium. Since the embodiment of the computer-readable storage medium is similar to the above-mentioned method embodiment, the description is simple, and please refer to the description of the above-mentioned method embodiment for relevant points, and the computer-readable storage medium described below is only an exemplary one.
The invention provides a computer storage medium containing one or more program instructions for executing the method for realizing the facial microexpression change recognition based on the AI technology.
Fig. 3 is a schematic diagram of a server according to an embodiment of the present invention. In particular, the server may include a processor 301 and a memory 302; the memory 302 is configured to store a program of a method for implementing facial micro-expression change recognition based on an AI technology, and after the device is powered on and the processor 301 runs the program of the method for implementing facial micro-expression change recognition based on the AI technology, the following steps are performed:
the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data;
obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range. In addition, the server of the present invention may further execute any one of the above methods for recognizing the change of the facial micro-expression based on the AI technology.
In an embodiment of the invention, the processor or processor module may be an integrated circuit chip having signal processing capabilities. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (ddr Data Rate SDRAM), Enhanced SDRAM (ESDRAM), synclink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method for realizing face micro-expression change recognition based on AI technology is characterized by comprising the following steps:
the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data;
obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
2. The AI-technology-based method for implementing facial micro-expression change recognition according to claim 1, wherein the feature values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
3. The AI-technology-based method for realizing face micro-expression change recognition according to claim 1, wherein the determining, according to the size change information of the feature values within the preset time range, the change of the mood of the target object within the preset time range specifically includes:
determining range thresholds of different types of characteristic values respectively corresponding to the target object according to the age characteristic information, the occupation characteristic information and the gender information of the target object;
and determining the change condition of the internal mood of the target object in the preset time range according to the size change information of the characteristic value in the preset time range and the range threshold value.
4. The AI-based method for implementing facial micro-expression change recognition according to claim 1, further comprising:
and outputting a dynamic change curve graph for displaying the change condition of the internal emotion of the target object according to the change condition of the internal emotion of the target object within the preset time range.
5. The AI-based method for implementing facial micro-expression change recognition according to claim 1, wherein the feature value is a probability value representing the corresponding mood of the target object.
6. A device for realizing face micro-expression change recognition based on AI technology is characterized by comprising:
the target video data extraction unit is used for acquiring video data of a target object through a preset image acquisition device and extracting target video data within a preset time range from the video data;
the characteristic value obtaining unit is used for obtaining video frames contained in the target video data and obtaining characteristic values corresponding to the faces of the target objects in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and the internal emotion determining unit is used for obtaining the size change information of the characteristic value within the preset time range and determining the change condition of the internal emotion of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
7. The AI-based technology enabled device for recognizing micro-episodic changes of human faces according to claim 6, wherein the characteristic value is a probability value representing the corresponding inner emotion of the target object;
the characteristic values specifically include: at least one of a first class of feature values for representing the degree of mental joy of the target object, a second class of feature values for representing the degree of mental anger of the target object, a third class of feature values for representing the degree of mental sadness of the target object, a fourth class of feature values for representing the degree of mental dysphoria of the target object, and a fifth class of feature values for representing the degree of mental startle of the target object.
8. The AI-technology-based apparatus for implementing facial micro-expression change recognition according to claim 6, further comprising:
and the output unit is used for outputting a dynamic change curve chart for displaying the change condition of the internal emotion of the target object according to the change condition of the internal emotion of the target object in the preset time range.
9. A server, comprising:
a processor; and
the memory is used for storing a program of the method for realizing the facial micro-expression change recognition based on the AI technology, and after the equipment is powered on and runs the program of the method for realizing the facial micro-expression change recognition based on the AI technology through the processor, the following steps are executed:
the method comprises the steps of obtaining video data of a target object through a preset image acquisition device, and extracting target video data within a preset time range from the video data;
obtaining video frames contained in the target video data, and obtaining characteristic values corresponding to the faces of the target object in each frame of image in the video frames by using a preset identification model; wherein the identification model is a characteristic value which is determined according to the facial micro-expression of the target object in a continuous time range and is used for representing the internal emotion of the target object;
and obtaining the size change information of the characteristic value within the preset time range, and determining the change condition of the internal mood of the target object within the preset time range according to the size change information of the characteristic value within the preset time range.
10. A computer-readable storage medium containing one or more program instructions for executing, by a server, the method for implementing facial microexpression change recognition based on AI techniques as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911251927.1A CN111178151A (en) | 2019-12-09 | 2019-12-09 | Method and device for realizing human face micro-expression change recognition based on AI technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911251927.1A CN111178151A (en) | 2019-12-09 | 2019-12-09 | Method and device for realizing human face micro-expression change recognition based on AI technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111178151A true CN111178151A (en) | 2020-05-19 |
Family
ID=70657155
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911251927.1A Pending CN111178151A (en) | 2019-12-09 | 2019-12-09 | Method and device for realizing human face micro-expression change recognition based on AI technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111178151A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597938A (en) * | 2020-12-29 | 2021-04-02 | 杭州海康威视系统技术有限公司 | Expression detection method and device, electronic equipment and storage medium |
| CN114241565A (en) * | 2021-12-15 | 2022-03-25 | 北京易华录信息技术股份有限公司 | Facial expression and target object state analysis method, device and equipment |
| CN116844208A (en) * | 2023-07-03 | 2023-10-03 | 北京捷通华声科技股份有限公司 | Emotion prediction method and device based on image analysis |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271930A (en) * | 2018-09-14 | 2019-01-25 | 广州杰赛科技股份有限公司 | Micro- expression recognition method, device and storage medium |
| WO2019029261A1 (en) * | 2017-08-07 | 2019-02-14 | 深圳市科迈爱康科技有限公司 | Micro-expression recognition method, device and storage medium |
| CN110147729A (en) * | 2019-04-16 | 2019-08-20 | 深圳壹账通智能科技有限公司 | User emotion recognition methods, device, computer equipment and storage medium |
| CN110210289A (en) * | 2019-04-19 | 2019-09-06 | 平安科技（深圳）有限公司 | Emotion identification method, apparatus, computer readable storage medium and electronic equipment |
-
2019
- 2019-12-09 CN CN201911251927.1A patent/CN111178151A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019029261A1 (en) * | 2017-08-07 | 2019-02-14 | 深圳市科迈爱康科技有限公司 | Micro-expression recognition method, device and storage medium |
| CN109271930A (en) * | 2018-09-14 | 2019-01-25 | 广州杰赛科技股份有限公司 | Micro- expression recognition method, device and storage medium |
| CN110147729A (en) * | 2019-04-16 | 2019-08-20 | 深圳壹账通智能科技有限公司 | User emotion recognition methods, device, computer equipment and storage medium |
| CN110210289A (en) * | 2019-04-19 | 2019-09-06 | 平安科技（深圳）有限公司 | Emotion identification method, apparatus, computer readable storage medium and electronic equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597938A (en) * | 2020-12-29 | 2021-04-02 | 杭州海康威视系统技术有限公司 | Expression detection method and device, electronic equipment and storage medium |
| CN114241565A (en) * | 2021-12-15 | 2022-03-25 | 北京易华录信息技术股份有限公司 | Facial expression and target object state analysis method, device and equipment |
| CN114241565B (en) * | 2021-12-15 | 2025-05-27 | 北京易华录信息技术股份有限公司 | A method, device and equipment for analyzing facial expression and target object state |
| CN116844208A (en) * | 2023-07-03 | 2023-10-03 | 北京捷通华声科技股份有限公司 | Emotion prediction method and device based on image analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bandini et al. | Analysis of facial expressions in parkinson's disease through video-based automatic methods | |
| CN108388876B (en) | An image recognition method, device and related equipment | |
| US10445562B2 (en) | AU feature recognition method and device, and storage medium | |
| KR102286468B1 (en) | Method and system for integrity verification of fake video created through deep learning | |
| US10311289B2 (en) | Face recognition method and device and apparatus | |
| US9405962B2 (en) | Method for on-the-fly learning of facial artifacts for facial emotion recognition | |
| CN101334839B (en) | Image processing device and image processing method | |
| WO2020253349A1 (en) | Image recognition-based driving behavior warning method and apparatus, and computer device | |
| CN112417414B (en) | A privacy protection method, device and equipment based on attribute desensitization | |
| CN108182714B (en) | Image processing method and device and storage medium | |
| CN110121715A (en) | Method, device, electronic device and storage medium for calling for help based on facial expression recognition | |
| WO2019095571A1 (en) | Human-figure emotion analysis method, apparatus, and storage medium | |
| WO2016129192A1 (en) | Emotion estimation device and emotion estimation method | |
| CN111183455A (en) | Image data processing system and method | |
| CN111144293A (en) | Human face identity authentication system with interactive living body detection and method thereof | |
| CN111373408B (en) | Expression recognition device | |
| CN111178151A (en) | Method and device for realizing human face micro-expression change recognition based on AI technology | |
| CN107820017A (en) | Image capturing method, device, computer-readable recording medium and electronic equipment | |
| CN110046955A (en) | Marketing method, device, computer equipment and storage medium based on recognition of face | |
| KR102439216B1 (en) | Mask-wearing face recognition method and server using artificial intelligence deep learning model | |
| KR101939772B1 (en) | Method and apparatus for inferring facial emotion recognition, system for inferring facial emotion, and media for recording computer program | |
| CN109409322B (en) | Living body detection method, device, face recognition method and face detection system | |
| Zhao et al. | Applying contrast-limited adaptive histogram equalization and integral projection for facial feature enhancement and detection | |
| KR20130117624A (en) | Method and apparatus for detecting talking segments in a video sequence using visual cues | |
| CN110287765A (en) | Baby appearance prediction method, device and storage medium based on face recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230428 Address after: 230071 Comprehensive Building 3-2985, No. 55 Science Avenue, High tech Zone, Shushan District, Hefei City, Anhui Province Applicant after: Hefei Kelast Network Technology Co.,Ltd. Address before: 100021 6235, 6th floor, jinyayuan crossing building, YunhuiLi, Haidian District, Beijing Applicant before: QUANTUM CLOUD FUTURE (BEIJING) INFORMATION TECHNOLOGY CO.,LTD. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200519 |
|
| RJ01 | Rejection of invention patent application after publication |