CN111476198A - Gait recognition method, device and system based on artificial intelligence, storage medium and server - Google Patents
Gait recognition method, device and system based on artificial intelligence, storage medium and server Download PDFInfo
- Publication number
- CN111476198A CN111476198A CN202010334982.3A CN202010334982A CN111476198A CN 111476198 A CN111476198 A CN 111476198A CN 202010334982 A CN202010334982 A CN 202010334982A CN 111476198 A CN111476198 A CN 111476198A
- Authority
- CN
- China
- Prior art keywords
- gait
- video
- human body
- module
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a gait recognition method, a device, a system, a storage medium and a server based on artificial intelligence, wherein the gait recognition method comprises the following steps: A. identifying a human body in the video; B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence; C. learning and extracting gait features in the video sequence; D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result. The gait recognition device comprises a moving human body recognition module, a human body joint tracking module, a self-coding learning module, a gait feature recognition module and a storage module. The gait recognition system comprises a video acquisition terminal, a use terminal and a server. The method is based on the artificial intelligence technology, avoids invalid search of the optimal gait feature in a high-dimensional space caused by manually selecting the gait feature, and realizes robust and complete extraction of the optimal gait feature to the maximum extent.
Description
Technical Field
The invention belongs to the technical field of biological feature recognition, and particularly relates to a gait recognition method, device, system, storage medium and server based on artificial intelligence.
Background
The gait refers to the posture of the human body when walking and all the movements of walking, and the gait information includes but is not limited to the information such as the step frequency, the step length, the lower limb swing angle and the acceleration when walking. In general, a human gait can consist of 7 time-sequential phases, respectively: heel strike, ball strike, heel lift, toe-off, swing prophase, swing metaphase, and swing caudal (heel strike), these 7 phases constitute a basic gait cycle. The gait cycle of each person represents unique biological characteristics, so that the gait is the same as the characteristics of fingerprints, irises, faces and the like, and can be used as the biological characteristics of the human body for identifying and locking natural persons. At present, gait recognition can be matched with face recognition, iris recognition and the like and is applied to access control recognition, public security skynet monitoring systems and the like, and the gait of a person with unknown identity is searched in a database for the gait of the person matched with the person, so that the identity of the person is confirmed.
At present, most of the gait cycle and gait sequence feature extraction is manually selected, only the features of the gait to be extracted can be manually selected, the gait features, such as the walking step frequency feature, the span feature or the swing duration feature, are selected from the video through a corresponding program or algorithm, and the special gait features of the human body are extracted from the video through the data of the target features through a tracking algorithm. The characteristic space of the gait characteristics is generally a high-dimensional space, the characteristics of the gait characteristics comprise both space dimensions and time dimensions, the gait characteristics are artificially determined, and the characteristic vector which can represent the most human gait characteristics cannot be accurately judged, so that when the gait characteristics are selected from the characteristic space of a gait sequence, the probability of omitting the most valuable characteristics and representing the most accurate human gait characteristics is very high, and the possibility of obtaining the optimal gait characteristics is relatively low. Meanwhile, these deficiencies are also one of the important reasons that the current gait recognition technology cannot be utilized and deployed on a large scale.
Accordingly, the prior art is in need of improvement and development.
Disclosure of Invention
The invention provides a gait recognition method, a device, a system, a storage medium and a server based on artificial intelligence, which automatically extracts and recognizes gait features through an intelligent module, avoids invalid search of optimal gait features in a high-dimensional space caused by extracting the gait features by an artificial method, and realizes robust and complete extraction of the optimal gait features to the greatest extent.
In order to solve the technical problem, the gait recognition method based on artificial intelligence provided by the invention comprises the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait features comprise position information of the primary joint in the video frame and motion information of the primary joint in the video sequence;
D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
Further, the step d compares the gait characteristics with gait characteristics preset in the storage module, and the identification result is obtained as follows: calculating the distance of the gait feature in a feature space relative to the gait feature preset in the storage module through a measurement function, wherein if the distance is in an interval [ a, 1], the gait feature is the gait feature of different angles of the human body to which the gait feature preset in the storage module belongs, and if the distance is in an interval [0, a ], the gait feature is not matched with the gait feature preset in the storage module; the gait characteristics of the same human body preset in the storage module are in multi-element Gaussian distribution in the characteristic space.
Further, step Bn. is included after step B to interpolate the video to generate a gait video frame with a frame number greater than or equal to 20; and B1, judging the gait duration of the human body in the video before executing the step Bn, and executing the step Bn if the gait duration is less than 0.5 s.
Further, the above-mentioned main joints include ankle joints, hip joints, knee joints, wrist joints, elbow joints and shoulder joints.
Further, in the step a, in identifying the human bodies in the video, the human bodies identified from the video all generate corresponding ID numbers.
The invention provides a gait recognition device based on artificial intelligence, which comprises a moving human body recognition module, a human body joint tracking module, a self-coding learning module, a gait feature recognition module and a storage module;
the motion human body recognition module is used for recognizing a human body from the video;
the human body joint tracking module is connected with the moving human body identification module and is used for identifying and tracking main joints of the human body and storing the position information of the main joints in a video frame to form a video sequence;
the self-coding learning module is connected with the human joint tracking module and is used for learning and extracting gait features in the video sequence; the gait characteristics comprise position information of the main joints in the video frames and motion information of the main joints in the video sequence;
the gait feature recognition module is connected with the self-coding learning module and used for comparing the gait features with gait features preset in the storage module to obtain a recognition result;
the storage module and the gait feature recognition module are used for storing preset gait features.
Further, the gait recognition device based on artificial intelligence also comprises a gait video frame generation module, wherein the gait video frame generation module is connected with the moving human body recognition module and used for interpolating a gait video frame with the frame number being more than or equal to 20 in the video when the time length of the human body with gait in the video is less than 0.5 s.
The gait recognition system based on artificial intelligence provided by the invention comprises a video acquisition terminal, a use terminal and a server;
the video acquisition terminal is used for acquiring a video of human body movement;
the server is used for extracting the gait features in the video and comparing the gait features with the gait features preset in the storage module to obtain an identification result;
the use terminal is used for monitoring the gait recognition process and outputting a recognition result.
The present invention provides a storage medium in which a computer program is stored, which, when run on a computer, causes the computer to execute the artificial intelligence based gait recognition method described above.
The server provided by the invention comprises a processor and a memory, wherein the memory is stored with a computer program, and the processor is used for executing the gait recognition method based on artificial intelligence by calling the computer program stored in the memory.
The gait recognition method based on artificial intelligence converts a human body into a virtual human body formed by the mutual connection of joint points, characterizes the gait characteristics of the human body by the motion data of the joints of the human body, improves the identifiability for the extraction of the subsequent gait characteristics, and avoids omitting the characteristics which are most valuable and can most accurately characterize the gait of the human body. The gait recognition device based on artificial intelligence accurately recognizes the human body and the main joints thereof through the moving human body recognition module and the human body joint tracking module, and accurately tracks to obtain the motion of the main joints; the self-coding learning module replaces manual design and gait feature extraction, so that the optimal gait feature can be accurately retrieved and extracted from a high-dimensional feature space, the complete and robust extraction of the gait feature is realized, and meanwhile, high-dimensional time and space features are converted into low-dimensional feature vectors, so that the calculation burden during gait feature retrieval is greatly reduced; and finally, the gait feature recognition module is used for realizing recognition, so that the recognition precision is improved.
Drawings
Fig. 1 is a flow chart of the gait recognition method based on artificial intelligence of the invention.
Fig. 2 is another flowchart of the gait recognition method based on artificial intelligence shown in fig. 1.
Fig. 3 is a schematic structural diagram of the gait recognition device based on artificial intelligence of the invention.
Fig. 4 is another schematic diagram of the artificial intelligence based gait recognition device shown in fig. 3.
Fig. 5 is another schematic diagram of the artificial intelligence based gait recognition device shown in fig. 4.
Fig. 6 is a schematic structural diagram of an artificial intelligence-based gait recognition system according to the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; may be mechanically connected, may be electrically connected or may be in communication with each other; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature being "on," "above" and "over" the second feature includes the first feature being directly on and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly under and obliquely below the second feature, or simply meaning that the first feature is at a lesser elevation than the second feature.
The following disclosure provides many different embodiments or examples for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art may recognize applications of other processes and/or uses of other materials.
As shown in fig. 1, the gait recognition method based on artificial intelligence of the invention comprises the following steps:
A. a human body in the video is identified.
In the step, a trained motion human body recognition module can be adopted for execution, the human body in the video is distinguished from other animals or articles, when one or more human bodies appear in the video, different human bodies are accurately recognized in real time, and identification and tracking are carried out.
Specifically, the mark may be a frame of a human body using a mark frame, and the mark frame may be a rectangular frame. Therefore, the user can quickly see the human body from the video conveniently.
In some embodiments, the human bodies identified from the video are each generated with a corresponding ID number. Therefore, the gait characteristics of different human bodies are marked and distinguished, and subsequent storage, retrieval and identification work is facilitated.
B. And identifying and tracking main joints of the human body, and storing the position information of the main joints in video frames to form a video sequence.
This step can be performed using a trained human joint tracking module. When a human body appears in the video, the human body joint tracking module identifies the main joints of the human body in real time and stores the positions of the main joints in the video frames, and the human body joint tracking module identifies and stores the positions of the main joints in each frame of the video frames, so that the real-time tracking of the main joints of the human body is realized, and the recorded and stored position information of the main joints forms the important characteristics of the human body posture. Specifically, the position information of the main joint may be coordinate information of the main joint in the video frame.
When the human body moves, the movement of the human body joints is represented as that the positions of all joint points change along with a time axis, the gait characteristics of the human body are represented as the movement characteristics of the human body joints, the movement characteristics of the whole body (the gait characteristics of the human body of the embodiment) are obtained by tracking the movement characteristics of the middle point (the main human body joints) of the whole body (the human body of the embodiment), the identifiability is improved for the extraction of the subsequent gait characteristics, the omission of the most valuable characteristics which can represent the gait of the human body most accurately is avoided.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait features of the human body are expanded to the movement features of the joints of the whole body of the human body, the feature space of the gait features is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key features for identification are effectively increased, the identification degree of the gait features is further increased, and the error rate of the gait feature identification can be obviously reduced.
C. Learning and extracting gait features in the video sequence; the gait features include position information of the primary joint in a video frame and motion information of the primary joint in a video sequence. Specifically, the position information may be coordinate information of a main joint in a video frame, and the motion information may be vector information of the main joint in two consecutive video frames.
In particular, this step may be performed using a trained self-coding learning module. The self-coding learning module extracts the gait features from the video sequence according to the motion data of the main joints, codes the extracted gait features, and converts the high-dimensional feature space of the gait features into low-dimensional feature vectors, so that the calculation burden of retrieving the gait features is effectively reduced.
D. And comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
This step may be performed using a trained gait feature recognition module. If the gait characteristics of the same human body are identified in the storage module, the result display shows that the identification is successful, and the ID number of the corresponding human body is displayed; if the gait features of the same human body are not identified in the database, the gait features of the human body are proved not to be recorded in the case, and the result shows that no match exists.
The human body is shot in a video in multiple angles, so that the gait features of the same human body have the problem of multiple angles, namely, the coding of the gait features is deviated due to different shooting angles, so that the subsequent gait feature identification fails, and the problem is also an important factor influencing the gait feature identification precision.
In some embodiments, the step d compares the gait characteristics with the gait characteristics preset in the storage module, and the identification result is:
calculating the distance of the gait feature in a feature space relative to the gait feature preset in the storage module through a measurement function, wherein if the distance is in an interval [ a, 1], the gait feature is the gait feature of different angles of the human body to which the gait feature preset in the storage module belongs, and if the distance is in an interval [0, a ], the gait feature is not matched with the gait feature preset in the storage module; the gait features of the same human body preset in the storage module are in multivariate Gaussian distribution in the feature space, the measurement function is a probability density function corresponding to the multivariate Gaussian distribution, and the specific calculation method can be a calculation method of the probability density function in the prior art. Specifically, the critical value a may be adjusted in practical application, and the user may make the adjustment according to the error condition occurring in the application. Therefore, the accuracy and the uniqueness of gait feature identification are ensured.
Specifically, the method for establishing the storage module includes the steps of:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence;
C. learning and extracting gait features in the video sequence; and the following steps:
and E, storing the gait characteristics.
The gait feature extraction depends on the acquired video sequence, and the gait feature extraction is carried out according to the motion data of the main joints in the video sequence. If the human body only moves for a very short time in the video acquisition range, that is, the gait duration of the human body in the video is very short, the number of the obtained video sequence frames is too small, and the gait feature is difficult to extract. For example, a human body only passes through the edge of the video acquisition range without completely passing through the video acquisition range, the gait time of the human body in the video is less than 0.5s, for example, only 0.1s, and the obtained video sequence only has 2-3 frames and cannot show a gait cycle, so that the gait feature extraction is not enough.
As shown in fig. 2, in some embodiments, after step B, the following steps are further included:
bn. interpolation generates gait video frames with a frame number greater than or equal to 20 in the acquired video.
Before the step Bn is executed, the following steps are also executed:
B1. judging the gait time of the human body in the acquired video, and executing the step Bn if the gait time is less than 0.5 s.
And the step Bn is necessary supplement and reinforcement of the step B, and makes up for the technical defect that the frame number of the video sequence is less, so that the subsequent gait feature is difficult to extract.
Specifically, the above-mentioned steps Bn and B1 may be performed using a trained gait video frame generation module, the operation of which is based on artificial intelligence techniques. The gait video frame generating module identifies the curve trend of the movement of the gait video frame according to the incomplete gait cycle obtained from the video, and calculates the incomplete part of the movement curve by utilizing nonlinear regression to generate the gait video frame.
Specifically, before the human body in the video is identified, step a. further includes step A0. to obtain the video. In this step, the video may be captured in real time by using a camera, or the video may be imported by an electronic device or an external storage.
Specifically, the motion human body recognition module, the human body joint tracking module, the self-coding learning module, the gait feature recognition module and the gait video frame generation module are obtained through artificial intelligence technology training and learning. More specifically, the motion human body recognition module and the human body joint tracking module perform supervised learning training through the gait video frame generation module, and the self-coding learning module and the gait feature recognition module perform unsupervised learning training. The training of the above modules can use artificial intelligence module training techniques in the art, which are not described in detail herein.
In some embodiments, the training of the gait feature recognition module comprises: and E, extracting the gait features of the same human body from different angles from the stored gait features in the step E, and randomly extracting the gait features of any other human body for training. And D, judging that all the human gait characteristics stored in the step E are used and the error function is reduced to a reasonable threshold value at the end of the training process. Therefore, in the training process, manual marking and checking of a large amount of gait video data are not needed, and an accurate gait feature recognition module can be obtained on the premise of greatly saving manpower and material resource investment in the training process.
The gait recognition method based on artificial intelligence provided by the invention converts a human body into a virtual human body formed by the mutual connection of joint points, characterizes the gait characteristics of the human body by the motion data of the joints of the human body, improves the identifiability for the extraction of the subsequent gait characteristics, and can accurately acquire the most valuable characteristics which can most accurately characterize the gait of the human body so as to extract the optimal gait characteristics.
As shown in fig. 3, the gait recognition device based on artificial intelligence of the present invention includes a moving body recognition module 102, a body joint tracking module 103, a self-coding learning module 104, a gait feature recognition module 106 and a storage module 105.
The moving body recognition module 102 is used for recognizing a body in the video. Specifically, the moving human body recognition module 102 distinguishes human bodies in the video from other animals or objects, and when one or more human bodies appear in the video, different human bodies are accurately recognized in real time and identified and tracked. The above-mentioned sign can use the sign frame to frame out the human body, and this sign frame can use the rectangle frame. Therefore, the user can quickly see the human body from the video conveniently.
In some embodiments, the human bodies identified from the video are each generated with a corresponding ID number. Therefore, the gait characteristics of different human bodies are marked and distinguished, and subsequent storage, retrieval and identification work is facilitated.
The human joint tracking module 103 is connected to the moving human recognition module 102, and is configured to recognize and track a main joint of the human body, and store position information of the main joint in a video frame to form a video sequence. Specifically, when a human body appears in the video, the human body joint tracking module 103 identifies the main joints of the human body in real time and stores the positions of the main joints in the video frames, and the human body joint tracking module 103 identifies and stores the positions of the main joints in each frame of the video frames, so that the real-time tracking of the main joints of the human body is realized, and the recording of the stored position information of the main joints constitutes an important characteristic of the posture of the human body. Specifically, the position information of the main joint may be coordinate information of the main joint in the video frame.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait features of the human body are expanded to the movement features of the joints of the whole body of the human body, the feature space of the gait features is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key features for identification are effectively increased, the identification degree of the gait features is further increased, and the error rate of the gait feature identification can be obviously reduced.
The self-coding learning module 104 is connected with the human joint tracking module 103 and is used for learning and extracting gait features in the video sequence; the gait features include position information of the primary joint in a video frame and motion information of the primary joint in a video sequence. Specifically, the position information may be coordinate information of a main joint in a video frame, and the motion information may be vector information of the main joint in two consecutive video frames.
The gait feature recognition module 106 is connected to the self-coding learning module 104, and is configured to compare the gait features with gait features preset in the storage module 105 to obtain a recognition result.
In some embodiments, the gait feature recognition module 106 includes a metric function, and calculates a distance of the gait feature in the feature space relative to the gait features preset in the storage module, wherein the gait features are gait features of different angles of the human body to which the gait features preset in the storage module belong if the distance is in the interval [ a, 1], and the gait features do not match the gait features preset in the storage module if the distance is in the interval [0, a); the gait features of the same human body preset in the storage module are in multivariate Gaussian distribution in the feature space, the measurement function is a probability density function corresponding to the multivariate Gaussian distribution, and the specific calculation method can be a calculation method of the probability density function in the prior art. Specifically, the critical value a may be adjusted in practical application, and the user may make the adjustment according to the error condition occurring in the application. Therefore, the accuracy and the uniqueness of gait feature identification are ensured.
The storage module 105 is connected to the gait feature recognition module 106 and is configured to store preset gait features; specifically, the storage module may utilize a database technology, and the database may identify the human body in the video through the above step a; B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence; C. learning and extracting gait features in the video sequence; and E, storing gait characteristics and establishing and obtaining. The optimal gait features are accurately retrieved and extracted from the high-dimensional feature space through the self-coding learning module, and the high-dimensional time and space features are converted into low-dimensional feature vectors, so that the gait features are effectively stored.
As shown in fig. 4, in some embodiments, the gait recognition device based on artificial intelligence further includes a gait video frame generation module 107, where the gait video frame generation module 107 is configured to interpolate a gait video frame with a frame number greater than or equal to 20 in the video when the human body in the video has a gait time less than 0.5 s. The gait video frame generation module 107 is a necessary supplement and reinforcement of the human body joint tracking module 103, and makes up for the technical defect that the frame number of the video sequence is small, which causes difficulty in extracting the subsequent gait features.
Specifically, the gait video frame generating module identifies the curve trend of the movement of the gait video frame according to the incomplete gait cycle obtained from the video, and calculates the incomplete part of the movement curve by utilizing nonlinear regression to generate the gait video frame.
In some embodiments, the artificial intelligence based gait recognition device further comprises a video acquisition module 101, wherein the video acquisition module 101 is configured to acquire a video; specifically, the video acquisition 101 may employ a plurality of cameras, and the cameras are used for shooting conditions of a use site and generating videos, for example, when the door access identification is applied, the cameras may be used according to specific conditions to shoot a human body of the use site at multiple angles; the video acquisition 101 may employ a data access port, and acquire a video, such as a certificate in a public security office, from an electronic device or an external storage through the data access port.
In some embodiments, the artificial intelligence based gait recognition device further comprises a result output module 108, and the result output module 108 is configured to output the recognition result to inform the user. Specifically, the result output module 108 may adopt a computer, a mobile phone or a player, and inform the user of the recognition result in a form of text display or voice broadcast.
In some embodiments, the artificial intelligence based gait recognition device further comprises a retrieval module 109, the retrieval module 109 is connected to the storage module 105 and is used for retrieving gait features during training and deployment of working modules, and the working modules comprise a moving body recognition module 102, a body joint tracking module 103, a self-coding learning module 104, a gait feature recognition module 106 and a gait video frame generation module 107. Therefore, the gait feature retrieval speed is improved through the retrieval module.
As shown in fig. 5, in some embodiments, the functions of the above modules can be realized by providing a multifunctional database 100 connected to the gait feature recognition module 106 and the self-coding learning module 104 by using database technology. For example, the multi-functional database 100 may include the storage module 105 and the retrieval module 109, which implement a function of storing gait characteristics and a function of retrieving the stored gait characteristics.
During specific work, the video acquisition module 101 acquires a video and sends the video to the moving human body recognition module 102; the moving human body recognition module 102 recognizes a human body from the video, and meanwhile, the human body joint tracking module 103 recognizes and tracks the position of a main joint of the human body, stores the position of the main joint in a video frame to form a video sequence, and sends the video sequence to the self-coding learning module 104 to extract gait characteristics; the gait features are extracted and then sent to a gait feature recognition module 106 for recognition, the gait feature recognition module 106 performs recognition from the storage module 105, searches whether the gait features of the same human body are matched with the gait features to be recognized or not, and sends recognition results to a result output module 108. In some embodiments, the extracted gait features may also be sent to the storage module 105 for storage.
The gait recognition device based on artificial intelligence accurately recognizes the human body and the main joints thereof through the moving human body recognition module and the human body joint tracking module, and accurately tracks to obtain the motion of the main joints; the self-coding learning module replaces manual design and gait feature extraction, so that the optimal gait feature can be accurately retrieved and extracted from a high-dimensional feature space, the complete and robust extraction of the gait feature is realized, and meanwhile, high-dimensional time and space features are converted into low-dimensional feature vectors, so that the calculation burden during gait feature retrieval is greatly reduced; and finally, the gait feature recognition module is used for realizing recognition, so that the recognition precision is improved. The motion human body recognition module, the human body joint tracking module, the self-coding learning module, the gait feature recognition module and the gait video frame generation module are obtained through artificial intelligence technology training and learning.
When the method is specifically applied, the method can be applied to an entrance guard identification scene of a community, a camera is arranged at the door of the community to acquire the video of a visitor, a player can be arranged to broadcast an identification result, and gait characteristics of the person allowed to enter the community are stored in a storage module in advance; when a person walks to the gate of a cell, a camera at the gate acquires a video, the video identifies the human body and extracts gait features thereof through a moving human body identification module, a human body joint tracking module and a self-coding learning module, and then the gait features of the person are identified through a gait feature identification module; if the matched gait features are not identified in the storage module, the identification fails, the player broadcasts the prompt of 'identification failure', the cell door is still closed, the personnel cannot directly enter the cell, the personnel can be registered by a cell security guard and then released, or can enter the cell through other manual verification modes, and the safety of the cell is improved.
As shown in fig. 6, the gait recognition system based on artificial intelligence provided by the invention includes a video acquisition terminal 201, a user terminal 203 and a server 202.
The video acquiring terminal 201 is configured to acquire a video; specifically, the video acquiring terminal 201 may be a camera, a data access port, or the like.
The server 202 is configured to extract and identify the gait features in the video, and compare the gait features with gait features preset in the storage module to obtain an identification result.
Specifically, the step of extracting and identifying the gait features in the video and comparing the gait features with the gait features preset in the storage module to obtain the identification result includes the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait features comprise position information of the primary joint in the video frame and motion information of the primary joint in the video sequence;
D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
Specifically, the position information may be coordinate information of a main joint in a video frame, and the motion information may be vector information of the main joint in two consecutive video frames.
In some embodiments, the primary joints include ankle, hip, knee, wrist, elbow, and shoulder joints. Therefore, the gait features of the human body are expanded to the movement features of the joints of the whole body of the human body, the feature space of the gait features is increased, meanwhile, the human body joints outside the joints of the lower limbs are identified and tracked, the key features for identification are effectively increased, the identification degree of the gait features is further increased, and the error rate of the gait feature identification can be obviously reduced.
In some embodiments, the extracting and identifying the gait features in the video and comparing the gait features with the gait features preset in the storage module to obtain the identification result further includes the following steps:
bn. interpolation generates gait video frames with a frame number greater than or equal to 20 in the acquired video.
Before this step Bn, the following steps are also performed:
B1. judging the gait time of the human body in the acquired video, and executing the step Bn if the gait time is less than 0.5 s.
And the step Bn is necessary supplement and reinforcement of the step B, and makes up for the technical defect that the frame number of the video sequence is less, so that the subsequent gait feature is difficult to extract.
Specifically, the step Bn and the step B1 are executed after the step B, and may be executed by using a trained gait video frame generation module. The gait video frame generating module identifies the curve trend of the movement of the gait video frame according to the incomplete gait cycle obtained from the video, and calculates the incomplete part of the movement curve by utilizing nonlinear regression to generate the gait video frame.
In some embodiments, the step d compares the gait characteristics with the gait characteristics preset in the storage module, and the identification result is:
calculating the distance of the gait feature in a feature space relative to the gait feature preset in the storage module through a measurement function, wherein if the distance is in an interval [ a, 1], the gait feature is the gait feature of different angles of the human body to which the gait feature preset in the storage module belongs, and if the distance is in an interval [0, a ], the gait feature is not matched with the gait feature preset in the storage module; the gait features of the same human body preset in the storage module are in multivariate Gaussian distribution in the feature space, the measurement function is a probability density function corresponding to the multivariate Gaussian distribution, and the specific calculation method can be a calculation method of the probability density function in the prior art. Specifically, the critical value a may be adjusted in practical application, and the user may make the adjustment according to the error condition occurring in the application. Therefore, the accuracy and the uniqueness of gait feature identification are ensured.
Specifically, the storage module may be established by referring to the method for establishing the storage module in the gait feature recognition method.
The user terminal 203 is used for monitoring the gait recognition process and outputting the recognition result. The monitoring application range includes but is not limited to viewing the acquired video, the condition of the human body identified in the video, the identification result of the gait characteristics and the like. Specifically, the user terminal 203 may be a computer client or a mobile phone APP client.
When the gait feature recognition system is specifically applied to the legal department of public security, the video acquisition terminal is a data access port of a computer, the terminal is used as a computer client of a public security office, the gait features of the suspect are stored in the storage module in advance, the server extracts and recognizes the gait features of the video imported through the data access port of the computer, the gait features are compared with the gait features of the suspect preset in the storage module, the public security personnel check the recognition result from the computer client, if the recognition is successful, the suspect can be locked, and if the recognition is not matched, the suspect is not a target person.
The present invention provides a storage medium in which a computer program is stored, which, when run on a computer, causes the computer to execute the above-described artificial intelligence-based gait recognition method. In particular, the storage medium may include, but is not limited to: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, or the like.
The server provided by the invention comprises a processor and a memory, wherein the memory is stored with a computer program, and the processor is used for executing the gait recognition method based on artificial intelligence by calling the computer program stored in the memory.
In the description herein, references to the description of the terms "one embodiment," "certain embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
What has been described above are merely some embodiments of the present invention. It will be apparent to those skilled in the art that various changes and modifications can be made without departing from the inventive concept thereof, and these changes and modifications can be made without departing from the spirit and scope of the invention.
Claims (10)
1. The gait recognition method based on artificial intelligence is characterized by comprising the following steps:
A. identifying a human body in the video;
B. identifying and tracking main joints of the human body, and storing position information of the main joints in video frames to form a video sequence;
C. learning and extracting gait features in the video sequence;
the gait features comprise position information of the primary joint in the video frame and motion information of the primary joint in the video sequence;
D. and comparing the gait characteristics with gait characteristics preset in a storage module to obtain an identification result.
2. The gait recognition method based on artificial intelligence according to claim 1, wherein the step d. comparing the gait characteristics with the gait characteristics preset in the storage module, the recognition result is:
calculating the distance of the gait feature in a feature space relative to the gait feature preset in the storage module through a measurement function, wherein if the distance is in an interval [ a, 1], the gait feature is the gait feature of different angles of the human body to which the gait feature preset in the storage module belongs, and if the distance is in an interval [0, a ], the gait feature is not matched with the gait feature preset in the storage module;
the gait characteristics of the same human body preset in the storage module are in multi-element Gaussian distribution in the characteristic space.
3. The artificial intelligence based gait recognition method according to claim 1, characterized by further comprising a step Bn. of interpolating gait video frames with a number of frames greater than or equal to 20 in said video after step B;
and B1, judging the gait duration of the human body in the video before executing the step Bn, and executing the step Bn if the gait duration is less than 0.5 s.
4. The artificial intelligence based gait recognition method according to claim 1, characterized in that the main joints include ankle joints, hip joints, knee joints, wrist joints, elbow joints and shoulder joints.
5. The gait recognition method based on artificial intelligence according to claim 1, wherein the human body identified from the video is generated with a corresponding ID number.
6. The gait recognition device based on artificial intelligence is characterized by comprising a moving human body recognition module, a human body joint tracking module, a self-coding learning module, a gait feature recognition module and a storage module;
the motion human body identification module is used for identifying a human body in the video;
the human body joint tracking module is connected with the moving human body identification module and used for identifying and tracking main joints of the human body and storing position information of the main joints in video frames to form a video sequence;
the self-coding learning module is connected with the human body joint tracking module and is used for learning and extracting gait features in the video sequence; the gait features comprise position information of the main joint in a video frame and motion information of the main joint in a video sequence;
the gait feature recognition module is connected with the self-coding learning module and used for comparing the gait features with gait features preset in the storage module to obtain a recognition result;
the storage module is connected with the gait feature recognition module and used for storing preset gait features.
7. The gait recognition device based on artificial intelligence according to claim 6, further comprising a gait video frame generation module, wherein the gait video frame generation module is connected to the moving human body recognition module and is configured to interpolate a gait video frame with a frame number greater than or equal to 20 in the video when the time length of the human body gait in the video is less than 0.5 s.
8. The gait recognition system based on artificial intelligence is characterized by comprising a video acquisition terminal, a use terminal and a server;
the video acquisition terminal is used for acquiring a video;
the server is used for extracting the gait features in the video and comparing the gait features with the gait features preset in the storage module to obtain an identification result;
the use terminal is used for monitoring the gait recognition process and outputting a recognition result.
9. A storage medium having stored therein a computer program which, when run on a computer, causes the computer to execute the artificial intelligence based gait recognition method according to any one of claims 1 to 5.
10. A server comprising a processor and a memory, the memory having a computer program stored therein, the processor being configured to execute the artificial intelligence based gait recognition method according to any one of claims 1 to 5 by calling the computer program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334982.3A CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334982.3A CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111476198A true CN111476198A (en) | 2020-07-31 |
| CN111476198B CN111476198B (en) | 2023-09-26 |
Family
ID=71755578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010334982.3A Active CN111476198B (en) | 2020-04-24 | 2020-04-24 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111476198B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112022166A (en) * | 2020-08-08 | 2020-12-04 | 司法鉴定科学研究院 | Human body identity recognition method and system based on medical movement disorder feature recognition |
| CN111476198B (en) * | 2020-04-24 | 2023-09-26 | 广西安良科技有限公司 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
| CN118870306A (en) * | 2024-08-26 | 2024-10-29 | 天津大学合肥创新发展研究院 | Indoor simultaneous passive tracking and gait recognition method and system based on Wi-Fi |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060204049A1 (en) * | 2005-01-14 | 2006-09-14 | Schneider John K | Multimodal fusion decision logic system |
| US20080231745A1 (en) * | 2007-03-19 | 2008-09-25 | Masahiro Ogino | Video Processing Apparatus and Video Display Apparatus |
| CN101558996A (en) * | 2009-05-15 | 2009-10-21 | 天津大学 | Gait recognition method based on orthogonal projection three-dimensional reconstruction of human motion structure |
| CN103377366A (en) * | 2012-04-26 | 2013-10-30 | 哈尔滨工业大学深圳研究生院 | Gait recognition method and system |
| CN104463099A (en) * | 2014-11-05 | 2015-03-25 | 哈尔滨工程大学 | Multi-angle gait recognizing method based on semi-supervised coupling measurement of picture |
| KR101732402B1 (en) * | 2015-12-29 | 2017-05-08 | 건국대학교 산학협력단 | Gait monitoring apparatus by video observation and method of thereof |
| WO2017215618A1 (en) * | 2016-06-14 | 2017-12-21 | 中兴通讯股份有限公司 | Gait recognition method and apparatus |
| US20180007409A1 (en) * | 2015-07-06 | 2018-01-04 | Tencent Technology (Shenzhen) Company Limited | Video recommending method, server, and storage media |
| US10154228B1 (en) * | 2015-12-23 | 2018-12-11 | Amazon Technologies, Inc. | Smoothing video panning |
| WO2020249025A1 (en) * | 2019-06-12 | 2020-12-17 | 杭州海康威视数字技术股份有限公司 | Identity information determining method and apparatus, and storage medium |
| CN114821786A (en) * | 2022-04-27 | 2022-07-29 | 南京邮电大学 | A Gait Recognition Method Based on Fusion of Human Profile and Key Point Features |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476198B (en) * | 2020-04-24 | 2023-09-26 | 广西安良科技有限公司 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
-
2020
- 2020-04-24 CN CN202010334982.3A patent/CN111476198B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060204049A1 (en) * | 2005-01-14 | 2006-09-14 | Schneider John K | Multimodal fusion decision logic system |
| US20080231745A1 (en) * | 2007-03-19 | 2008-09-25 | Masahiro Ogino | Video Processing Apparatus and Video Display Apparatus |
| CN101558996A (en) * | 2009-05-15 | 2009-10-21 | 天津大学 | Gait recognition method based on orthogonal projection three-dimensional reconstruction of human motion structure |
| CN103377366A (en) * | 2012-04-26 | 2013-10-30 | 哈尔滨工业大学深圳研究生院 | Gait recognition method and system |
| CN104463099A (en) * | 2014-11-05 | 2015-03-25 | 哈尔滨工程大学 | Multi-angle gait recognizing method based on semi-supervised coupling measurement of picture |
| US20180007409A1 (en) * | 2015-07-06 | 2018-01-04 | Tencent Technology (Shenzhen) Company Limited | Video recommending method, server, and storage media |
| US10154228B1 (en) * | 2015-12-23 | 2018-12-11 | Amazon Technologies, Inc. | Smoothing video panning |
| KR101732402B1 (en) * | 2015-12-29 | 2017-05-08 | 건국대학교 산학협력단 | Gait monitoring apparatus by video observation and method of thereof |
| WO2017215618A1 (en) * | 2016-06-14 | 2017-12-21 | 中兴通讯股份有限公司 | Gait recognition method and apparatus |
| WO2020249025A1 (en) * | 2019-06-12 | 2020-12-17 | 杭州海康威视数字技术股份有限公司 | Identity information determining method and apparatus, and storage medium |
| CN114821786A (en) * | 2022-04-27 | 2022-07-29 | 南京邮电大学 | A Gait Recognition Method Based on Fusion of Human Profile and Key Point Features |
Non-Patent Citations (2)
| Title |
|---|
| 王健等: "步态序列图像中人体下肢关节点的定位", 《计算机仿真》 * |
| 王健等: "步态序列图像中人体下肢关节点的定位", 《计算机仿真》, no. 02, 15 February 2011 (2011-02-15) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476198B (en) * | 2020-04-24 | 2023-09-26 | 广西安良科技有限公司 | Gait recognition method, device, system, storage medium and server based on artificial intelligence |
| CN112022166A (en) * | 2020-08-08 | 2020-12-04 | 司法鉴定科学研究院 | Human body identity recognition method and system based on medical movement disorder feature recognition |
| CN118870306A (en) * | 2024-08-26 | 2024-10-29 | 天津大学合肥创新发展研究院 | Indoor simultaneous passive tracking and gait recognition method and system based on Wi-Fi |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111476198B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12175684B2 (en) | Pedestrian tracking method, computing device, pedestrian tracking system and storage medium | |
| Kale et al. | Gait-based recognition of humans using continuous HMMs | |
| CN110070010B (en) | Face attribute association method based on pedestrian re-recognition | |
| CN102426645B (en) | Multi-view and multi-state gait recognition method | |
| Wang et al. | Silhouette analysis-based gait recognition for human identification | |
| CN104573652B (en) | Determine the method, apparatus and terminal of the identity of face in facial image | |
| US8135220B2 (en) | Face recognition system and method based on adaptive learning | |
| CN101095149B (en) | Image comparison device and image comparison method | |
| WO2020042419A1 (en) | Gait-based identity recognition method and apparatus, and electronic device | |
| Cherla et al. | Towards fast, view-invariant human action recognition | |
| CN107798308B (en) | Face recognition method based on short video training method | |
| CN112668493B (en) | GAN and deep learning based re-identification and location tracking system for dressed pedestrians | |
| JP2006133946A (en) | Moving object recognition device | |
| CN111476198B (en) | Gait recognition method, device, system, storage medium and server based on artificial intelligence | |
| JP2004192637A (en) | Face detection | |
| CN105426827A (en) | Living body verification method, device and system | |
| CN108537181A (en) | A kind of gait recognition method based on the study of big spacing depth measure | |
| CN114663835B (en) | Pedestrian tracking method, system, equipment and storage medium | |
| CN107766819A (en) | A kind of video monitoring system and its real-time gait recognition methods | |
| CN112800950A (en) | Large security activity face searching method based on deep learning | |
| CN110599129A (en) | Campus attendance checking method, device, identification terminal and system based on image tracking | |
| WO2022134916A1 (en) | Identity feature generation method and device, and storage medium | |
| CN117975533A (en) | Video identity recognition method and system based on face contour | |
| Bouchrika | Parametric elliptic fourier descriptors for automated extraction of gait features for people identification | |
| CN119068559B (en) | Gait recognition method and system based on local and global difference modeling network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |