[go: up one dir, main page]

CN113589920B - Gesture recognition method, man-machine interaction method, device, equipment and storage medium - Google Patents

Gesture recognition method, man-machine interaction method, device, equipment and storage medium Download PDF

Info

Publication number
CN113589920B
CN113589920B CN202010367114.5A CN202010367114A CN113589920B CN 113589920 B CN113589920 B CN 113589920B CN 202010367114 A CN202010367114 A CN 202010367114A CN 113589920 B CN113589920 B CN 113589920B
Authority
CN
China
Prior art keywords
surface electromyographic
signal
signals
electromyographic signals
gesture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010367114.5A
Other languages
Chinese (zh)
Other versions
CN113589920A (en
Inventor
田彦秀
韩久琦
姚秀军
桂晨光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to CN202010367114.5A priority Critical patent/CN113589920B/en
Priority to PCT/CN2021/090715 priority patent/WO2021219039A1/en
Publication of CN113589920A publication Critical patent/CN113589920A/en
Application granted granted Critical
Publication of CN113589920B publication Critical patent/CN113589920B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Dermatology (AREA)
  • User Interface Of Digital Computer (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The application relates to a gesture recognition method, a human-computer interaction method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a surface electromyographic signal; acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period; extracting signal characteristics of the effective surface electromyographic signals; and according to the signal characteristics, obtaining gestures corresponding to the surface electromyographic signals. The method and the device are used for simplifying the gesture recognition process and improving the gesture recognition efficiency.

Description

Gesture recognition method, man-machine interaction method, device, equipment and storage medium
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a gesture recognition method, a human-computer interaction method, a device, equipment, and a storage medium.
Background
Gesture recognition is essentially a pattern recognition problem, requiring learning valid features from the input information and recognizing gesture tags using the extracted features. Gesture motion recognition technology based on surface electromyographic signals can be divided into two types according to whether human factors are added: the method comprises the steps of preprocessing collected surface electromyographic signals of different gesture actions, extracting various characteristic quantities of a time domain, a frequency domain and a time-frequency domain, then carrying out characteristic dimension reduction or characteristic selection on the extracted characteristic quantities, inputting the characteristic quantities into a classifier for model training, and using the trained classifier for gesture action real-time prediction; the other type is that the preprocessed surface electromyographic signals are directly used as input quantity, human factors such as feature extraction and the like are not added, the difference of the surface electromyographic signals of different gesture actions is actively grasped and learned by utilizing a deep learning frame, and a trained network model structure is used in actual test.
The first myoelectric gesture recognition method generally uses a signal analysis technology to manually extract various signal features from the surface myoelectric signals, and then inputs the extracted signal features into classifiers such as linear discriminant analysis, a support vector machine, a hidden Markov model and the like for gesture recognition. The deep neural network model in the second type of recognition technology is widely applied to convolutional neural networks.
However, whether the signals selected by the first myoelectric gesture recognition technology are good or bad can have a great influence on gesture recognition performance. The second myoelectric gesture recognition technology does not rely on manual feature extraction and complex and cumbersome feature optimization processes, but needs to automatically learn representative depth features from a large number of input samples, so that the calculation amount is very large. In addition, according to the two gesture recognition technologies, a user is required to acquire and record a section of myoelectric signal data of different gesture surfaces with marks in advance, the mark data can be used for prediction in practice after model training, the implementation process is inconvenient, and the user experience is poor.
Disclosure of Invention
The application provides a gesture recognition method, a human-computer interaction method, a device, equipment and a storage medium, which are used for simplifying a gesture recognition process and improving gesture recognition efficiency.
In a first aspect, the present application provides a gesture recognition method, including:
Acquiring a surface electromyographic signal;
acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
Extracting signal characteristics of the effective surface electromyographic signals;
and according to the signal characteristics, obtaining gestures corresponding to the surface electromyographic signals.
Optionally, the surface electromyographic signals are generated by forearm muscle group movements, and the surface electromyographic signals corresponding to different gestures are different.
Optionally, before obtaining the effective surface electromyographic signal from the surface electromyographic signal, the method further includes:
and filtering noise with preset frequency in the surface electromyographic signals.
Optionally, extracting the signal characteristics of the effective surface electromyographic signal includes:
Intercepting the surface electromyographic signals by adopting M data windows to obtain M sections of surface electromyographic signals, wherein the interval between two adjacent sections of surface electromyographic signals is a preset step, one data window comprises n data points, and the preset step comprises M data points;
The following feature extraction operation is carried out on each section of the surface electromyographic signals respectively: calculating the absolute value of the difference value of each pair of adjacent data points in the surface electromyographic signals of the p-th section, and carrying out summation processing on all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M;
and summing the summation results of the obtained M segments of the surface electromyographic signals, and taking the obtained results as signal characteristics of the effective surface electromyographic signals.
Optionally, according to the signal characteristics, obtaining the gesture corresponding to the surface electromyographic signal includes:
Determining a forearm muscle group corresponding to the signal characteristic;
And obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the pre-configured corresponding relation between the forearm muscle group and the gesture.
Optionally, obtaining an effective surface electromyographic signal from the surface electromyographic signal includes:
Correcting the surface electromyographic signals to obtain corrected signals;
Performing integral operation on the obtained correction signal to obtain an envelope signal;
and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as the effective surface electromyographic signal.
In a second aspect, the present application provides a human-computer interaction method, including: the gesture recognition method of the first aspect.
In a third aspect, the present application provides a gesture recognition apparatus, comprising:
The first acquisition module is used for acquiring the surface electromyographic signals;
The second acquisition module is used for acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
the extraction module is used for extracting signal characteristics of the effective surface electromyographic signals;
and the processing module is used for obtaining the gesture corresponding to the surface electromyographic signal according to the signal characteristics.
In a fourth aspect, the present application provides an electronic device, comprising: the gesture recognition system comprises a processor and a memory, wherein the processor is used for executing a program stored in the memory so as to realize the gesture recognition method.
Optionally, the electronic device is an armband device for wearing on the forearm near one third of the elbow joint.
In a fifth aspect, the present application provides a computer readable storage medium storing a computer program which when executed by a processor implements the gesture recognition method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the surface electromyographic signals are obtained, and the effective surface electromyographic signals are obtained from the surface electromyographic signals, so that the interference of the ineffective surface electromyographic signals is avoided. Further, signal characteristics of the effective surface electromyographic signals are extracted, and gestures corresponding to the surface electromyographic signals are directly acquired according to the signal characteristics. According to the method, a large number of gesture samples are not required to be collected in advance for training, gestures corresponding to the surface electromyographic signals are obtained according to training results, signal characteristics of the effective surface electromyographic signals can be directly extracted, the gestures corresponding to the surface electromyographic signals are rapidly and effectively obtained according to the signal characteristics, the operation process is simple and convenient, and the efficiency is high.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a gesture recognition method according to an embodiment of the present application;
Fig. 2 is a schematic flow chart of an effective surface electromyographic signal extraction method according to an embodiment of the present application;
Fig. 3 is a schematic flow chart of a method for extracting characteristics of surface electromyographic signals according to an embodiment of the application;
FIG. 4 is a schematic flow chart of a specific implementation process of a gesture recognition method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a gesture recognition apparatus according to an embodiment of the present application;
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application provides a gesture recognition method which can be applied to independent electronic equipment, wherein the electronic equipment collects surface electromyographic signals and recognizes gestures corresponding to the surface electromyographic signals. In addition, the method can be applied to intelligent terminal equipment, the surface electromyographic signals are collected through the collecting equipment, then the collecting equipment transmits the collected surface electromyographic signals to the intelligent terminal equipment, and the gesture corresponding to the surface electromyographic signals is recognized and obtained by the intelligent terminal equipment. Of course, the method can also be applied to a server, the surface electromyographic signals are collected through the collecting device, then the collecting device transmits the collected surface electromyographic signals to the server, and the gesture corresponding to the surface electromyographic signals is recognized and obtained by the server. The specific implementation process of the method is shown in fig. 1:
And step 101, acquiring a surface electromyographic signal.
Specifically, the surface electromyographic signal is a bioelectric signal collected by an electrode placed on the surface of the skin, is a non-stable and nonlinear weak electric signal, has randomness, is extremely easy to be interfered by the outside, and has low signal to noise ratio. In addition, the surface electromyographic signals can reflect the extension and flexion conditions of the joints of the human body and the shape and position of limbs.
In the embodiment of the application, the electrode for collecting the surface electromyographic signals is worn on the forearm of the upper limb of the human body so as to monitor the movement condition of the forearm muscle driven by gesture actions.
The surface electromyographic signals monitored by the electrodes worn on the forearm are generated by the movement of the forearm muscle groups, and different gestures drive different forearm muscle groups to move, that is, each gesture is generated by a part of the forearm muscle groups in the whole forearm muscle groups to play a leading role, and different forearm muscle groups move to generate different surface electromyographic signals.
For example, assuming that the OK gesture is generated by the forearm muscle a movement, the fist-making gesture is generated by the forearm muscle B movement, the scissor hand gesture is generated by the forearm muscle C movement, and the metal gift gesture is generated by the forearm muscle D and the forearm muscle E movement together. Therefore, when the detected object makes an OK gesture, the surface electromyographic signal A is monitored; when the detected object makes a fist making gesture, monitoring a surface electromyographic signal B; when the detected object makes a gesture of a scissor hand, monitoring a surface electromyographic signal C; when the detected object makes a gesture of a metal gift, the surface electromyographic signal DE is detected.
And 102, acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period.
In a specific embodiment, after the surface electromyographic signal is obtained, a preprocessing operation is required to be performed on the surface electromyographic signal before the effective surface electromyographic signal is obtained from the surface electromyographic signal, so as to filter interference in the surface electromyographic signal, and improve the accuracy of the hand types of subsequent gestures.
Specifically, noise with preset frequency in the surface electromyographic signals is filtered in the pretreatment process.
For example, since the surface electromyographic signal is easily interfered by the power frequency of 50Hz, noise having a frequency of 50Hz (Hz) in the surface electromyographic signal is removed by using a trap. And then, inputting the surface electromyographic signals obtained after the wave trap is filtered into a 6-order Butterworth filter to obtain the surface electromyographic signals with the frequency of 20-200 Hz after the filter is filtered. Among them, the surface electromyographic signal with a frequency of 20 to 200Hz is a surface electromyographic signal useful in the gesture recognition method. By preprocessing the surface electromyographic signals, the noise content of the signals is reduced, and the surface electromyographic signals which are more in line with the gesture recognition method are obtained.
In one embodiment, the electromyographic signal is a superposition of motor action potentials in time and space among a plurality of muscle fibers, and the surface electromyographic signal is the combined effect of electrical activity on the superficial muscles and nerve trunk on the skin surface. The surface electromyographic signals are divided into two types, one is the surface electromyographic signal corresponding to the resting potential, and the other is the surface electromyographic signal corresponding to the action potential. The effective surface electromyographic signals are surface electromyographic signals in a muscle activity time period, namely the effective surface electromyographic signals are surface electromyographic signals corresponding to action potentials.
In addition, the surface electromyographic signals corresponding to the resting potential are not useful for gesture recognition of the method, and the resting potential refers to that the muscle is in a relaxed state, and no action is executed. And, for the present method, unwanted signals belonging to the noise class. Thus, it is necessary to acquire an active segment of the surface electromyographic signal, i.e. to detect the starting and ending positions of the action potential of the surface electromyographic signal, in order to obtain an effective surface electromyographic signal.
Specifically, as shown in fig. 2, the specific process of detecting the start position and the end position of the action potential of the surface electromyographic signal to obtain the effective surface electromyographic signal is as follows:
In step 201, a correction process is performed on the surface electromyographic signal to obtain a correction signal.
In one embodiment, the correction of the surface electromyographic signal is performed on a pre-processed surface electromyographic signal.
First, a conversion operation is performed on the preprocessed surface electromyographic signal, and a converted signal is obtained. By performing a conversion operation on the surface muscle signals, effective corrections are made for different subjects, reducing the effects of impedance and muscle tension on the baseline threshold.
The converted signal is then corrected based on the baseline threshold value, obtaining a corrected signal. Specifically, the baseline threshold is determined as: thr=mean { MAV 1,MAV2,MAV3,...,MAVm } +a
Wherein thr is a baseline threshold, MAV i is the maximum value of the sliding window in the rest state data of the surface electromyographic signal, i is the subscript of the maximum value of the sliding window in the rest state data of the surface electromyographic signal, the value range is a positive integer between 1 and k, m is the number of the sliding windows, and A is a constant. And the surface electromyographic signals are corrected according to the baseline threshold value, so that the influence of individual differences on the surface electromyographic signals is effectively reduced.
Step 202, performing an integration operation on the obtained correction signal to obtain an envelope signal.
In one embodiment, the obtained correction signals are introduced into the kernel function one by one, and the kernel function is updated after each correction signal is introduced. The specific operation is as follows:
The kernel function is initialized.
The initialized kernel function is expressed as: kernel (j k)＝0,j1,j2,j3,...jn).
Introducing a correction signal s i into a kernel function, updating the kernel function into kernel= { j 2,...jn,si},j2,...jn =0, and calculating unit equidistant integration of the kernel function based on a trapezoidal method to obtain an envelope signal y i;
Introducing a correction signal s i+1 into a kernel function, updating the kernel function into kernel= { j 3,...jn,si,si+1},j3,j4...jn =0, and calculating unit equidistant integration of the kernel function based on a trapezoidal method to obtain an envelope signal y i+1;
And so on, the envelope signal { y 1,y2,y3,...yi, } is calculated from the correction signal { s 1,s2,s3,...si, }.
And calculating the unit equidistant integral of the kernel function based on a trapezoidal method to obtain an envelope signal corresponding to the correction signal.
The conversion signal obtained through conversion of the surface electromyographic signals is corrected by utilizing the baseline threshold value, and then the correction signal is processed by utilizing the kernel function to obtain an envelope signal, wherein the envelope signal increases the tiny difference between an action potential section and a resting potential section in the surface electromyographic signals, weakens the fluctuation of the surface electromyographic signals caused by muscle tension, and reduces the misjudgment of the active section caused by the muscle tension.
And 203, taking the surface electromyographic signals corresponding to the envelope signals with the amplitude larger than a preset threshold value as effective surface electromyographic signals.
In particular, the envelope is a plot of the amplitude of the random process over time. The envelope signal is a high frequency amplitude modulated signal whose amplitude is varied by a low frequency modulated signal. If the peak points of the high frequency amplitude modulated signal are connected, a curve corresponding to the low frequency modulated signal can be obtained. In addition, the envelope signal is also a new pulse signal (with larger period), and the pulse signal has a certain width (a period of 0 in each period) when observed in time, which is the width in time, namely the pulse envelope width. The bandwidth of the pulse is inversely proportional to the pulse width, i.e. the narrower the width in pulse time the greater the bandwidth in the frequency spectrum.
Specifically, the active segment includes a start position and an end position. If the amplitude of the previous envelope signal or envelope signals is not greater than a preset threshold value, determining the position of the surface electromyographic signal corresponding to the initial position of the envelope signal with the amplitude of the envelope signal being greater than the preset threshold value as the initial position of the active segment when the amplitude of the envelope signal is greater than the preset threshold value; and if the amplitude of the previous envelope signal or envelope signals is larger than a preset threshold value, determining the position of the surface electromyographic signal corresponding to the initial position of the envelope signal, of which the amplitude is smaller than the preset threshold value, as the end position of the active segment when the amplitude of the envelope signal is not larger than the preset threshold value. The surface electromyographic signal in the middle of the starting position to the ending position is the effective surface electromyographic signal.
For example, the preset threshold is set to zero. If the amplitude of the previous envelope signal or envelope signals is not more than zero, determining the position of the surface electromyographic signal corresponding to the initial position of the amplitude of the envelope signal which is larger than zero as the initial position of the active segment when the amplitude of the envelope signal is larger than zero; if the amplitude of the previous envelope signal or signals is/are larger than zero, when the amplitude of the envelope signal is not larger than zero, the position of the surface electromyographic signal corresponding to the initial position where the amplitude of the envelope signal is smaller than zero is determined as the end position of the active segment. The surface electromyographic signal in the middle of the starting position to the ending position is the effective surface electromyographic signal.
And 103, extracting signal characteristics of the effective surface electromyographic signals.
In a specific embodiment, the surface electromyographic signals are intercepted by adopting M data windows to obtain M sections of surface electromyographic signals, the interval between two adjacent sections of surface electromyographic signals is a preset step, and the characteristic extraction operation is performed on each section of surface electromyographic signals respectively, wherein one data window comprises n data points, and the preset step comprises M data points. As shown in fig. 3, the following description is given by taking feature extraction of the surface electromyographic signal of the p-th segment as an example:
In step 301, the absolute value of the difference between each pair of adjacent data points in the p-th segment surface electromyographic signal is calculated, wherein p is greater than or equal to 1 and less than or equal to M.
Specifically, the signal wavelength of one data window includes n data points, expressed by the formula: dv= { S 1,S2,S3,...,Si,...,Sn-1,Sn }, where S i is a data point.
Separately calculating the absolute value of the difference between each pair of adjacent data points, i.e |S2-S1|,|S3-S2|,...,|Si-Si-1|,...,|Sn-Sn-1|.
Step 302, summing the absolute values of all the calculated differences to obtain a summation result.
Specifically, the absolute values of all the calculated differences are summed to obtain the sum result:
The summation result obtained in step 302 is taken as the feature extraction result of the p-th segment surface electromyographic signal.
After the respective feature extraction results of the M-segment surface electromyographic signals are obtained, the feature extraction results (i.e., M summation results) of the obtained M-segment surface electromyographic signals are summed, and the obtained results are used as the signal features of the entire effective surface electromyographic signals.
Specifically, the feature extraction results of the M-segment surface electromyographic signals are summed, and the obtained results can be expressed as:
The signal characteristics of the surface electromyographic signals are extracted by simply superposing the signal wavelengths of the M-segment surface electromyographic signals, so that the complexity of the electromyographic signal waveforms is reflected, and the final obtained result is the combined effect of the electromyographic signal amplitude, frequency, duration and the like.
And 104, obtaining a gesture corresponding to the surface electromyographic signal according to the signal characteristics.
In one embodiment, the forearm muscle groups corresponding to each gesture are different, and the forearm muscle groups corresponding to the signal characteristics are determined according to the signal characteristics. Further, according to the determined corresponding relation between the forearm muscle groups and the pre-configured forearm muscle groups and the gestures, the gestures corresponding to the surface electromyographic signals are obtained.
For example, determining the forearm muscle group corresponding to the signal characteristic as forearm muscle B through the signal characteristic; and obtaining the gesture corresponding to the surface electromyographic signal as a fist-making gesture according to the pre-configured corresponding relation between the forearm muscle group and the gesture corresponding to the forearm muscle B as the fist-making gesture.
Specifically, after the gesture corresponding to the surface electromyographic signal is obtained according to the signal characteristics, the corresponding relation is stored as a matching template, and the matching template is used as a standard template for different gesture recognition. After a new surface electromyographic signal is obtained, the surface electromyographic signal can be input into a matching template, and the gesture corresponding to the surface electromyographic signal can be directly obtained. The matching template can be used for gesture recognition in both network environment and non-network environment, and is not limited by the network.
For example, the matching template is stored in any intelligent device, and the obtained surface electromyographic signals can be input into the intelligent device through a data wire or Bluetooth mode in the environment without a network or a weak network. The intelligent equipment performs matching in the matching template according to the input surface electromyographic signals, and gestures corresponding to the surface electromyographic signals can be obtained. Further, the problem that gesture recognition cannot be performed in a netless environment or a weak network environment is solved.
Specifically, the gesture recognition process is described in detail with reference to fig. 4:
in step 401, a surface electromyographic signal is obtained.
Step 402, performing a preprocessing operation on the surface electromyographic signals to obtain processed surface electromyographic signals.
Step 403, performing active segment detection on the processed surface electromyographic signals.
And step 404, extracting waveform length characteristics of the surface electromyographic signals by utilizing a sliding window algorithm to obtain an extracted characteristic result.
And step 405, performing feature matching according to the obtained extracted feature result and the matching template.
In step 406, a gesture corresponding to the surface electromyographic signal is obtained.
According to the method provided by the embodiment of the application, the surface electromyographic signals are obtained, and the effective surface electromyographic signals are obtained from the surface electromyographic signals, so that the interference of the ineffective surface electromyographic signals is avoided. Further, signal characteristics of the effective surface electromyographic signals are extracted, and gestures corresponding to the surface electromyographic signals are directly acquired according to the signal characteristics. According to the method, a large number of gesture samples are not required to be collected in advance for training, gestures corresponding to the surface electromyographic signals are obtained according to training results, signal characteristics of the effective surface electromyographic signals can be directly extracted, the gestures corresponding to the surface electromyographic signals are rapidly and effectively obtained according to the signal characteristics, the operation process is simple and convenient, and the efficiency is high.
The embodiment of the present application further provides a gesture recognition apparatus, and the specific implementation of the apparatus may be referred to the description of the embodiment of the method, and the repetition is not repeated, as shown in fig. 5, where the apparatus mainly includes:
the first acquisition module 501 is configured to acquire a surface electromyographic signal.
The second obtaining module 502 is configured to obtain an effective surface electromyographic signal from the surface electromyographic signal, where the effective surface electromyographic signal is a surface electromyographic signal in a muscle activity period.
The extracting module 503 is configured to extract a signal feature of the effective surface electromyographic signal.
And the processing module 504 is configured to obtain a gesture corresponding to the surface electromyographic signal according to the signal characteristic.
In a specific embodiment, the extracting module 503 is specifically configured to intercept the surface electromyographic signals by using M data windows to obtain M segments of surface electromyographic signals, where an interval between two adjacent segments of surface electromyographic signals is a preset step, and one of the data windows includes n data points, and the preset step includes M data points;
The following feature extraction operations are respectively carried out on each section of surface electromyographic signals: calculating the absolute value of the difference value of each pair of adjacent data points in the surface electromyographic signals of the p-th section, and carrying out summation processing on all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M;
and summing the summation results of the obtained M sections of surface electromyographic signals, and taking the obtained results as the signal characteristics of the effective surface electromyographic signals.
In one embodiment, the processing module 504 is specifically configured to determine a forearm muscle group corresponding to the signal feature; and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the pre-configured corresponding relation between the forearm muscle group and the gesture.
In a specific embodiment, the second obtaining module 502 is specifically configured to perform correction processing on the surface electromyographic signal to obtain a correction signal; performing integral operation on the obtained correction signal to obtain an envelope signal; and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as an effective surface electromyographic signal. The conversion signal obtained through surface electromyographic signal conversion is corrected by utilizing the baseline threshold value, and then the correction signal is processed by utilizing the kernel function to obtain an envelope signal, wherein the envelope signal increases the tiny difference between an action potential section and a resting potential section in the initial surface electromyographic signal, weakens the fluctuation of the surface electromyographic signal caused by muscle tension, and reduces the misjudgment of an active section caused by the muscle tension.
According to the device provided by the embodiment of the application, the surface electromyographic signals are acquired through the first acquisition module 501; and by utilizing the second acquisition module 502, the effective surface electromyographic signals are acquired from the surface electromyographic signals, so that the interference of the ineffective surface electromyographic signals is avoided. Further, the signal characteristics of the effective surface electromyographic signals are extracted by the extraction module 503. Finally, the processing module 504 directly acquires the gesture corresponding to the surface electromyographic signal according to the signal characteristic. The device does not need to collect a large number of gesture samples in advance for training, obtains the gesture corresponding to the surface electromyographic signals according to the training result, can directly extract the signal characteristics of the effective surface electromyographic signals, and the gesture corresponding to the surface electromyographic signals is obtained fast and effectively according to the signal characteristics, so that the operation process is simple and convenient, and the efficiency is high.
Based on the same concept, the embodiment of the application also provides an electronic device, as shown in fig. 6, where the electronic device mainly includes: a processor 601 and a memory 602. Specifically, the electronic device further includes: a communication component 603 and a communication bus 604, wherein the processor 601, the communication component 603 and the memory 602 perform communication with each other via the communication bus 604. The memory 602 stores a program executable by the processor 601, and the processor 601 executes the program stored in the memory 502 to implement the following steps: acquiring a surface electromyographic signal; obtaining an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period; extracting signal characteristics of an effective surface electromyographic signal; and according to the signal characteristics, obtaining the gestures corresponding to the surface electromyographic signals.
The communication bus 604 mentioned in the above electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, or the like. The communication bus 604 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus.
The communication component 603 is used for communication between the electronic device and other devices described above.
The memory 602 may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (nonvolatile memory), such as at least one disk memory. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor 601.
The processor 601 may be a general-purpose processor, including a central Processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a digital signal processor (DIGITAL SIGNAL Processing, DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
In one embodiment, the electronic device may be an armband device that is worn on the forearm near one third of the elbow joint. The arm ring device can be provided with a switch, the switch is turned on before the detected object makes a gesture, and the gesture is recognized after the detected object makes the gesture.
The embodiment of the application also provides a man-machine interaction method, which comprises the implementation method described in the above embodiment, and the specific implementation can be referred to the description of the embodiment part of the gesture recognition method, and the repetition is omitted.
Specifically, the arm ring device can be applied to bionic prostheses, serves some special people, and can enable the partial people to control the motions of the bionic prostheses through own will and muscles. Taking a bionic artificial limb installed on the left leg of a certain tested object as an example for explanation:
first, the relationship between different gestures and leg movements is pre-saved in the arm ring device, e.g., the gesture of making a fist corresponds to a left leg going forward. When the tested object walks, after the right leg walks forward, the left leg is required to walk forward, at this time, the tested object only needs to make a fist, and the armring equipment acquires and obtains the surface electromyographic signals generated by the forearm muscle group of the tested object. And then, obtaining the gesture of the tested object as a fist making according to the collected surface electromyographic signals through operations such as extraction and the like, and finally, obtaining the instruction of the left leg to the previous step according to the corresponding relation between the gesture of the fist making and the left leg to the previous step.
The arm and ring device, after obtaining the forward step command, sends the command to the biomimetic prosthesis. The bionic artificial limb receives an instruction sent by the arm ring device and completes walking of the tested object. Wherein, the arm ring device is in communication connection with the bionic artificial limb.
In addition, the armring device can be applied to a presentation file (PPT), and the relation between different gestures and execution actions of the PPT can be preset. For example, the gesture of OK corresponds to an action of executing the next page, the gesture of scissors hand corresponds to an action of executing automatic play, and the gesture of making a fist corresponds to an action of executing the previous page. The PPT playing method is characterized in that communication connection is established between the arm ring equipment and a computer, and the computer completes PPT playing according to specific operation instructions sent by the arm ring equipment.
According to the device provided by the embodiment of the application, the processor 601 is used for acquiring the surface electromyographic signals; obtaining an effective surface electromyographic signal from the surface electromyographic signal; extracting signal characteristics of an effective surface electromyographic signal; according to the signal characteristics, gestures corresponding to the surface electromyographic signals are directly acquired. In the processing process, the effective surface electromyographic signals are obtained from the surface electromyographic signals, so that the interference of the ineffective surface electromyographic signals is avoided. The device does not need to collect a large number of gesture samples in advance for training, obtains the gestures corresponding to the surface electromyographic signals according to the training result, can directly extract the signal characteristics of the effective surface electromyographic signals, and rapidly and effectively obtains the gestures corresponding to the surface electromyographic signals according to the signal characteristics, so that the operation process is simple and convenient, and the efficiency is high.
In yet another embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the gesture recognition method described in the above embodiment.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, by a wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, microwave, etc.) means from one website, computer, server, or data center to another. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape, etc.), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (9)

1. A method of gesture recognition, comprising:
Acquiring a surface electromyographic signal;
acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
Extracting signal features of the effective surface electromyographic signals, including: intercepting the surface electromyographic signals by adopting M data windows to obtain M sections of surface electromyographic signals, wherein the interval between two adjacent sections of surface electromyographic signals is a preset step, one data window comprises n data points, and the preset step comprises M data points; the following feature extraction operation is carried out on each section of the surface electromyographic signals respectively: calculating the absolute value of the difference value of each pair of adjacent data points in the surface electromyographic signals of the p-th section, and carrying out summation processing on all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M; summing the summation results of the obtained M segments of the surface electromyographic signals, and taking the obtained results as signal characteristics of the effective surface electromyographic signals;
according to the signal characteristics, obtaining the gesture corresponding to the surface electromyographic signal comprises: determining a forearm muscle group corresponding to the signal characteristic; and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the pre-configured corresponding relation between the forearm muscle group and the gesture.
2. The gesture recognition method of claim 1, wherein the surface electromyographic signals are generated by forearm muscle group movements, and the surface electromyographic signals corresponding to different gestures are different.
3. The gesture recognition method of claim 2, further comprising, prior to obtaining the effective surface electromyographic signal from the surface electromyographic signal:
and filtering noise with preset frequency in the surface electromyographic signals.
4. A gesture recognition method according to claim 3, wherein obtaining an effective surface electromyographic signal from the surface electromyographic signal comprises:
Correcting the surface electromyographic signals to obtain corrected signals;
Performing integral operation on the obtained correction signal to obtain an envelope signal;
and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as the effective surface electromyographic signal.
5. A human-computer interaction method, comprising: the gesture recognition method of any one of claims 1-4.
6. A gesture recognition apparatus, comprising:
The first acquisition module is used for acquiring the surface electromyographic signals;
The second acquisition module is used for acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
The extraction module is used for extracting the signal characteristics of the effective surface electromyographic signals and comprises the following steps: intercepting the surface electromyographic signals by adopting M data windows to obtain M sections of surface electromyographic signals, wherein the interval between two adjacent sections of surface electromyographic signals is a preset step, one data window comprises n data points, and the preset step comprises M data points; the following feature extraction operation is carried out on each section of the surface electromyographic signals respectively: calculating the absolute value of the difference value of each pair of adjacent data points in the surface electromyographic signals of the p-th section, and carrying out summation processing on all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M; summing the summation results of the obtained M segments of the surface electromyographic signals, and taking the obtained results as signal characteristics of the effective surface electromyographic signals;
The processing module is used for obtaining the gesture corresponding to the surface electromyographic signal according to the signal characteristics, and comprises the following steps: determining a forearm muscle group corresponding to the signal characteristic; and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the pre-configured corresponding relation between the forearm muscle group and the gesture.
7. An electronic device, comprising: a processor and a memory for executing a program stored in the memory to implement the gesture recognition method of any one of claims 1-4.
8. The electronic device of claim 7, wherein the electronic device is an armband device for wearing on an upper limb forearm near one third of an elbow joint.
9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the gesture recognition method of any one of claims 1-4.
CN202010367114.5A 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium Active CN113589920B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010367114.5A CN113589920B (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium
PCT/CN2021/090715 WO2021219039A1 (en) 2020-04-30 2021-04-28 Gesture recognition method and apparatus, man-machine interaction method and apparatus, and device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010367114.5A CN113589920B (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113589920A CN113589920A (en) 2021-11-02
CN113589920B true CN113589920B (en) 2024-06-18

Family

ID=78237601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010367114.5A Active CN113589920B (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium

Country Status (2)

Country Link
CN (1) CN113589920B (en)
WO (1) WO2021219039A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO348382B1 (en) 2021-11-05 2024-12-23 Elliptic Laboratories Asa Remote presence detection system
CN113970968B (en) * 2021-12-22 2022-05-17 深圳市心流科技有限公司 Intelligent bionic hand action pre-judging method
CN113986017B (en) * 2021-12-27 2022-05-17 深圳市心流科技有限公司 Myoelectric gesture template generation method and device and storage medium
CN114167996B (en) * 2022-02-14 2022-05-17 浙江强脑科技有限公司 Sensor-based action pre-judging method and device and storage medium
CN114690905A (en) * 2022-03-30 2022-07-01 歌尔股份有限公司 Gesture control method, system and device, wearable product and medium
CN114625257B (en) * 2022-05-16 2022-08-16 浙江强脑科技有限公司 Action recognition method and device based on electromyographic signals
CN115114962B (en) * 2022-07-19 2025-07-04 歌尔股份有限公司 Control method, device and wearable device based on surface electromyography signal
CN115131878B (en) * 2022-08-30 2022-11-11 深圳市心流科技有限公司 Method, device, terminal and storage medium for determining motion of intelligent artificial limb
CN116400812B (en) * 2023-06-05 2023-09-12 中国科学院自动化研究所 Emergency rescue gesture recognition method and device based on surface electromyographic signals
CN116725556B (en) * 2023-07-12 2024-11-22 中国科学院苏州生物医学工程技术研究所 A method and device for identifying movement intention based on surface electromyography signal
CN117032448B (en) * 2023-07-13 2025-09-26 安徽理工大学 PPT control method based on human posture estimation

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110420025A (en) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 Surface electromyogram signal processing method, device and wearable device
CN110618754A (en) * 2019-08-30 2019-12-27 电子科技大学 Surface electromyogram signal-based gesture recognition method and gesture recognition armband
CN110826625A (en) * 2019-11-06 2020-02-21 南昌大学 A Finger Gesture Classification Method Based on Surface EMG Signals

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6610917B2 (en) * 1998-05-15 2003-08-26 Lester F. Ludwig Activity indication, external source, and processing loop provisions for driven vibrating-element environments
RU2683859C1 (en) * 2017-12-27 2019-04-02 ОБЩЕСТВО С ОГРАНИЧЕННОЙ ОТВЕТСТВЕННОСТЬЮ "Би-оН ЭМГ" Method and system for controlling electronic devices by electromyographic reading device
CN111026268A (en) * 2019-12-02 2020-04-17 清华大学 Gesture recognition device and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110618754A (en) * 2019-08-30 2019-12-27 电子科技大学 Surface electromyogram signal-based gesture recognition method and gesture recognition armband
CN110420025A (en) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 Surface electromyogram signal processing method, device and wearable device
CN110826625A (en) * 2019-11-06 2020-02-21 南昌大学 A Finger Gesture Classification Method Based on Surface EMG Signals

Also Published As

Publication number Publication date
WO2021219039A1 (en) 2021-11-04
CN113589920A (en) 2021-11-02

Similar Documents

Publication Publication Date Title
CN113589920B (en) Gesture recognition method, man-machine interaction method, device, equipment and storage medium
Jose et al. Classification of forearm movements from sEMG time domain features using machine learning algorithms
Manimegalai Hand gesture recognition based on EMG signals using ANN
CN111317600A (en) A prosthetic control method, device, system, device and storage medium
CN113111831A (en) Gesture recognition technology based on multi-mode information fusion
Hamedi et al. Surface electromyography-based facial expression recognition in Bi-polar configuration
Powar et al. A novel pre-processing procedure for enhanced feature extraction and characterization of electromyogram signals
Oweis et al. ANN-based EMG classification for myoelectric control
Peng et al. Feasibility of NeuCube spiking neural network architecture for EMG pattern recognition
Li et al. Wireless sEMG-based identification in a virtual reality environment
Wang et al. Classification of surface EMG signals using harmonic wavelet packet transform
CN114098768A (en) Cross-individual surface electromyographic signal gesture recognition method based on dynamic threshold and EasyTL
Pirasteh et al. EEG-based brain-computer interface methods with the aim of rehabilitating advanced stage ALS patients
CN110604578A (en) SEMG-based motion recognition method of human hand
Ibrahimy et al. Design and performance analysis of artificial neural network for hand motion detection from EMG signals
CN116831874A (en) A control method for lower limb rehabilitation device based on electromyographic signals
CN111923048A (en) Electromyographic signal classification and exoskeleton robot control method and device
Choudhary et al. A machine learning approach to aid paralysis patients using EMG signals
Dzitac et al. Identification of ERD using fuzzy inference systems for brain-computer interface
CN119820582A (en) Adaptive control method and system for humanoid robot, electronic equipment and storage medium
CN115965127A (en) Muscle fatigue prediction method and system based on multi-element signal fusion and electric stimulator
Ye Optimization and Application of Gesture Classification Algorithm Based on EMG
CN114983446A (en) A method for continuous motion estimation of finger multi-joint based on EMG signals
Fu et al. Identification of finger movements from forearm surface EMG using an augmented probabilistic neural network
CN106510716B (en) Method and device for calculating respiration rate based on electrocardiosignals

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant after: Jingdong Technology Information Technology Co.,Ltd.

Address before: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant before: Jingdong Shuke Haiyi Information Technology Co.,Ltd.

Address after: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant after: Jingdong Shuke Haiyi Information Technology Co.,Ltd.

Address before: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Beijing Economic and Technological Development Zone, Beijing 100176

Applicant before: BEIJING HAIYI TONGZHAN INFORMATION TECHNOLOGY Co.,Ltd.

GR01 Patent grant
GR01 Patent grant