CN114005182A

CN114005182A - Gesture recognition method, device and electronic device

Info

Publication number: CN114005182A
Application number: CN202111333990.7A
Authority: CN
Inventors: 赵小川; 郑君哲; 黄杰; 马燕琳; 宋刚; 刘莹
Original assignee: China North Computer Application Technology Research Institute
Current assignee: China North Computer Application Technology Research Institute
Priority date: 2021-11-11
Filing date: 2021-11-11
Publication date: 2022-02-01
Anticipated expiration: 2041-11-11
Also published as: CN114005182B

Abstract

The present application provides a gesture recognition method, device and electronic device, wherein the gesture recognition method includes: acquiring a hand gesture signal reflecting the bending degree of fingers of an operator's hand; acquiring an arm reflecting the gravitational acceleration of the operator's arm hand gesture signal; according to the first model, the hand gesture recognition result corresponding to the hand gesture signal is obtained; wherein, the first model reflects the first mapping relationship between the hand gesture signal and the hand gesture recognition result; according to the second model to obtain the arm gesture recognition result corresponding to the arm gesture signal; wherein, the second model reflects the second mapping relationship between the arm gesture signal and the arm gesture recognition result; according to the hand gesture recognition result and the arm gesture From the recognition result, the operator's gesture recognition result is obtained; according to the gesture recognition result, the corresponding operation is performed.

Description

Gesture recognition method and device and electronic equipment

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a gesture recognition method and device and electronic equipment.

Background

Gesture recognition is a subject of computer science and language technology, with the aim of recognizing human gestures by mathematical algorithms. Gesture recognition may recognize motion from various parts of a person's body, but generally refers to motion of a hand. A user can use simple gestures to control or interact with a device, letting a computer understand human behavior.

In the prior art, when gesture recognition is performed by an optical marking-based method, a gesture recognition result is affected by factors such as illumination and shielding, and the stability of the recognition result is poor. When gesture recognition is carried out on the basis of the data gloves, only the gesture of the hand can be recognized, and the gesture expansion capability is low. When gesture recognition is performed based on a gyroscope and an accelerometer, although the degree of freedom of measurement of gesture postures is higher, and more gesture postures can be carved, the calculation process is complex, and the sensor drift phenomenon is easily generated, so that the gesture recognition result generates errors.

Disclosure of Invention

The embodiment of the application aims to provide a gesture recognition method, a gesture recognition device and electronic equipment, and the problems that the gesture expansion capability is low and the stability of a gesture recognition result is poor can be solved.

In order to solve the technical problem, the present application is implemented as follows:

in a first aspect, an embodiment of the present application provides a gesture recognition method, including:

acquiring a hand posture signal reflecting the degree of curvature of fingers of the hand of an operator;

acquiring an arm attitude signal reflecting the gravitational acceleration of the arm of the operator;

obtaining a hand gesture recognition result corresponding to the hand gesture signal according to a first model; wherein the first model reflects a first mapping relationship between the hand gesture signal and the hand gesture recognition result;

obtaining an arm gesture recognition result corresponding to the arm gesture signal according to a second model; wherein the second model reflects a second mapping relationship between the arm pose signal and the arm pose recognition result;

obtaining a gesture recognition result of the operator according to the hand gesture recognition result and the arm gesture recognition result;

and executing corresponding operation according to the gesture recognition result.

Optionally, the acquiring a hand gesture signal reflecting the degree of finger curvature of the hand of the operator includes:

acquiring a hand electric signal meeting a first stable condition; the hand electric signals comprise multi-channel electric signals which respectively reflect the bending degree of each finger of the operator, the first stable condition comprises that the deviation amount between the currently sampled hand electric signals and the previously sampled hand electric signals for N times is within a first set range, and N is greater than or equal to 1;

and obtaining the hand gesture signal according to the hand electric signal meeting the first stable condition.

Optionally, the obtaining the hand posture signal according to the hand electric signal satisfying the first stable condition includes:

and normalizing the hand electric signals meeting the first stable condition to obtain the hand posture signals.

Optionally, the acquiring an arm posture signal reflecting a gravitational acceleration of the arm of the operator includes:

acquiring a gravity acceleration signal reflecting the posture of the arm of the operator; wherein the gravitational acceleration signal comprises a first gravitational acceleration signal reflecting a big arm posture and a second gravitational acceleration signal reflecting a small arm posture of the operator;

and obtaining the arm posture signal according to the first gravity acceleration signal and the second gravity acceleration signal.

Optionally, the gravitational acceleration signal satisfies a second stability condition; the second stability condition comprises: and the total deviation amount of the gravity acceleration signal of the current acquisition period and the gravity acceleration signals of the previous M sampling periods is within a second set range, wherein M is greater than or equal to 1.

Optionally, the gravitational acceleration signal comprises gravitational acceleration values of an x-axis, a y-axis, and a z-axis; said M ═ 4;

the total deviation is expressed as:

where i ═ 0 denotes a first gravitational acceleration signal, i ═ 1 denotes a second gravitational acceleration signal, a ═ x denotes a gravitational acceleration value on the x axis, a ═ y denotes a gravitational acceleration value on the y axis, a ═ z denotes a gravitational acceleration value on the z axis, T denotes a sampling period, T denotes a current sampling period, T-4, T-3, T-2, T-1, and T denote the previous 4 consecutive sampling periods, D denotes a sampling period, and c denotes a sampling period on the x axis and a sampling period on the y axis, and c denotes a sampling period on the z axis²Representing the derivative squared of the gravitational acceleration.

Optionally, the step of constructing the first model comprises:

acquiring a hand gesture signal sample set corresponding to each preset hand gesture;

constructing training data, verification data and test data based on the hand gesture signal sample set corresponding to each preset hand gesture;

obtaining at least two first basic models corresponding to the first model;

training model parameters of each first basic model by using the training data and the verification data to obtain second basic models corresponding to the first basic models;

testing the preset performance indexes of the second basic models by using the test data to obtain index values of the preset performance indexes corresponding to the second basic models;

and taking a second basic model of which the index value meets a preset condition as the first model.

Optionally, the second model is a data lookup table that embodies the second mapping relationship.

In a second aspect, an embodiment of the present application provides a gesture recognition apparatus, including:

the first acquisition module is used for acquiring a hand gesture signal reflecting the finger curvature of the hand of the operator;

the second acquisition module is used for acquiring an arm posture signal reflecting the gravity acceleration of the arm of the operator;

the hand gesture recognition module is used for obtaining a hand gesture recognition result corresponding to the hand gesture signal according to the first model; wherein the first model reflects a first mapping relationship between the hand gesture signal and the hand gesture recognition result;

the arm gesture recognition module is used for obtaining an arm gesture recognition result corresponding to the arm gesture signal according to the second model; wherein the second model reflects a second mapping relationship between the arm pose signal and the arm pose recognition result;

the gesture recognition module is used for obtaining a gesture recognition result of the operator according to the hand gesture recognition result and the arm gesture recognition result;

and the execution module is used for executing corresponding operation according to the gesture recognition result.

In a third aspect, an embodiment of the present application provides an electronic device, including the gesture recognition apparatus according to the second aspect; alternatively, the electronic device comprises a memory for storing a computer program and a processor for controlling the electronic device to perform the method steps according to any of the first aspect under control of the computer program.

According to the gesture recognition method, the arm posture signals reflecting the gravity acceleration of the arm of the operator are selected to recognize the posture of the arm, and the gravity acceleration is only related to the position and the direction of the sensor relative to the ground, so that the problems of integral calculation and data drifting are not involved in the process of obtaining the arm posture signals, the accuracy of the obtained arm posture signals is high, and the accuracy of the arm posture recognition result and the accuracy of the gesture recognition result are improved. Meanwhile, the final gesture recognition result is obtained through the fusion of the arm gesture recognition result and the hand gesture recognition result, the types of recognizable gestures are increased, and the expandability of the gestures is improved. Furthermore, the acquisition process of the hand gesture signal and the arm gesture signal is not influenced by the external environment, so that the stability of the gesture recognition result can be improved.

Other features of the present description and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the specification and together with the description, serve to explain the principles of the specification.

FIG. 1 is a schematic structural diagram of a gesture recognition system that can be used to implement a gesture recognition method according to an embodiment of the present application;

FIG. 2 is a flow diagram of a gesture recognition method according to one embodiment;

FIG. 3 is a schematic diagram of a properly-matched inertial sensor, according to one embodiment;

FIG. 4 is a schematic diagram of a format for recording hand electrical signal samples, according to one embodiment;

FIG. 5 is a schematic diagram illustrating the display effect of samples in a sample set in a three-dimensional space according to an embodiment;

FIG. 6 is a test result diagram of the accuracy of the predicted results of the models, according to one embodiment;

FIG. 7 is a flowchart diagram of an example of a gesture recognition method according to one embodiment;

FIG. 8 is a functional block diagram of a gesture recognition device according to one embodiment;

FIG. 9 is a schematic structural diagram of an electronic device according to one embodiment;

fig. 10 is a schematic structural diagram of an electronic device according to another embodiment.

Detailed Description

Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

< hardware configuration >

Fig. 1 is a schematic structural diagram of a gesture recognition system that can be used to implement a gesture recognition method according to an embodiment of the present application.

As shown in fig. 1, the gesture recognition system 1000 includes a data glove 1100, a first inertial sensor 1200, a second inertial sensor 1300, a core processor 1400, and a smart terminal 1500. The data glove 1100, the first inertial sensor 1200 and the second inertial sensor 1300 are respectively in communication connection with the core processor 1400, and the core processor 1400 is in communication connection with the intelligent terminal 1500.

In one embodiment, data glove 1100 includes a glove body, 5 flexible piezoresistive sensors, a hub module, and a first data acquisition module.

In the embodiment, the glove body is provided with the interlayer, and the appearance of the glove is the same as that of a common glove. The 5 flexible piezoresistive sensors and the line concentration structure are arranged in the interlayer of the glove body. Wherein, 5 flexible piezoresistive sensors are all slender thin strips, set up respectively in the intermediate layer of 5 finger backs of gloves body from the joint department of connecting of finger and palm to the fingertip, cover the dorsal part of every finger, can follow the motion of finger and take place the deformation. This line concentration structure sets up in the intermediate layer of this back of the hand region of this gloves for connect 5 flexible piezoresistive sensors and first data acquisition module, so that the wiring between 5 flexible piezoresistive sensors and the first data acquisition module is more reasonable.

In this embodiment, the first data acquisition module is disposed on a back side of the glove body near the wrist. The first data acquisition module comprises a voltage acquisition chip, a first processor and a first wireless communication module.

In this embodiment, the voltage acquisition chip is configured to acquire a plurality of voltage signals output by the 5 flexible piezoresistive sensors, and send the acquired plurality of voltage signals to the first processor through a Serial Peripheral Interface (SPI).

In this embodiment, the first processor performs smoothing filtering and average filtering processing on the multiple voltage signals under the condition of the received multiple voltage signals; calculating a plurality of paths of resistance signals reflecting the bending degree of each finger of the operator according to the processed plurality of paths of voltage signals; generating a hand electric signal according to the calculated multi-path resistance signal; and sending the hand electric signal to the first wireless communication module through the Serial Peripheral Interface (SPI). The first processor may be an ARM processor, or may be another processor, which is not specifically limited herein.

In this embodiment, the first wireless communication module receives the hand electrical signal and sends the hand electrical signal to the core processor 1400. The first wireless communication module may be a WIFI module, or may be another module capable of implementing wireless communication, which is not limited specifically herein.

The surface of the flexible piezoresistive sensor is a layer of special resistance material, and when the flexible piezoresistive sensor is subjected to stress and is bent and deformed, the resistance value of the surface changes. The larger the bending degree is, the larger the resistance value is, that is, the finger bending degree and the resistance value of the flexible piezoresistive sensor have a corresponding relationship, so that the finger bending degree of the operator can be reflected by the resistance value of the flexible piezoresistive sensor.

In one embodiment, the first inertial sensor 1200 and the second inertial sensor 1300 are identical in structure. The hardware configuration of the first inertial sensor 1200 and the second inertial sensor 1300 will be described below by taking the first inertial sensor as an example.

In this embodiment, the first inertial sensor 1200 includes an elastic band and a second data acquisition module. The second data acquisition module is arranged on the elastic band, and can be fixed at the corresponding position of the arm of the operator through elasticity.

In this embodiment, the second data acquisition module includes an attitude sensor, a second processor, and a second wireless communication module.

In this embodiment, the attitude sensor may collect three-axis x-axis, y-axis, and z-axis gravitational acceleration signals, and send the collected three-axis gravitational acceleration signals to the second processor. The attitude sensor may be a MEMS accelerometer, or may be another motion sensor that can acquire a three-axis gravity acceleration signal, which is not limited in this respect.

In this embodiment, the second processing unit performs smoothing filtering and mean filtering on the gravitational acceleration signal when receiving the gravitational acceleration signal; and sending the processed gravity acceleration signal to a second wireless communication module through a Serial Peripheral Interface (SPI). The second processor may be an ARM processor, or may be another processor, which is not specifically limited herein.

In this embodiment, the second wireless communication module sends the processed gravitational acceleration signal to the core processor 1400 when receiving the processed gravitational acceleration signal. The second wireless communication module may be a WIFI module, or may be another module capable of implementing wireless communication, and is not limited specifically herein.

When the posture of the arm changes, the gravity acceleration signals of the posture sensor in the directions of the x axis, the y axis and the z axis are changed, namely the posture of the arm and the three-axis gravity acceleration signals have a corresponding relation, so that the posture of the arm of the operator can be reflected through the three-axis gravity acceleration signals.

In one embodiment, the arm positions of the operator include a big arm position and a small arm position. In order to respectively acquire the upper arm posture and the lower arm posture of the operator, a first inertial sensor can be fixed at the middle position of the upper arm of the operator and used for acquiring a first gravity acceleration signal reflecting the upper arm posture. A second inertial sensor may be affixed to the operator's forearm proximate the wrist for acquiring a second gravitational acceleration signal reflecting the operator's forearm posture. The first gravitational acceleration signal and the second gravitational acceleration signal constitute a gravitational acceleration signal reflecting the posture of the arm of the operator.

In one embodiment, the core processor 1400 includes a third wireless communication module, a third processor, a smart chip, and a data link module.

In this embodiment, the third wireless communication module sends the hand electric signal and the gravity acceleration signal to the third processor when receiving the hand electric signal and the gravity acceleration signal; the third wireless communication module may be a WIFI module, or may be another module capable of implementing wireless communication, and is not limited specifically herein.

In this embodiment, the third processor obtains a hand posture signal according to the hand electrical signal satisfying the first stable condition, when the hand electrical signal and the gravitational acceleration signal are judged to be stable according to the preset stable condition; obtaining an arm posture signal according to the gravity acceleration signal meeting the stable condition; sending the hand posture signal and the arm posture signal to an intelligent chip; and generating a corresponding operation instruction according to the gesture recognition result returned by the intelligent chip and sending the operation instruction to the data chain module. The third processor may be an ARM processor, or may be another processor, which is not specifically limited herein.

In this embodiment, the smart chip stores a preset first model, a preset second model, and a preset third model, and when the hand gesture signal is received, a hand gesture recognition result corresponding to the hand gesture signal can be obtained according to the first model. In the case where the arm posture signal is received, an arm posture recognition result corresponding to the arm posture signal may be obtained according to the second model. Under the condition that the hand gesture recognition result and the arm gesture recognition result are obtained, the hand gesture recognition result and the gesture recognition result of the arm gesture recognition result can be corresponding according to the third model, and the obtained gesture recognition result is sent to the third processor.

After receiving the operation instruction corresponding to the gesture recognition result, the data link module sends the operation instruction to the intelligent terminal 1500, and the intelligent terminal responds to the operation instruction and performs corresponding operation.

It should be understood that although fig. 1 shows only one data glove 1100, first inertial sensor 1200, second inertial sensor 1300, core processor 1400, and smart terminal 1500, the respective numbers are not meant to be limiting, and multiple data gloves 1100, first inertial sensors 1200, second inertial sensors 1300, core processor 1400, and smart terminals 1500 may be included in the present system.

Various embodiments and examples according to the present invention are described below with reference to the accompanying drawings.

< method examples >

FIG. 2 is a flow diagram of a gesture recognition method according to one embodiment.

As shown in fig. 2, the gesture recognition method of the present embodiment may include steps S210-S260.

S210, a hand posture signal reflecting the finger curvature of the hand of the operator is obtained.

When the operator makes different hand gestures, the degree of curvature of each finger changes. For example, when the operator makes a hand gesture of "2", the corresponding finger curvatures may be: the curvature of the forefinger and the middle finger is 0 degree, and the curvature of the thumb, the ring finger and the little finger is 180 degrees. For another example, when the operator makes a hand gesture of "0", the corresponding degree of curvature of each finger may be 180 degrees. For another example, when the operator makes a hand gesture of "5", the degree of curvature corresponding to each finger may be 0 degree. Therefore, the hand gesture recognition can be carried out through the hand gesture signal reflecting the finger curvature of the hand of the operator.

In one embodiment, obtaining a hand gesture signal reflecting a degree of finger curvature of an operator' S hand includes steps S211-S212.

S211, acquiring a hand electric signal meeting a first stable condition; the hand electric signal comprises a plurality of paths of electric signals which respectively reflect the bending degree of each finger of an operator.

In one embodiment, the operator's hand electrical signals may be collected by data glove 1100 of FIG. 1. The operator can make a hand gesture to be recognized while wearing the data glove 1100. The 5 flexible piezoresistive sensors in the data glove 1100 each output a voltage signal according to the bending of the corresponding finger. Under the condition that a first data acquisition module in the data glove 1100 receives 5 paths of voltage signals output by 5 flexible piezoresistive sensors, respectively preprocessing the 5 paths of voltage signals, and respectively calculating 5 paths of resistance signals reflecting the bending degree of each finger according to the preprocessed 5 paths of voltage signals; and generating a hand electric signal according to the 5 paths of calculated resistance signals, wherein the hand electric signal comprises a plurality of paths of electric signals respectively reflecting the bending degree of each finger of the operator, and the plurality of paths of electric signals are the plurality of paths of resistance signals.

In one embodiment, the data glove 1100 collects hand electrical signals according to a first predetermined frequency and transmits the collected hand electrical signals to the core processor 1400 at a second predetermined frequency. The first frequency and the second frequency may be preset frequencies, or may be configured according to a specific application scenario, and are not limited in this respect. The first frequency and the second frequency may be the same or different, and are not limited herein.

For example, the first frequency of the data glove 1100 is preset to be equal to the second frequency to be equal to 100Hz, and at this time, the data glove 1100 will collect hand electric signals at a sampling frequency of 100 times per second and send the collected hand electric signals to the core processor 1400 in real time.

The operator's hand gesture is a moving to static process. When the hand gesture remains unchanged within the set time, the hand electric signal output by the data glove 1100 is also in a stable state, and the hand gesture at this time is the hand gesture to be recognized.

In one embodiment, in order to improve the accuracy of hand gesture recognition, after receiving the hand electrical signal sent by the data glove 1100, the core processor 1400 first determines whether the current hand electrical signal is stable according to a preset first stable condition, and in case that the current hand electrical signal is determined to be stable, acquires the hand electrical signal meeting the first stable condition to execute step S212.

In one embodiment, the first stable condition may include that a deviation amount between the currently sampled hand electric signal and the hand electric signal sampled N times before is within a first set range, where N is greater than or equal to 1. N and the first setting range may be preset values, or may be configured according to a specific application scenario, and are not specifically limited herein.

For example, if N is set to 19 in advance and the first setting range is 0, the first stable condition is that the hand electrical signal obtained by the current sampling is the same as the hand electrical signal obtained by the previous 19 times of sampling, and it is considered that the hand electrical signal obtained by the current sampling satisfies the first stable condition.

And S212, obtaining a hand posture signal according to the hand electric signal meeting the first stable condition.

In one embodiment, deriving the hand gesture signal from the hand electrical signal satisfying the first stability condition comprises: and normalizing the hand electric signals meeting the first stable condition to obtain hand posture signals.

And S220, acquiring an arm posture signal reflecting the gravity acceleration of the arm of the operator.

When an operator makes different arm postures, the position and the direction of any position of the arm relative to the ground can be changed, so that the gravity acceleration signal acquired by the inertial sensor fixed at the set position of the arm of the operator is changed. As can be seen, since the acceleration value of gravity corresponding to the set position of the arm has a certain correspondence relationship with the posture of the arm, the posture of the arm can be recognized by an arm posture signal reflecting the acceleration of gravity of the arm of the operator.

In one embodiment, acquiring an arm posture signal reflecting the gravitational acceleration of the operator' S arm includes steps S221-S222.

S221, acquiring a gravity acceleration signal reflecting the posture of the arm of the operator; wherein the gravitational acceleration signal includes a first gravitational acceleration signal reflecting the posture of the upper arm and a second gravitational acceleration signal reflecting the posture of the lower arm of the operator.

The arm posture is composed of a big arm posture and a small arm posture. For example, the posture of the arm part can be that the forearm is level with the ground, and the forearm is everted; the big arm of the small arm is vertically downward; the big arm can be horizontal to the ground, the small arm can be vertical to the ground, and the like.

In order to accurately identify the posture of the arm part of the operator, a first inertial sensor is fixed at the middle position of the upper arm of the operator and used for acquiring a first gravity acceleration signal reflecting the posture of the upper arm. And a second inertial sensor is fixed at the position of the forearm of the operator close to the wrist and is used for acquiring a second gravity acceleration signal reflecting the posture of the forearm of the operator.

In one embodiment, the first inertial sensor and the second inertial sensor may each acquire three-axis gravitational acceleration signals for the x-axis, the y-axis, and the z-axis. When an operator correctly wears the first inertial sensor and the second inertial sensor, the matching graph of the x axis, the y axis and the z axis of the two inertial sensors and the arm direction is shown in fig. 3, when the arm of the operator naturally hangs down, the five fingers stretch straight, and the palm center faces backwards, the x axis of the two inertial sensors points to the finger tip direction, and the y axis, the z axis and the x axis meet the right hand coordinate system.

When the posture of the arm is kept unchanged within a set time, the gravity acceleration signal is in a stable state, and the posture of the arm at the moment is the posture of the arm to be recognized.

In one embodiment, in order to improve the accuracy of the arm posture recognition, the core processor 1400 determines whether the gravity acceleration signal satisfies a preset second stable condition after the current sampling period is ended, and if the second stable condition is satisfied, takes the gravity acceleration signal obtained last in the current sampling period as the gravity acceleration signal satisfying the second stable condition, so as to execute step S222.

In one embodiment, the second stable condition includes: the total deviation amount of the gravity acceleration signal of the current acquisition period and the gravity acceleration signals of the previous M sampling periods is within a second set range, and M is greater than or equal to 1. Here, M and the second setting range may be preset values, or may be configured according to a specific application scenario, and are not specifically limited herein.

In one embodiment, the gravitational acceleration signal comprises an x-axis, a y-axis, and a z-axis gravitational acceleration signal, and the corresponding gravitational acceleration signal comprises a first gravitational acceleration signal and a second gravitational acceleration signal that also comprise an x-axis, a y-axis, and a z-axis gravitational acceleration signal.

In this embodiment, the deviation amount of the gravitational acceleration signal in the current sampling period is equal to the sum of squares of derivatives of the gravitational acceleration signal in three axes in the first gravitational acceleration signal and the gravitational acceleration signal in three axes in the second gravitational acceleration signal with respect to time, which is the total of 6 axes of the gravitational acceleration signal.

When M is set to 4 in advance and the second setting range is 0, the total deviation amount may be expressed as:

where i ═ 0 denotes a first gravitational acceleration signal, i ═ 1 denotes a second gravitational acceleration signal, a ═ x denotes a gravitational acceleration value on the x axis, a ═ y denotes a gravitational acceleration value on the y axis, a ═ z denotes a gravitational acceleration value on the z axis, T denotes a sampling period, T denotes a current sampling period, T-4, T-3, T-2, T-1, and T denote the previous 4 consecutive sampling periods, D denotes a sampling period, and c denotes a sampling period on the x axis and a sampling period on the y axis, and c denotes a sampling period on the z axis²Representing the derivative squared of the gravitational acceleration with respect to time.

When the posture of the arm of the operator is stable, the derivative of the gravitational acceleration signal of each axis with respect to time should be 0, but since the human body has unavoidable high-frequency vibration to be detected by the inertial sensor, the derivative of the gravitational acceleration signal of each axis with respect to time tends to fluctuate around the value of 0. In order to avoid the influence of the high-frequency vibration on the stability judgment, a derivative threshold value is usually preset, and when the derivative of the gravity acceleration signal of any axis with respect to time is less than or equal to the preset derivative threshold value, the derivative of the gravity acceleration signal of the axis with respect to time can be directly set to 0. Through multiple experimental analyses, the derivative threshold may be set to 0.3, or may be set according to a specific application scenario, and is not specifically limited herein.

And S222, obtaining an arm posture signal according to the first gravity acceleration signal and the second gravity acceleration signal.

The value range of the gravity acceleration signal is-9.8 to 9.8. When the positive direction of a certain axis points to the ground vertically, the acceleration value of the corresponding axis is +9.8, otherwise, the acceleration value is-9.8.

In an embodiment where the first gravitational acceleration signal and the second gravitational acceleration signal each include a gravitational acceleration signal of an x-axis, a y-axis, and a z-axis, step S222 includes steps S222-1 to S222-3 in order to improve the efficiency of the arm posture recognition.

S222-1, respectively mapping the three-axis gravity acceleration signal values of the first gravity acceleration signal into three reference quantities of 0,1 and 2 through a preset data mapping step to obtain a mapped first gravity acceleration signal.

The following takes x-axis as an example to illustrate the specific steps of data mapping: the value of the x-axis gravity acceleration signal is divided by 9.8 to obtain first mapping data of the x axis. Mapping the first mapping data of the x-axis to one of three reference quantities, namely 0,1 and 2 according to a data mapping table shown in table 1 to obtain second mapping data of the x-axis. Taking the example of the x-axis, second mapping data of the y-axis and the z-axis are obtained respectively. And obtaining a first mapped gravity acceleration signal according to the second mapping data of the x axis, the y axis and the z axis.

For example, when the second mapping data of the x-axis is 1, the second mapping data of the y-axis is 2, and the second mapping data of the z-axis is 1, the mapped first gravitational acceleration signal is [1, 2, 1 ].

Table 1:

mapping rules	Reference quantity
		First mapping data > first mapping threshold	2
Second mapping threshold < first mapping data < first mapping threshold	1
		First mapping data < second mapping threshold	0

The second mapping threshold in table 1 is less than the first mapping threshold. For example, the first mapping threshold may be 0.6 and the second mapping threshold may be-0.6.

And S222-2, referring to the step S222-1, respectively mapping the triaxial gravity acceleration signal values of the second gravity acceleration signal into three reference quantities of 0,1 and 2 to obtain a mapped second gravity acceleration signal.

S222-3, obtaining an arm posture signal according to the mapped first gravity acceleration signal and the mapped second gravity acceleration signal.

In one embodiment, the arm attitude signal may be derived from second mapping data of the x-axis in the first gravitational acceleration signal and the second gravitational acceleration signal.

For example, the first gravitational acceleration signal is [1, 1, 2], the second mapping data 0 of the x-axis in the second gravitational acceleration signal, and the arm posture signal is [1, 1, 2, 0 ].

And S230, obtaining a hand gesture recognition result corresponding to the hand gesture signal according to the first model. The first model reflects a first mapping relation between the hand gesture signal and the hand gesture recognition result.

In one embodiment, the step of constructing the first model comprises steps S231-S236.

And S231, acquiring a hand gesture signal sample set corresponding to each preset hand gesture.

In this embodiment, step S231 specifically includes steps S231-1 to S231-2.

S231-1, acquiring hand electric signal sample sets corresponding to the preset hand postures.

In this embodiment, gather the corresponding hand electric signal sample of each preset hand gesture through data gloves 1100, carry out the automatic recording of hand electric signal through the python script, the flow is as follows:

1) and inputting the number of the hand gesture to be recorded when the script is started.

The numbers are typically numbered in numerical order and incremented in sequence. For example, the number of the first hand gesture to be recorded may be 1, the number of the second hand gesture to be recorded may be 2, the number of the third hand gesture to be recorded may be 3, and so on, the number of each hand gesture to be recorded is set.

2) And after the number of the hand gesture needing to be recorded is input, starting countdown for 5 seconds, making the hand gesture needing to be recorded, and keeping the fingers as still as possible. And after the countdown is finished, the script records hand electric signal samples corresponding to the current hand gestures, and displays the hand electric signal samples and the recorded number of the hand electric signal samples in the terminal equipment.

3) And returning the hand gesture to be recorded after the fingers are moved rapidly, keeping the hand gesture for 0.2 second, taking the hand electric signal stably kept for 0.2 second as an effective hand electric signal sample by the script, recording, and updating the number of the recorded hand electric signal samples.

After the displayed hand electric signal sample is recorded, if the finger is kept still, the same hand electric signal sample cannot be repeatedly recorded, so that the hand electric signal sample needs to be acquired by returning to the hand gesture needing to be recorded again after the finger is moved rapidly.

4) And (5) repeatedly executing the step 3) until the number of the recorded hand electric signal samples is within the first range, and interrupting the Python terminal. Wherein, the first range may be 100-200.

And stopping collecting the hand electric signal sample after the Python terminal is interrupted. The script writes the recorded hand electrical signal samples into a file in the format of/data/[ i ]/YYYY _ MM _ DD _ hh _ MM _ ss.txt. The format of the hand electrical signal samples recorded in each txt file is shown in fig. 4, which includes n rows and 5 columns. n is the number of effective hand electric signal samples corresponding to the hand gesture required to be recorded, 5 means that each hand electric signal comprises 5 channels, five channels respectively correspond to 5 fingers, and the numerical value under each channel is a resistance signal value reflecting the curvature of the finger.

5) And taking all the recorded hand electric signal samples as a hand electric signal sample set corresponding to the hand gesture to be recorded.

6) And repeating the steps 1) to 5) to obtain a hand electric signal sample set corresponding to each preset hand gesture.

S231-2, preprocessing the hand electric signal sample set corresponding to each preset hand gesture to obtain a hand gesture signal sample set corresponding to each preset hand gesture.

In one embodiment, the pre-processing consists essentially of:

1) and clearing the deviated samples in each sample set.

In the process of collecting hand electric signal samples corresponding to each preset hand gesture, fingers have certain intermediate states to be used as effective actions when performing action deformation, and at the moment, hand electric signal samples collected by the data glove 1100 are obviously different from hand electric signal samples corresponding to hand gestures to be collected and should be removed, so that the accuracy of the collected hand electric signal samples is improved.

2) And carrying out linear normalization processing on the samples in each sample set.

The value range of each path of resistance signal in the hand electric signal output by the data glove 1100 is between [50 and 250], and in order to improve the training convergence speed of the model, the value range of each path of resistance signal in the hand electric signal is scaled to be between [0 and 1] by adopting a linear normalization method in consideration of small data volume.

3) And reducing the sample data in each sample set subjected to linear normalization processing to 3 dimensions by adopting a Principal Component Analysis (PCA), and displaying all the 3-dimensional sample data in a three-dimensional space. The samples in different sample sets can be marked by different colors or shapes, so that whether obvious boundaries exist between the samples in the different sample sets can be clearly judged in a three-dimensional space. If there is a significant boundary between the samples in the different sample sets, as shown in fig. 5, the normalized sample set corresponding to each preset hand gesture is used as the hand gesture signal sample set corresponding to each preset hand gesture. If the samples of the different sample sets do not have distinct boundaries, the data preprocessing step needs to be performed anew.

And S232, constructing training data, verification data and test data based on the hand gesture signal sample set corresponding to each preset hand gesture.

In this embodiment, the training data and the verification data are used to train and optimize parameters of the first base model to obtain the second base model. The test data is used for testing the preset performance index of the second basic model to obtain the index value of the preset performance index of the second basic model.

In one embodiment, 20% of samples are extracted from the hand gesture signal sample set corresponding to each preset hand gesture to construct test data. Of the remaining 80% of the samples in each sample set, 20% of the samples are extracted to construct a first subset, and the step of constructing the first subset is repeated until 5 subsets are obtained. 5-fold cross validation is adopted, each subset is respectively taken as validation data, and the rest 4 subsets are taken as training data, so that 5 groups of training data and validation data are generated. The parameters of the first base model are trained and optimized based on the generated 5 sets of training data and validation data, respectively.

S233, at least two first basic models corresponding to the first model are obtained.

In this embodiment, the first base model is a machine learning model that can be used to perform the classification. For example, the regression model may be logistic regression, Naviebayes, SVM, Decision Tree, or Random Forest.

And S234, training the model parameters of the first basic models respectively by adopting the training data and the verification data to obtain second basic models corresponding to the first basic models.

And S235, testing the preset performance indexes of the second basic models by adopting the test data to obtain the index values of the preset performance indexes corresponding to the second basic models.

In this embodiment, the preset performance index may be an average value of the accuracy of the prediction result, may be a convergence rate of the model training, and may also be another index value reflecting the performance of the model, which is not specifically limited herein.

And S236, taking the second basic model with the index value meeting the preset condition as the first model.

In one embodiment, the predetermined performance metric includes a mean value of the accuracy of the predicted results.

In this embodiment, the second base model with the largest average of the accuracy of the prediction results may be selected as the first model.

In one embodiment, a logistic regression model Logistic regression, a naive Bayesian model Naviebays, a support vector machine model SVM, a Decision Tree model Decision Tree and a Random Forest model Random Forest are selected, and 5 first basic models are calculated.

In this embodiment, training and optimizing the model parameters of the 5 first basic models are performed by using the training data and the verification data constructed in step S232, so as to obtain 5 second basic models. The test data constructed in S232 are used to test the 5 second basic models respectively, and the test result of the accuracy of the prediction result of each model is shown in fig. 6. In fig. 6, dots represent the mean value of the accuracy of the prediction result, and the I-shaped mark represents the variance of the accuracy of the prediction result, so that it can be seen that the mean value of the accuracy of the preset result corresponding to the second base model based on Random Forest is the largest, so that the second base model based on Random Forest is selected as the final first model.

After training is complete, the final first model is saved to models/[ model name ]/YYYY _ MM _ DD _ hh _ MM _ ss.pkl by python's json.

And S240, obtaining an arm posture recognition result corresponding to the arm posture signal according to the second model. Wherein the second model reflects a second mapping relationship between the arm pose signal and the arm pose recognition result.

In one embodiment, the second model is a data lookup table embodying the second mapping. The data lookup table may be as shown in table 2.

Table 2:

X1	Y1	Z1	X2	gesture
						1	1	2	1	The lower arm and the upper arm are horizontally arranged on the ground with the palm center facing downwards
1	0	1	1	Level ground of big arm of forearm, eversion of forearm
					1	2	1	1	Big arm of forearm horizontal ground, forearm turned inwards
1	1	0	1	The lower arm and the upper arm are horizontal to the ground with the palm upward
					2	1	1	2	The big arm of the small arm is vertically upward
0	1	1	0	The big arm of the small arm is vertically downward
					1	1	0	0	The big arm is vertical to the ground, the small arm is horizontal to the ground, and the palm is upward
1	1	2	0	The big arm is vertical to the ground, the small arm is horizontal to the ground, and the palm is downward
					1	0	1	0	Big arm vertical ground, small arm horizontal ground, small arm eversion
2	1	1	1	The big arm is horizontally arranged on the ground, and the small arm is vertically arranged on the ground

In table 2, the large arm inertial navigation state is relatively small, and only x-axis data is used for classification. X1, Y1 and Z1 represent values of gravity acceleration signals of X, Y and Z axes of the forearm inertial sensor, and X2 represents a value of a gravity acceleration signal of X axis of the large arm inertial sensor.

And S250, obtaining a gesture recognition result of the operator according to the hand gesture recognition result and the arm gesture recognition result.

Specifically, gesture recognition, hand gesture recognition and arm gesture recognition are independent of each other, i.e., a person can make a hand gesture and an arm gesture at the same time. If there is a hand gesture of N_fingerArm posture has N_imIn theory, the combination of the two can have N ═ N_finger×N_imuA different gesture. The final gesture recognition result is obtained through fusion of the arm gesture recognition result and the hand gesture recognition result, the types of recognizable gestures can be increased, and the gesture expandability is improved.

In one embodiment, the hand gesture recognition result is 5 fingers open, the arm gesture recognition result is upper arm vertical ground, lower arm horizontal ground, and palm down, and the gesture recognition result of the operator is upper arm vertical ground, lower arm horizontal ground, 5 fingers open, and palm down.

And S260, executing corresponding operation according to the gesture recognition result.

In this embodiment, a control instruction may be generated according to the gesture recognition result, where the control instruction is used to control the intelligent terminal 1500 to execute a corresponding operation.

In one embodiment, the smart terminal 1500 may be a drone. Under the condition of obtaining the gesture recognition result, the core processor 1400 may generate a control instruction for controlling the drone according to the gesture recognition result, and control the drone to execute a corresponding operation through the control instruction.

For example, the obtained gesture recognition result is: big arm perpendicular ground, forearm level ground, 5 indicate to open, under the decurrent condition of palm, core processor 1400 generates the instruction of control unmanned aerial vehicle descending according to this gesture recognition result, and unmanned aerial vehicle carries out the descending operation after receiving this descending instruction.

The steps S210 and S220 may be performed simultaneously or in a predetermined order, and are not limited in detail. The steps S230 and S240 may be performed simultaneously or in a predetermined order, and are not limited in detail.

< example >

FIG. 7 is a flowchart illustration of an example of a gesture recognition method according to an embodiment.

As shown in fig. 7, the method may include steps S701 to S000.

In this example, the gesture made by the operator is a gesture to control the drone to land.

And S701, acquiring the hand electric signal meeting the first stable condition.

The hand electric signal comprises a plurality of paths of resistance numbers which respectively reflect the bending degree of each finger of an operator. The first stable condition includes that the current sampled hand electrical signal is the same as the hand electrical signal obtained from the previous 19 samples.

S702, normalizing the hand electric signals meeting the first stable condition to obtain hand posture signals.

And S703, acquiring a hand gesture recognition result corresponding to the hand gesture signal according to the first model.

The first model reflects a first mapping relation between the hand gesture signal and the hand gesture recognition result. The hand gesture recognition result is that five fingers are open.

And S704, acquiring a gravity acceleration signal meeting a second stable condition.

In this embodiment, the gravitational acceleration signal includes a first gravitational acceleration signal reflecting the posture of the large arm of the operator and a second gravitational acceleration signal reflecting the posture of the small arm of the operator.

In this embodiment, the second stable condition includes: the total deviation amount of the gravity acceleration signal of the current acquisition period and the gravity acceleration signals of the previous 4 sampling periods is 0. The deviation amount of the gravity acceleration signal in the current sampling period is equal to the sum of squares of derivatives of the gravity acceleration signal in three axes in the first gravity acceleration signal and the gravity acceleration signal in three axes in the second gravity acceleration signal with respect to time. The total deviation amount can be expressed as:

And S705, obtaining an arm posture signal according to the first gravity acceleration signal and the second gravity acceleration signal.

The method specifically comprises the following steps: and respectively mapping the three-axis gravity acceleration signal values of the first gravity acceleration signal into three reference quantities of 0,1 and 2 through a preset data mapping step to obtain the mapped first gravity acceleration signal. And respectively mapping the triaxial gravity acceleration signal values of the second gravity acceleration signal into three reference quantities of 0,1 and 2 by referring to the data mapping step of the first gravity acceleration signal to obtain a mapped second gravity acceleration signal. And obtaining an arm posture signal according to the mapped first gravity acceleration signal and the mapped second gravity acceleration signal. The arm attitude signal is: [1,1,2,0].

S706, according to the data comparison table showing the second mapping relation, the arm posture recognition result corresponding to the arm posture signal is obtained. And the second mapping relation is the mapping relation between the arm posture signal and the arm posture recognition result.

In this embodiment, the data lookup table embodying the second mapping relationship is as table 2 above. The data comparison table is inquired to obtain the arm posture recognition results corresponding to the arm posture signals [1, 1, 2, 0] that the upper arm is vertical to the ground, the lower arm is horizontal to the ground and the palm is downward.

And S707, obtaining a gesture recognition result of the operator according to the hand gesture recognition result and the arm gesture recognition result.

In this embodiment, when the hand gesture recognition result is that the five fingers are open, and the arm gesture recognition result is that the upper arm is vertical to the ground, the lower arm is horizontal to the ground, and the palm is downward, the gesture recognition result of the operator is obtained as follows: the big arm is vertical to the ground, the small arm is horizontal to the ground, the five fingers are open, and the palm is downward.

And S708, controlling the unmanned aerial vehicle to execute corresponding operation according to the gesture recognition result.

In this embodiment, an instruction for controlling the unmanned aerial vehicle to land is generated according to the gesture recognition results that the large arm is vertical to the ground, the small arm is horizontal to the ground, the five fingers are open, and the palm is downward. And responding to the instruction by the unmanned aerial vehicle, and executing landing operation.

< apparatus embodiment >

FIG. 8 is a functional block diagram of a gesture recognition device according to one embodiment.

As shown in fig. 8, the gesture recognition apparatus 8000 may include:

a first obtaining module 8100, configured to obtain a hand posture signal reflecting a degree of finger curvature of a hand of an operator;

a second obtaining module 8200, configured to obtain an arm posture signal that reflects a gravitational acceleration of an arm of an operator;

a hand gesture recognition module 8300 for obtaining a hand gesture recognition result corresponding to the hand gesture signal according to the first model; the first model reflects a first mapping relation between the hand gesture signal and the hand gesture recognition result;

an arm gesture recognition module 8400 for obtaining an arm gesture recognition result corresponding to the arm gesture signal according to the second model; the second model reflects a second mapping relation between the arm posture signal and the arm posture recognition result;

a gesture recognition module 8500 for obtaining a gesture recognition result of the operator according to the hand gesture recognition result and the arm gesture recognition result;

and an executing module 8600, configured to execute a corresponding operation according to the gesture recognition result.

In one embodiment, the first obtaining module 8100 is specifically configured to obtain a hand electrical signal that satisfies a first stability condition; the hand electric signals comprise multi-channel electric signals which respectively reflect the bending degree of each finger of an operator, the first stable condition comprises that the deviation amount between the currently sampled hand electric signals and the previously sampled hand electric signals for N times is within a first set range, and N is greater than or equal to 1; and obtaining a hand posture signal according to the hand electric signal meeting the first stable condition.

In an embodiment, the first obtaining module 8100 is specifically configured to perform normalization processing on the hand electrical signals meeting the first stability condition to obtain the hand posture signals when the hand posture signals are obtained according to the hand electrical signals meeting the first stability condition.

In one embodiment, the second obtaining module 8200 is specifically configured to obtain a gravitational acceleration signal reflecting an arm posture of an operator; wherein the gravity acceleration signal comprises a first gravity acceleration signal reflecting the posture of the big arm and a second gravity acceleration signal reflecting the posture of the small arm of the operator; and obtaining an arm posture signal according to the first gravity acceleration signal and the second gravity acceleration signal.

In one embodiment, the gravitational acceleration signal satisfies a second stability condition; the second stability condition includes: the total deviation amount of the gravity acceleration signal of the current acquisition period and the gravity acceleration signals of the previous M sampling periods is within a second set range, and M is greater than or equal to 1.

In one embodiment, the gravitational acceleration signal comprises gravitational acceleration values for an x-axis, a y-axis, and a z-axis; m is 4; the total deviation is expressed as:

In one embodiment, the hand gesture recognition module 8300 further includes a first model construction module, which is specifically configured to obtain a hand gesture signal sample set corresponding to each preset hand gesture; constructing training data, verification data and test data based on a hand gesture signal sample set corresponding to each preset hand gesture; obtaining at least two first basic models corresponding to the first model; training the model parameters of each first basic model by adopting training data and verification data to obtain a second basic model corresponding to each first basic model; testing the preset performance indexes of the second basic models by adopting the test data to obtain index values of the preset performance indexes corresponding to the second basic models; and taking the second basic model with the index value meeting the preset condition as the first model.

In one embodiment, the second model is a data lookup table embodying the second mapping.

< electronic device embodiment >

FIG. 9 is a schematic diagram of a structure of an electronic device according to one embodiment.

As shown in fig. 9, the electronic device 9000 comprises a memory 9200 and a processor 9100, wherein the memory 9200 is used for storing computer programs, and the processor 9100 is used for controlling the electronic device 9000 to execute the method described in any method embodiment of the present specification under the control of the computer programs.

Each block of the electronic device 9000 can be realized by the processor 9100 executing a computer program stored in the memory 9200 in this embodiment, or can be realized by another circuit configuration, which is not limited herein.

As shown in fig. 10, the electronic apparatus a000 includes the above gesture recognition device 8000.

One or more embodiments of the present description may be a system, method, and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the specification.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations for embodiments of the present description may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), can execute computer-readable program instructions to implement various aspects of the present description by utilizing state information of the computer-readable program instructions to personalize the electronic circuit.

Aspects of the present description are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the description. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present description. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

The foregoing description of the embodiments of the present specification has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the application is defined by the appended claims.

Claims

1. A gesture recognition method, comprising:

2. The method of claim 1, wherein obtaining a hand gesture signal reflecting a degree of finger curvature of the operator's hand comprises:

3. The method of claim 2, wherein deriving the hand gesture signal from the hand electrical signal satisfying the first stability condition comprises:

4. The method of claim 1, wherein said obtaining an arm attitude signal reflecting a gravitational acceleration of the operator's arm comprises:

5. The method of claim 4, wherein the gravitational acceleration signal satisfies a second stability condition; the second stability condition comprises: and the total deviation amount of the gravity acceleration signal of the current acquisition period and the gravity acceleration signals of the previous M sampling periods is within a second set range, wherein M is greater than or equal to 1.

6. The method of claim 5, wherein the gravitational acceleration signal comprises gravitational acceleration values for an x-axis, a y-axis, and a z-axis; said M ═ 4;

the total deviation is expressed as:

7. The method of claim 1, wherein the step of constructing the first model comprises:

obtaining at least two first basic models corresponding to the first model;

8. The method of claim 1, wherein the second model is a data lookup table embodying the second mapping.

9. A gesture recognition apparatus, comprising:

10. An electronic device comprising the gesture recognition apparatus of claim 9; or,

the electronic device comprises a memory for storing a computer program and a processor for controlling the electronic device to perform the method steps according to any of claims 1-8 under the control of the computer program.