CN104123008A - Man-machine interaction method and system based on static gestures - Google Patents

Abstract

The invention provides a human-computer interaction method and system based on static gestures. The human-computer interaction method includes a gesture recognition method. The gesture recognition method includes the steps of establishing a real-time skin color model, establishing a gesture geometric model, establishing a tracking model, and identifying. . The beneficial effects of the invention are: the invention can allow people to realize the interaction between man and machine and give instructions to the machine by making gestures. From the aspect of interaction mode, the present invention provides a novel, concise and more humanized human-computer interaction mode. From the level of system implementation method, the gesture recognition unit used in the system effectively overcomes the problems of poor stability of traditional gesture recognition, small amount of instructions, non-expandable, and heavy dependence on the PC platform.

Description

A static gesture-based human-computer interaction method and system

technical field

The invention relates to the field of artificial intelligence, in particular to a static gesture-based human-computer interaction method and system. the

Background technique

Gesture interaction is one of the ideal human-computer interaction methods, and gesture recognition technology is an important technical means of gesture interaction. Gesture recognition technology generally includes four parts: skin color segmentation, gesture region extraction, gesture feature extraction, and gesture feature classification (recognition). In the traditional gesture recognition technology, the realization of these technical modules is different, but there is room for improvement in general. Traditional gesture recognition technology generally has the following disadvantages:

1. In the traditional gesture recognition method, the skin color segmentation part generally uses RGB color conversion to Ycbcr or HSV color space, and then obtains the skin color segmentation module by limiting the threshold. Such a skin color segmentation processing method usually becomes unstable due to the influence of ambient light. Moreover, it cannot handle the interference of skin-like areas in the environment well, and has relatively high requirements for the use environment. the

2. In the gesture area extraction part of the traditional gesture recognition method, the skin color area is directly identified as the gesture area in many cases. Even if the gesture area judgment is added, the largest connected area of skin color is generally used as the gesture area. This weak feature judgment is used in practical applications. It is difficult to distinguish the skin color area of the face from the skin color area of the gesture, which eventually leads to the failure of gesture recognition

3. In the gesture feature extraction part of the traditional gesture recognition method, the convex hull area occupancy ratio is generally used as the key feature of the gesture type. In addition, the number of fingers and the gesture centroid are also often used as gesture type features. These features can provide characteristic data for gesture classification to a certain extent, but when there are many types of gestures, it will show obvious limitations and the misrecognition rate will increase significantly. the

4. In the category of gesture feature classification, traditional gesture recognition methods generally focus on static gesture recognition of a single image, ignoring the continuity of video data, and failing to make full use of the data to further improve the reliability of recognition. the

Contents of the invention

In order to solve the problems in the prior art, the present invention provides a static gesture-based human-computer interaction method. the

The present invention provides a static gesture-based human-computer interaction method, including a gesture recognition method, including:

Steps of establishing a real-time skin color model: extracting skin color blocks from the image;

Steps for establishing a gesture geometric model: extract gesture features from skin color images, and define static gesture instructions;

Steps to establish a tracking model: track the gesture image;

Recognition step: used for gesture command recognition;

Include in described real-time skin color model step:

Initial skin color acquisition step: use strict skin color threshold limit and dynamic frame difference to get initial skin color data block;

Skin color model calculation steps: use the existing skin color database to calculate the multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process;

Skin color judgment step: According to the calculated skin color model, the skin color judgment is performed on the pixel point, and when the probability is greater than the set threshold, the skin color judgment is given, otherwise, the non-skin color judgment is made. the

As a further improvement of the present invention, include in described gesture geometry model step:

Gesture model construction steps: use line segments and circles to geometrically reconstruct gesture hand shapes for all skin color regions;

Judgment step of hand skin color area: Based on the constructed gesture geometric model, judge whether the model satisfies the actual characteristics of the hand. If it is reasonable, make a gesture skin color judgment, otherwise make a non-gesture skin color judgment;

Static gesture instruction entry steps: meet user-defined gesture instructions. the

As a further improvement of the present invention, the tracking model step can complete the tracking of 1 to 2 hands of the user, the tracking model step obtains input data from the real-time skin color model step, obtains the initial tracking window from the gesture geometry model step, and finally Complete the tracking of the specific gesture skin color block. During the tracking process, use the inter-frame information to provide gesture skin color position information for the gesture geometric model step and recognition step. the

As a further improvement of the present invention, in the identifying step, a specific gesture instruction is identified according to the geometric features obtained from the gesture geometric model and the tracking information obtained from the tracking model. the

As a further improvement of the present invention, the human-computer interaction method includes:

Video collection step: collect user gesture data and transmit it to the core processing step;

Core processing steps: analyze the gesture commands in the video through the gesture recognition method, and then issue the gesture commands to the command execution step;

Instruction execution step: Execute the instruction program corresponding to the gesture instruction. the

The present invention also provides a human-computer interaction system based on static gestures, including a gesture recognition unit, including:

Real-time skin color model module: used to extract skin color blocks from images;

Gesture geometric model module: used to extract gesture features from skin color images and define static gesture instructions;

Tracking model module: used to track gesture images;

Recognition module: used for gesture command recognition;

Include in described real-time skin color model module:

Initial skin color acquisition module: use strict skin color threshold limit and dynamic frame difference to obtain initial skin color data block;

Skin color model calculation module: use the existing skin color database to calculate the multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process;

Skin color judgment module: According to the calculated skin color model, the skin color judgment is performed on the pixels. When the probability is greater than the set threshold, the skin color judgment is given, otherwise the non-skin color judgment is made. the

As a further improvement of the present invention, include in described gesture geometry model module:

Gesture model building blocks: for geometrically reconstructing gesture hand shapes using line segments and circles for all skin tone regions;

Judgment module of hand skin color area: it is used to judge whether the model meets the actual characteristics of the hand on the basis of the constructed gesture geometric model. If it is reasonable, it will judge the skin color of the gesture, otherwise it will judge the skin color of the non-gesture;

Static gesture instruction input module: used to meet user-defined gesture instructions. the

As a further improvement of the present invention, the tracking model module can complete the tracking of 1 to 2 hands of the user, the tracking model module obtains input data from the real-time skin color model module, obtains the initial tracking window from the gesture geometry model module, and finally Complete the tracking of the specific gesture skin color block. During the tracking process, use the inter-frame information to provide gesture skin color position information for the gesture geometric model module and recognition module. the

As a further improvement of the present invention, in the recognition module, a specific gesture instruction is recognized according to the geometric features obtained by the gesture geometric model module and the tracking information obtained by the tracking model module. the

As a further improvement of the present invention, the human-computer interaction system includes:

Video acquisition unit: used to collect user gesture data and transmit it to the core processing unit;

Core processing unit: used to analyze the gesture commands in the video through the gesture recognition unit, and then issue the gesture commands to the command execution unit;

Instruction execution unit: used to execute instruction programs corresponding to gesture instructions. the

The beneficial effects of the invention are: the invention can allow people to realize the interaction between man and machine and give instructions to the machine by making gestures. From the aspect of interaction mode, the present invention provides a novel, concise and more humanized human-computer interaction mode. From the perspective of system implementation method, the gesture recognition unit used in the system effectively overcomes the problems of poor stability of traditional gesture recognition, small number of instructions, non-expandable, and heavy dependence on the PC platform. the

Description of drawings

Fig. 1 is a schematic diagram of a static gesture instruction of the present invention;

Fig. 2 is a functional block diagram of the human-computer interaction system of the present invention;

Fig. 3 is a functional block diagram of an embodiment of the human-computer interaction system of the present invention. the

Detailed ways

The invention discloses a static gesture-based human-computer interaction method, including a gesture recognition method, which includes:

Steps of establishing a real-time skin color model: extracting skin color blocks from the image;

Steps for establishing a gesture geometric model: extract gesture features from skin color images, and define static gesture instructions;

Steps to establish a tracking model: track the gesture image;

Recognition step: used for gesture command recognition;

Include in described real-time skin color model step:

Initial skin color acquisition step: use strict skin color threshold limit and dynamic frame difference to get initial skin color data block;

Skin color model calculation steps: use the existing skin color database to calculate the multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process;

Skin color judgment step: According to the calculated skin color model, the skin color judgment is performed on the pixel point, and when the probability is greater than the set threshold, the skin color judgment is given, otherwise, the non-skin color judgment is made. the

As an embodiment of the present invention, in the step of real-time skin color model, it is first necessary to establish a real-time skin color database to provide initial data for the multi-Gaussian probability model. The real-time skin color database is based on frame difference method and strict skin color data model. After completing the real-time skin color database, a multi-Gaussian probability skin color model based on brightness index can be established according to the following principles. The principle of establishing a multi-Gaussian probability skin color model based on brightness index is:

Calculate brightness index: Y＝0.299×r+0.587×g+0.114×b

Compute the eigenvectors:

I ₁ =(r+g+b)/3

I ₂ =rb

I ₃ =(2×grb)/2

I ₄ =0.492×(bY)

I ₅ =0.877×(rY)

Gaussian probability calculation model:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\stackrel{\mathrm{<span\; class="notranslate">\; no</span>}}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; -</span>\frac{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; \}</span>$

Include in described gesture geometry model step:

Gesture model construction steps: use line segments and circles to geometrically reconstruct gesture hand shapes for all skin color regions;

Judgment step of hand skin color area: Based on the constructed gesture geometric model, judge whether the model satisfies the actual characteristics of the hand (such as the intersection point between the fingers and the palm, the proportional relationship between the length of the finger and the radius of the palm, etc.), and if it is reasonable, do Gesture skin color judgment, otherwise non-gesture skin color judgment;

Static gesture instruction entry steps: meet user-defined gesture instructions. the

As an embodiment of the present invention, in the gesture geometric modeling step, it is first necessary to calculate the position of the fingertip through the edge curvature. The fingertip point is the boundary point p _i satisfying the following conditions:

p _i p _ik ×p _i p _ik ≥0

Among them, p _i is the boundary point where the edge is continuous. Ω is the value range of the curvature of the fingertip point satisfying a certain curvature threshold.

After the fingertip point is determined, the position of the center of the palm needs to be determined again. In the present invention, the distance transformation palm center positioning method is adopted. The specific principles are as follows:

Compute the distance image:

$\mathrm{<span\; class="notranslate">\; dis</span>}\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; ce</span>}<span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Get the effective distance image template:

${\mathrm{<span\; class="notranslate">\; mask</span>}}_{\mathrm{<span\; class="notranslate">\; plan</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; dist</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; dist</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}\end{array}\right.$

Calculate the center of mass in the effective distance image template to obtain the center of the palm:

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; mask</span>}}_{\mathrm{<span\; class="notranslate">\; plan</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; i</span>}}{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; mask</span>}}_{\mathrm{<span\; class="notranslate">\; plan</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; mask</span>}}_{\mathrm{<span\; class="notranslate">\; plan</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; mask</span>}}_{\mathrm{<span\; class="notranslate">\; plan</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}$

Through the calculation of fingertip points and palm points and the intersection principle of circles and straight lines, circles are used to simulate the palm area, and straight lines are used to simulate fingers, thereby establishing a gesture geometric model. the

The tracking model step can complete the tracking of 1 to 2 hands of the user, the tracking model step obtains input data from the real-time skin color model step, obtains the initial tracking window from the gesture geometric model step, and finally completes the tracking of the specific gesture skin color block Tracking, in the tracking process, use the inter-frame information to provide gesture skin color position information for the gesture geometric model step and recognition step. the

As an embodiment of the present invention, in the step of tracking the model, the Camshift algorithm is used to complete the tracking of the gesture image. The Camshift algorithm requires an initial search window. In the present invention, the palm area in the gesture geometric model can be directly used as the initial search window, and the determination of the search window in the subsequent tracking process can be obtained according to the following process. the

Calculate the zero step distance

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

Calculate the first moment

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; xI</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

Calculate search window centroid

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}$

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}$

Adjust the size of the search window

Width: $\mathrm{the\; s}=\sqrt{m_{00}/256}$

Length: 1.2s

Through the continuous iteration of the search window, the gesture image tracking is finally realized. the

In the recognition step, a specific gesture command is identified according to the geometric features obtained from the gesture geometric model and the tracking information obtained from the tracking model; specifically, the gesture geometric model feature parameters in each frame of image are obtained from the gesture geometric model, and then synthesized The continuous information between frames obtained by tracking is used for specific gesture judgment, that is, gesture recognition. the

The human-computer interaction method includes:

Video collection step: collect user gesture data and transmit it to the core processing step;

Core processing steps: analyze the gesture commands in the video through the gesture recognition method, and then issue the gesture commands to the command execution step;

Instruction execution step: Execute the instruction program corresponding to the gesture instruction. the

As shown in Figures 1 to 3, the present invention also discloses a human-computer interaction system based on static gestures, including a gesture recognition unit, which includes:

Real-time skin color model module: used to extract skin color blocks from images;

Gesture geometric model module: used to extract gesture features from skin color images and define static gesture instructions;

Tracking model module: used to track gesture images;

Recognition module: used for gesture command recognition;

Include in described real-time skin color model module:

Initial skin color acquisition module: use strict skin color threshold limit and dynamic frame difference to obtain initial skin color data block;

Skin color model calculation module: use the existing skin color database to calculate the multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process;

Skin color judgment module: According to the calculated skin color model, the skin color judgment is performed on the pixels. When the probability is greater than the set threshold, the skin color judgment is given, otherwise the non-skin color judgment is made. the

Include in described gesture geometry model module:

Gesture model building blocks: for geometrically reconstructing gesture hand shapes using line segments and circles for all skin tone regions;

Judgment module of hand skin color area: it is used to judge whether the model satisfies the actual characteristics of the hand (such as the intersection of fingers and palm, the proportional relationship between finger length and palm radius, etc.) on the basis of the constructed gesture geometric model, if reasonable Then make a gesture skin color judgment, otherwise make a non-gesture skin color judgment;

Static gesture instruction input module: used to meet user-defined gesture instructions. the

The tracking model module can complete the tracking of 1 to 2 hands of the user. The tracking model module obtains input data from the real-time skin color model module, obtains the initial tracking window from the gesture geometry model module, and finally completes the tracking of the specific gesture skin color block. Tracking, during the tracking process, the inter-frame information is used to provide gesture skin color position information for the gesture geometric model module and recognition module, which further optimizes the calculation of the geometric model. the

In the identification module, a specific gesture command is identified according to the geometric features obtained by the gesture geometric model module and the tracking information obtained by the tracking model module. the

The human-computer interaction system includes:

Video acquisition unit: used to collect user gesture data and transmit it to the core processing unit;

Core processing unit: used to analyze the gesture commands in the video through the gesture recognition unit, and then issue the gesture commands to the command execution unit;

Instruction execution unit: used to execute instruction programs corresponding to gesture instructions. the

Among them, the tracking model module, the real-time skin color model module, and the gesture geometry model module have a relationship of mutual correction of data. Recognition obtains hand shape feature data from the gesture geometric model module, and combines the inter-frame continuous information obtained by the tracking model to make specific gesture judgments. the

When the user makes a specific gesture instruction within the visual range of the video acquisition unit of the device, the video acquisition unit obtains video data through the video acquisition module, and hands it over to the gesture recognition unit on the core processing unit. The processing flow is as follows: The skin color data block is obtained, and the gesture skin color data is obtained from the skin color data block according to the reliability of the gesture geometric model. At the same time, the tracking model module also obtains the corresponding data from the skin color segmentation, and then determines the gesture skin color database according to the geometric model to track specific gesture commands. . The final processing data is aggregated to the gesture command recognizer for specific gesture command recognition. the

The present invention is widely used, for example:

Solution 1: Correspond the specific gesture command with the robot action command, for example, when the specific gesture command 1 is made, it corresponds to the motion command of the robot to move forward. The user makes a gesture command 1 within the vision range of the robot, and the video acquisition unit of the robot obtains video data with a specific gesture command 1, and after it is transmitted to the core processing unit, the acquired data is used as the input data of the gesture recognition program, and the program outputs a specific gesture order 1. Then, according to the previously defined command correspondence, the robot gets the command to move forward and starts to move forward. the

Solution 2: Correspond the specific command with the PPT display command on the PC, for example, when the specific gesture 1 is made, the corresponding PPT is switched to the next page. The user makes a gesture command 1 within the visual range of the PC camera, and the PC video acquisition unit obtains video data with a specific gesture command 1, and after transmitting it to the core processing unit, the acquired data is used as the input data of the gesture recognition program, and the program outputs Specific gesture commands1. Then, according to the previously defined command correspondence, the PPT application switches the currently displayed PPT page to the next page. the

Solution 3: Corresponding specific commands to commands on the smart TV platform, for example, when a specific gesture 1 is made, it corresponds to switching of TV channels. The user makes a gesture instruction 1 within the visual range of the camera of the smart TV, and the video acquisition unit of the smart TV obtains video data with a specific gesture instruction 1, and after transmitting it to the core processing unit, the acquired data is used as the input data of the gesture recognition program , the program outputs a specific gesture command 1. Then, according to the previously defined command correspondence, the TV channel is switched. the

On the one hand, the present invention improves some defects of the existing gesture recognition technical solutions, on the other hand, it applies the gesture recognition method to specific practical application scenarios, and provides a more convenient and effective interaction mode and instruction issuing method for human-machine interaction . Compared with traditional gesture recognition methods, the present invention provides a more robust recognition model. It can effectively reduce the influence of light on the skin color threshold that is difficult to determine. In terms of instruction set selection, a user-defined instruction interface is provided. If necessary, users can customize gesture instructions within a reasonable range according to their needs. On the other hand, compared with the traditional human-machine interaction method, the present invention does not require an additional control terminal, and only needs to use bare hands to make corresponding gesture commands within the scope of machine vision to realize the interaction with the machine and issue instructions. the

The invention allows people to realize the interaction between man and machine and give instructions to the machine by making gestures. From the aspect of interaction mode, the present invention provides a novel, concise and more humanized human-computer interaction mode. From the perspective of system implementation method, the gesture recognition unit used in the system effectively overcomes the problems of poor stability of traditional gesture recognition, small number of instructions, non-expandable, and heavy dependence on the PC platform. the

The present invention also has beneficial effects such as:

1. Effectively improve the interactive experience between humans and machines. It provides a more humanized experience for the way humans and machines interact. In the case of using the present invention, people can realize the interaction with the machine and issue instructions without being separated from an additional control terminal such as a remote controller. the

2. The real-time skin color database processing scheme adopted in the present invention can effectively overcome the influence brought by the change of light intensity. By establishing a multi-Gaussian probability model, the judgment of skin color data is effectively realized, and the method is applied to complex scenes, which effectively reduces the misjudgment of similar skin color data. the

3. By establishing a gesture geometric model, it is possible to effectively judge the gesture skin color area and the non-gesture skin color area. Reduce the interference of multi-skinned areas in the application environment. the

4. The introduction of gesture tracking model can effectively enhance the stability of gesture recognition results. the

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention. the

Claims (10)

1. A human-computer interaction method based on static gestures, characterized in that, comprising a gesture recognition method, including in the gesture recognition method: Establishing a real-time skin color model step: extracting skin color blocks from the image; Steps for establishing a gesture geometric model: extract gesture features from skin color images, and define static gesture instructions; Steps to establish a tracking model: track the gesture image; Recognition step: used for gesture instruction recognition; Include in described real-time skin color model step: Initial skin color acquisition step: use strict skin color threshold limit and dynamic frame difference to obtain initial skin color data block; Skin color model calculation steps: use the existing skin color database to calculate a multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process; Skin color judgment step: According to the calculated skin color model, the skin color judgment is performed on the pixel point, and when the probability is greater than the set threshold, the skin color judgment is given, otherwise, the non-skin color judgment is made. 2. human-computer interaction method according to claim 1, is characterized in that, comprises in described gesture geometry model step: Gesture model construction step: use line segments and circles to geometrically reconstruct gesture hand shapes for all skin color areas; hand skin color area judgment step: judge whether the model satisfies the actual characteristics of the hand on the basis of the constructed gesture geometric model, if If it is reasonable, make a gesture skin color judgment, otherwise make a non-gesture skin color judgment; Static gesture instruction entry steps: meet user-defined gesture instructions. 3. The human-computer interaction method according to claim 1, wherein the tracking model step can complete the tracking of 1 to 2 hands of the user, and the tracking model step obtains input data from the real-time skin color model step, The initial tracking window is obtained from the gesture geometric model step, and the tracking of the specific gesture skin color block is finally completed. During the tracking process, the inter-frame information is used to provide gesture skin color position information for the gesture geometric model step and the recognition step. 4. The human-computer interaction method according to claim 1, characterized in that, in the identifying step, the specific gesture instruction is identified according to the geometric features obtained from the gesture geometric model and the tracking information obtained from the tracking model. 5. The human-computer interaction method according to any one of claims 1 to 4, characterized in that the human-computer interaction method comprises: Video collection step: collect user gesture data and transmit it to the core processing step; Core processing steps: analyze the gesture commands in the video through the gesture recognition method, and then issue the gesture commands to the command execution step; Instruction execution step: Execute the instruction program corresponding to the gesture instruction. 6. A human-computer interaction system based on static gestures, characterized in that it includes a gesture recognition unit, including in the gesture recognition unit: Real-time skin color model module: used to extract skin color blocks from images; Gesture geometric model module: used to extract gesture features from skin color images and define static gesture instructions; Tracking model module: used to track gesture images; Recognition module: used for gesture command recognition; Include in described real-time skin color model module: Initial skin color acquisition module: use strict skin color threshold limit and dynamic frame difference to obtain initial skin color data block; Skin color model calculation module: use the existing skin color database to calculate the multi-Gaussian skin color model based on brightness index, and update the model parameters in real time according to the currently obtained skin color image during the gesture recognition process; Skin color judgment module: According to the calculated skin color model, the skin color judgment is performed on the pixels. When the probability is greater than the set threshold, the skin color judgment is given, otherwise the non-skin color judgment is made. 7. human-computer interaction system according to claim 6, is characterized in that, comprises in described gesture geometry model module: Gesture model building block: for geometrically reconstructing gesture hand shapes using line segments and circles for all skin tone regions; Judgment module of hand skin color area: it is used to judge whether the model meets the actual characteristics of the hand on the basis of the constructed gesture geometric model. If it is reasonable, it will judge the skin color of the gesture, otherwise it will judge the skin color of the non-gesture; Static gesture instruction input module: used to meet user-defined gesture instructions. 8. The human-computer interaction system according to claim 6, wherein the tracking model module can complete the tracking of 1 to 2 hands of the user, and the tracking model module obtains input data from the real-time skin color model module, The initial tracking window is obtained from the gesture geometric model module, and the tracking of the specific gesture skin color block is finally completed. During the tracking process, the inter-frame information is used to provide gesture skin color position information for the gesture geometric model module and the recognition module. 9 . The human-computer interaction system according to claim 6 , wherein, in the recognition module, the specific gesture instruction is recognized according to the geometric features obtained by the gesture geometric model module and the tracking information obtained by the tracking model module. 10. The human-computer interaction system according to any one of claims 6 to 9, wherein the human-computer interaction system comprises: Video acquisition unit: used to collect user gesture data and transmit it to the core processing unit; Core processing unit: used to analyze the gesture commands in the video through the gesture recognition unit, and then issue the gesture commands to the command execution unit; Instruction execution unit: used to execute instruction programs corresponding to gesture instructions.