CN106200971A - Man-machine interactive system device based on gesture identification and operational approach - Google Patents

Abstract

The invention discloses a human-computer interaction system device and an operation method based on gesture recognition. The system device includes a monocular camera, a PC installed with a human-computer interaction program, a PLC board, and physical application equipment; the operation method includes gesture input. module, fingertip positioning module, gesture tracking module and keyboard and mouse mapping module. The present invention adopts the human-computer interaction scheme based on gesture recognition, solves the limitation problem of the existing human-computer interaction mode that requires a large number of hardware devices to sense, and makes the human-computer interaction more natural and convenient.

Description

Human-computer interaction system device and operation method based on gesture recognition

technical field

The invention relates to a human-computer interaction system device and an operating method, in particular to a gesture recognition-based human-computer interaction system device and an operating method.

Background technique

With the rapid development of science and technology and the increasing popularity of computer vision, people have higher and higher requirements for the naturalness of human-computer communication. All traditional human-computer communication methods, such as mouse, keyboard, microphone, etc., are increasingly unable to meet people's needs. Using human hands as a means of communication between computers and humans is more natural, concise, rich, and direct than other means of communication. Therefore, a computer that can provide gesture recognition will make communication between humans and machines more natural and convenient.

The current gesture recognition system mostly uses the following two types:

(1) Data gloves or wearables: This method can reduce the complexity of detection and recognition algorithms, but the wearable operation method is obviously difficult to meet the needs of natural human-computer interaction;

(2) Based on 3D depth camera: 3D scanning equipment is large in size, high in hardware cost, and requires higher computing power, so it is difficult to integrate and apply to popular smart terminals.

Secondly, the traditional palm positioning algorithm has a flaw: when the hand is in a horizontal state, it cannot accurately locate one of the palm points, which leads to the limitation of gesture recognition.

The invention patent application with the application publication number CN105138136A discloses a "gesture recognition device, gesture recognition method and gesture recognition system". The recognition system includes a gesture recognition device. The gesture recognition device includes at least one A sensor, an input mode judging unit and an input content generating unit. This invention patent application realizes virtual human-computer interaction, which can obtain fast and accurate simulated virtual content, but its device needs to be equipped with sensors for detection and recognition, which requires a lot of hardware and is expensive.

The invention patent application with the application publication number CN104992171A discloses a "a method and system for gesture recognition and human-computer interaction based on 2D video sequences". Recognize the gestures and gestures of human hands in the foreground of 2D camera motion. The patent application for this invention realizes the selection of target personnel in complex backgrounds, and realizes the accurate and high-stability tracking of personnel, but it needs to traverse and extract the joint feature model to match the samples in the sample library. There are many modules and processes, which are relatively complex. It is cumbersome and time-consuming, and there is no specific and precise positioning to the fingertips.

The invention patent application with the application publication number CN105045398A discloses a "virtual reality interactive device based on gesture recognition", which includes a 3D camera interface, a helmet-mounted virtual reality display, a signal processing component, and a mobile device interface. The invention patent application can capture the image sequence of the user's hand containing depth information and process and recognize it to realize virtual reality interaction. However, the user needs to wear a helmet to use it, which is difficult to meet the needs of natural human-computer interaction.

Contents of the invention

The object of the present invention is to overcome the deficiencies in the prior art, and provide a human-computer interaction system device based on gesture recognition. The identification process of the system finally realizes the manipulation of the physical application equipment.

Another object of the present invention is to provide an operation method for applying the above-mentioned human-computer interaction system device based on gesture recognition, propose a solution for palm point calculation, and use the Camshift algorithm to realize the simulation of the keyboard and mouse, and solve the existing problems. The limitations of the human-computer interaction method that needs to be sensed by a large number of hardware devices make the human-computer interaction more natural and convenient, and better solve the problems caused by the ambiguity and diversity of gestures.

The technical scheme that the present invention solves the problems of the technologies described above is:

A human-computer interaction system device and operation method based on gesture recognition, characterized in that:

The system device includes a monocular camera, a PC installed with a man-machine interaction program, a PLC board and physical application equipment. Wherein, the monocular camera is used to capture the gesture image of the user; the PC obtains the gesture image captured by the monocular camera through the USB interface, and the human-computer interaction program will identify, analyze and process the image; The above-mentioned PLC board obtains the gesture information processed by the PC through the serial port or USB; the above-mentioned physical application device obtains the information through the circuit and performs feedback on the information.

The system operation method includes a gesture input module, a fingertip positioning module, a gesture tracking module and a keyboard and mouse mapping module, wherein:

The gesture input module is used to capture user gestures with a monocular camera, and input the captured gesture images into the fingertip positioning module;

The fingertip positioning module is used to locate the fingertip position in the gesture image, and recognize the type of user gesture and the position of the gesture based on this, as the input of the gesture tracking module and the keyboard and mouse mapping module;

The gesture tracking module is used to track the moving trajectory of the user transfer gesture, and obtain the movement amount of the gesture as the input of the keyboard and mouse mapping module;

The keyboard and mouse mapping module is used to recognize the type of user gestures and the amount of gesture movement as corresponding keyboard and mouse operations, and perform computer control accordingly.

The fingertip positioning module in the gesture recognition-based human-computer interaction system operating method of the present invention comprises the following steps:

(1) Camera loading image: call out the camera loading image;

(2) Image preprocessing: perform color space conversion, skin color threshold processing, image denoising processing, image binary processing and open operation processing on the image;

(3) Contour and fingertip search: based on the palm point and fingertip point search on the contour;

(4) Palm location and fingertip filtering: perform palm location calculation through fingertip location, and the location between palm and fingertips plays a role of mutual restriction.

In the fingertip positioning module of the gesture recognition-based human-computer interaction system operating method of the present invention, in step (2), the image preprocessing includes color space conversion, skin color threshold processing, image denoising processing, image binary processing and Open operation processing, in which:

Color space conversion: convert the RGB image to the HSV color model;

Skin color thresholding: using OpenCV's otsu adaptive threshold segmentation;

Image denoising processing: remove the noise around the identified target in the image;

Image binary processing: segment the foreground and background of the image;

Open operation processing: Eliminate disconnected scattered points in the image after binary processing, and fill in missing points.

In the fingertip positioning module of the gesture recognition-based human-computer interaction system operating method of the present invention, in step (3), the contour and fingertip search includes contour search, fingertip point search, palm positioning and fingertip filtering, in:

Contour search: filter unique contours after obtaining multiple contours through four-connected or eight-connected;

Fingertip search: identify the number and position of the current fingertips;

Palm positioning: Calculate the minimum value point to get the palm positioning;

Fingertip Filter: Filter and remove redundant fingertip points.

In the fingertip positioning module of the gesture recognition-based human-computer interaction system operating method of the present invention, in step (4), the palm positioning and fingertip filtering, wherein, for the palm point calculation, two solutions are proposed : Use the principle of minimum value and the principle that the sum of the two sides of a triangle is greater than the third side.

The gesture tracking module of the human-computer interaction system operating method based on gesture recognition of the present invention comprises the following steps:

(1) Tracking area selection: select the area of interest through fingertip positioning;

(2) Camshift tracking hand algorithm processing: mainly through the color information of moving objects in the video to achieve the purpose of tracking;

(3) Tracking center of gravity and area extraction: use the frame difference method to calculate the coordinates of the fingertip and the moving vector of the tracking center of gravity, track the change of the area and the number of points between the fingertip and the palm, and make simple judgments based on this.

Further, the Camshift hand tracking algorithm is processed by combining the fingertip coordinate positioning with the Camshift algorithm, and the Camshift algorithm is improved into an unsupervised learning target tracking method.

Compared with the prior art, the present invention has the following beneficial effects:

1. Recognize the images captured by the 2D camera, and realize the keyboard and mouse simulation through the three modules of fingertip positioning, gesture tracking and mapping, so as to overcome the disadvantages of many hardware devices and high cost required by the existing technical solutions;

2. Two solutions are proposed to calculate the palm point, which solves the problem that the existing algorithm cannot accurately locate one of the palm points, and improves the accuracy of gesture recognition.

Description of drawings

FIG. 1 is a schematic structural diagram of a specific embodiment of the gesture recognition-based human-computer interaction system device of the present invention.

Fig. 2 is a block diagram of the module structure of the human-computer interaction system in a specific embodiment of the present invention.

Fig. 3 is a working flow chart of the image preprocessing steps of the fingertip positioning module in a specific embodiment of the present invention.

Fig. 4 is a comparison diagram before and after the gesture image is processed by opening operation in a specific embodiment of the present invention.

Fig. 5 is a working flow chart of the fingertip positioning module outline and fingertip finding steps in a specific embodiment of the present invention.

Fig. 6 is a principle comparison diagram of four-connected regions and eight-connected regions.

Figure 7 is a convex hull set, that is, a set of fingertip points.

Figure 8 is a schematic diagram of the point between the palms.

Fig. 9 is a diagram illustrating the principle that the distance from a triangle vertex to two points is greater than the distance from all points on the side to two points.

Fig. 10 is a schematic diagram of tracking area selection.

detailed description

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Referring to Fig. 1, the human-computer interaction system device based on gesture recognition of the present invention is composed of several important components in the figure, wherein:

The user 1 expresses its operation instructions through gestures; the monocular camera 2 runs and captures the gesture image of the user, wherein the monocular camera 2 transmits the captured image to the PC 3; after the PC 3 receives the image, the human-computer interaction program therein recognizes, analyzes and processes the gesture image, wherein the PC passes the processed gesture information through a serial port or a USB interface Send to the described PLC board 4; The described PLC board 4 obtains the information sent by the PC and transmits the information to the described physical application device 5 through the circuit; the described physical application device 5 obtains the information and feeds back the information Implementation, wherein the physical application device 5 described in this embodiment is an air conditioner as an example, but it is not limited thereto.

Referring to Fig. 2, the operation method includes a gesture input module, a fingertip positioning module, a gesture tracking module and a keyboard and mouse mapping module, wherein:

(1) Gesture input module: the monocular camera captures the user's gesture image, and transmits the captured gesture image to the PC, and then inputs the fingertip positioning module;

(2) Fingertip positioning module: Perform a series of preprocessing on the pictures captured by the camera, and then locate by looking for contours and fingertip points, and re-position by palm points. The fingertip positioning module consists of the following parts: camera loading image, image preprocessing, contour and fingertip point finding, palm positioning:

(2.1) Image loading from the camera: Call up the camera to load the image.

(2.2) Image preprocessing: referring to Fig. 3, the preprocessing process includes color space conversion, skin color threshold processing, image denoising processing, image binary processing, and opening operation processing to the image, and the specific description is as follows:

(2.2.1) Color space conversion: In general, pictures are under the RGB color model. However, there is often a high correlation between the RGB three components, and the direct use of these components often cannot achieve the expected effect, so it is necessary to convert the RGB image to the HSV color model. Among them, the values of H, S, and V can be obtained by formulas (2), (3), and (4) respectively.

V=MAX (4)

In the above formula (1), MAX and MIN are the maximum and minimum values of the RGB image respectively, and H, S and V are the H value, S value and V value of the HSV image respectively.

After converting to the HSV color space, the required binary image can be obtained by taking the H value from 0 to 180 for threshold segmentation.

(2.2.2) Skin color thresholding: using OpenCV's otsu adaptive threshold segmentation. The program flow is: calculate the histogram and normalize the histogram; calculate the image gray average avgValue; calculate the zero-order w[i] and first-order moment u[i] of the histogram; calculate and find the largest inter-class variance (between-class variance).

variance[i]=(avgValue*w[i]-u[i])*(avgValue*w[i]-u[i])/(w[i]*(1-w[i])) corresponds to this The gray value of the maximum variance is the threshold to be found.

(2.2.3) Image denoising processing: In reality, digital images are often affected by imaging equipment and external environmental noise interference during digitization and transmission, so image denoising processing is required. In this embodiment, the cluster area threshold method is used for image filtering and denoising, and the noise around the target object in the image is removed. The process is as follows:

The connected component extraction algorithm in binary mathematical morphology is used to calculate the area of the clusters. The clusters smaller than the threshold are noise, and the gray value of the pixels of the clusters is set to 255 to remove the noise.

(2.2.4) Image binary processing: perform image binarization, and segment the foreground and background of the image. Image binarization refers to setting the gray value of pixels on the image to 0 or 255, so that the entire image presents an obvious black and white effect. Image binarization is the most common and important processing method in image analysis and processing, which greatly reduces the amount of data in the image, thereby highlighting the outline of the target. In OpenCV, the key function cvThreshold() can be used to realize the binarization of the image.

(2.2.5) Opening operation processing: In order to eliminate the disconnected scattered points after binarization and fill the missing points in the hand at the same time, so as to achieve a better image effect, use the opening operation in the morphological method, that is, first corrode and then swell. Let f(x, y) be the input image, b(x, y) be the structural element, and use the structural element b to corrode and dilate the input image f, respectively defined as:

(f⊙b)(s, t)=min{f(s+x, t+y)+b(x, y)|(s+x, t+y)∈D _f , (x, y)∈ D _b } (5)

Among them, s, t are the parameters of the input image f, x, y are the parameters of the structural element b, D _f is a set of image f, and D _b is a set of structural element b.

Referring to FIG. 3 , it is a comparison diagram of gesture images processed by opening operation.

(2.3) Contour and fingertip point search: perform contour search on the preprocessed image, and perform a series of steps based on the contour to find the fingertip point. Referring to Figure 5, the contour and fingertip search process includes contour search, fingertip search, palm positioning, and fingertip filtering. The specific description is as follows:

(2.3.1) Contour search: A contour generally corresponds to a series of points, that is, a curve in the image, and the boundary is traced by sequentially finding edge points. Since the pixel values in each region are the same, the contour search can be performed through four-connected or eight-connected regions. Four-connectivity and eight-connectivity can mark the connected parts in the binary image, and the syntax is realized as L=(BW,n)[L,num]. Among them, BW is the input image; the value of n can be 4 or 8 to indicate the connection of four-connected or eight-connected regions; num is the number of connected regions found; L is the output image matrix, its element value is an integer, the background is marked as 0, the first One connected region is labeled 1, the second 2, and so on.

See Figure 6, the principle comparison diagram of four-connected and eight-connected, the 0 in the figure is the position of the central pixel point is the four-connected or eight-connected area, that is, the four-connected area refers to the four points up, down, left, and right of 0, and the eight-connected also includes There are four positions in the upper left corner, upper right corner, lower left corner and lower right corner, so the eight-connected region contains the four-connected region.

After the contour search is completed, multiple contours will be obtained, and the only contour obtained through the maximum contour definition filter will be used as the gesture contour, which will be used for subsequent fingertip search.

(2.3.2) Fingertip point search: From the point of view of the point set, the fingertip is a convex hull in the outline of the palm, and the convex hull refers to a minimum convex polygon, satisfying that all points in a point set are on the polygon side or Internally, see Figure 7, the polygon surrounded by line segments is the convex hull of the point set {p ₀ , p ₁ ...p ₁₂ }, where p ₀ , p ₁ , ...p ₁₂ all refer to nodes or vertices. Here, the convex hull search is used to locate the fingertips, that is, to identify the number and position of the convex hulls in the gesture, and then the number and position of the current fingertips can be identified.

(2.3.3) Palm location: The palm location is calculated through the fingertip location, and the location between the palm and the fingertip plays a role of mutual restriction. Refer to the schematic diagram of the inter-palm in Fig. 8, A-E in the figure are the corresponding inter-palm points. The palm positioning is obtained by calculating the minimum value point, that is, the judgment is made based on the formula (7) to find the coordinates of each point between the fingertip points and the relative position between the corresponding front and rear coordinate points. The minimum point between the points.

f(x ₀ )≥f(x ₁ )and f(x ₀ )≥f(x ₂ ) (7)

Among them, f(x ₀ ) is the coordinate value of the pixel point in the current vertical direction, f(x ₁ ) and f(x ₂ ) are the predecessor coordinate value and the successor coordinate value respectively, and the predecessor and successor span selected in the code is 3 pixels, The intent is to prevent layout effects of lines formed between pixels. This algorithm has a flaw. When the hand is in a horizontal state, it cannot accurately locate a point between the palms.

Due to the principle that the three sides of the triangle surrounded by a line passing through the opposite side through any angle in the interior of the triangle are not as large as the three sides of the original triangle. Referring to Figure 9, it can be seen that the distance from the triangle vertex to the two points is greater than the distance from all points on the side to the two points, that is, AB+AC>BD+DC, so the solution can be based on this principle to obtain adjacent fingertip points through formula (8) (that is, the convex hull point) between the contour point and the Euclidean distance between the two points and the maximum point, this method can effectively solve the problem that the palm point cannot be detected when the hand is in a horizontal state.

Where ACB is the sum of Euclidean distances from adjacent fingertips to the target, (C _x , C _y ) is the coordinate of the target pixel point C, (A _x , A _y ), (B _x , By _y ) are the adjacent fingertips The coordinates of points A and B. The target point C starts from point A, and when ABC is greater than MAX, it is assigned to MAX, and so on until C reaches point B, and the MAX point finally obtained is the required palm point.

This method does not calculate all the inter-palm points between the fingertips, and sometimes includes the palm or both sides of the wrist, which can be removed by the vector angle formula (11).

p _x =A _x -C _x , p _y =A _y -C _y , q _x =B _x -C _x , q _y =B _y -C _y (10)

p _x , p _y and q _x , q _y represent the vectors starting from point C and pointing to points A and B respectively, so that cosθ can be obtained from the vector angle formula, so it can be defined that when cosθ is less than 0, the point will be removed.

(2.3.4) Fingertip filtering: Get the most important part after the fingertip point. However, there is not only one fingertip point at the fingertip, so the main task is to screen, first of all, redundant fingertip points need to be removed. Use the function deldot() to delete the fingertip points that are close to each other. After processing the close points, remove the non-fingertip points. At this time, you need to use the palm point. Through the feature that the palm point is between the two fingertips, you can find that there is a palm point between adjacent points and the distance from the palm point exceeds a self-defined threshold. When conditions are met, the fingertip point and palm point can be drawn.

(3) Gesture Tracking Module: track the moving trajectory of the user's gesture, and obtain the movement amount of the gesture as the input information of the keyboard and mouse mapping module. Using Camshift improved algorithm for hand tracking does not need to use the mouse to select the tracking area before tracking. The gesture tracking module consists of the following parts: tracking area selection, Camshift algorithm improvement, tracking center of gravity, area extraction:

(3.1) Tracking area selection: select the area of interest by fingertip positioning.

(3.2) Improvement of Camshift tracking hand algorithm: Camshift algorithm, namely "Continuously adaptive Mean-Shift" algorithm, is a motion tracking algorithm. In the usual tracking process, it mainly uses the color information of the moving object in the video to achieve the purpose of tracking. It is an improvement of the Meanshift algorithm.

The Mean-Shift algorithm only processes the points in the local area of the data, and then moves the area after the processing is completed. Unlike Mean-Shift, the Camshift search window is automatically resized. If there is an easy-to-segment distribution (such as hand features that keep tight), this algorithm can automatically adjust the window size according to the size of the hand when it is opened and closed. In tracking applications, it will use the new size calculated from the previous frame as the tracking area for the next frame.

In the Mean-Shift and Camshift tracking algorithms, the region of interest needs to be selected in advance. Generally speaking, a mouse event is triggered to select the tracking range, but this is supervised and does not meet the expectations of this project. Therefore, it is necessary to improve the Camshift algorithm so that it can unsupervisedly select the region of interest. This article mainly obtains this area through fingertip positioning, that is, after the function of fingertip positioning is achieved, a coordinate composed of fingertip coordinates x and y values is selected as the upper left corner point, and the width and height are the smallest of the longest horizontal and vertical distances. value. Referring to Figure 10, point A and point B are respectively the upper left corner point and the lower right corner point of the fingertip, and the coordinate values of the upper left corner point can be retained, and then the difference between the two points is calculated by the formula MIN(|A _x -B _x |, |A _y -B _y |) calculates the span size. Wherein, (A _x , A _y ), (B _x , B _y ) are the coordinates of point A and point B, respectively.

(3.3) Tracking center of gravity and area extraction: mainly for the algorithm design of simulating keyboard and mouse. The patent of the present invention uses the frame difference method to calculate the coordinates of the fingertips and track the movement vector of the center of gravity, track the change of the area and the number of points between the fingertips and the palm, and make a simple judgment.

(4) Mapping module: Integrating the fingertip positioning module and the gesture tracking module, it plays the role of building a hub between the operation under the camera and the simulated keyboard and mouse operation. It includes keyboard emulation and mouse emulation:

(4.1) Keyboard simulation: The parameters used are tracking center of gravity and tracking area. First, the center of gravity movement vector and area change vector are calculated by the frame difference method. In the calculation of simulating the direction key, the time defined in advance is used to calculate the moving speed, and when the speed exceeds the speed, the direction key is judged. Since the vertical direction does more work than the horizontal direction, the priority of the vertical direction key is higher by default. The priority of the space is the lowest. When judging the button, compare the changed area with the area of the previous frame. If the changed area is less than 0.6 times the previous area, it is determined to activate the space bar.

(4.2) Mouse simulation: the number of fingertip points and palm points is used. Like the keyboard simulation, the frame difference is used to calculate the movement vector of the fingertip and the number of points between the fingertip and the palm. Then the moving speed is calculated by the time defined in advance, and when the speed exceeds the speed, the moving vector is judged and assigned to the current mouse coordinates. The number of palm points is to determine whether to move the mouse and whether to click the mouse.

The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above content, and any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention are all Replacement methods that should be equivalent are all included within the protection scope of the present invention.

Claims (6)

1. A human-computer interaction system device and operating method based on gesture recognition, characterized in that, the simulation of keyboard and mouse is realized through four modules and then human-computer interaction is realized, wherein: The system device includes a monocular camera, a PC installed with a human-computer interaction program, a PLC board and physical application equipment, wherein the monocular camera is used to capture the user's gesture image; the PC obtains the image through a USB interface. The gesture image captured by the monocular camera, wherein the human-computer interaction program will identify, analyze and process the image; the PLC board obtains the gesture information processed by the PC through a serial port or USB; the physical application device passes The circuit acquires the information and performs feedback on the information. The operation method includes a gesture input module, a fingertip positioning module, a gesture tracking module and a keyboard and mouse mapping module, wherein: The gesture input module is used to capture user gestures with a monocular camera, and input the captured gesture images into the fingertip positioning module; The fingertip positioning module is used to locate the fingertip position in the gesture image, and recognize the type of user gesture and the position of the gesture based on this, as the input of the gesture tracking module and the keyboard and mouse mapping module; The gesture tracking module is used to track the movement track of the user's gesture, and obtain the movement amount of the gesture as the input of the keyboard and mouse mapping module; The keyboard and mouse mapping module is used to recognize the type of user gestures and the amount of gesture movement as corresponding keyboard and mouse operations, and perform computer control accordingly. 2. The human-computer interaction system device and operating method based on gesture recognition according to claim 1, wherein the fingertip positioning module comprises the following steps: (1) Camera loading image: call out the camera loading image; (2) Image preprocessing: perform color space conversion, skin color threshold processing, image denoising processing, image binary processing and open operation processing on the image; (3) Contour and fingertip search: based on the palm point and fingertip point search on the contour; (4) Palm location and fingertip filtering: perform palm location calculation through fingertip location, and the location between palm and fingertips plays a role of mutual restriction. 3. The human-computer interaction system device and operating method based on gesture recognition according to claim 1, wherein the gesture tracking module combines the Camshift algorithm with the fingertip coordinate positioning to improve the Camshift algorithm into a wireless A target tracking method for supervised learning, wherein the module includes the following steps: (1) Tracking area selection: select the area of interest through fingertip positioning; (2) Camshift tracking hand algorithm processing: mainly through the color information of moving objects in the video to achieve the purpose of tracking; (3) Tracking center of gravity and area extraction: use the frame difference method to calculate the coordinates of the fingertip and the moving vector of the tracking center of gravity, track the change of the area and the number of points between the fingertip and the palm, and make simple judgments based on this. 4. The human-computer interaction system device and operation method based on gesture recognition according to any one of claims 1-2, characterized in that, the fingertip positioning module, wherein, for the palm point calculation, two methods are proposed: Solution: Use the principle of minimum value and the principle that the sum of the two sides of a triangle is greater than the third side. 5. in the human-computer interaction system operating method based on gesture recognition according to claim 2, it is characterized in that, the fingertip positioning module, in step (2), the image preprocessing includes color space conversion, skin color Threshold processing, image denoising processing, image binary processing and opening operation processing, in which: Color space conversion: convert the RGB image to the HSV color model; Skin color thresholding: using OpenCV's otsu adaptive threshold segmentation; Image denoising processing: remove the noise around the identified target in the image; Image binary processing: segment the foreground and background of the image; Open operation processing: Eliminate disconnected scattered points in the image after binary processing, and fill in missing points. 6. in the human-computer interaction system operating method based on gesture recognition according to claim 2, it is characterized in that, described fingertip positioning module, in step (3), described contour and fingertip search comprise contour search, Fingertip point search, palm positioning and fingertip filtering, among which: Contour search: filter unique contours after obtaining multiple contours through four-connected or eight-connected; Fingertip search: identify the number and position of the current fingertips; Palm positioning: Calculate the minimum value point to get the palm positioning; Fingertip Filter: Filter and remove redundant fingertip points.