TWI623889B - 3d hand gesture image recognition method and system thereof - Google Patents

Abstract

A three-dimensional gesture image recognition system includes a light field photographing unit, a calculation unit and an output unit. A light gesture is taken by the light field photographing unit to obtain a 3D gesture image. The calculation unit is connected to the light field photography unit, and then the 3D gesture image is projected into a predetermined recognition space to obtain a feature vector, and then the feature vector and the plurality of sample feature vectors are compared and classified to classify the 3D gesture image to identify the type of the 3D gesture image. The output unit is connected to the calculation unit to output the type of the 3D gesture image.

Description

Three-dimensional gesture image recognition method and system thereof

The invention relates to a three-dimensional gesture image recognition method and a system thereof; in particular, to a three-dimensional gesture image recognition method and a system thereof using various light field capturing.

A related application device for recognizing gesture image recognition, for example, the Republic of China Patent Publication No. M382675, a new type of surveillance camera control device based on gesture recognition, which discloses that a camera lens can be turned up and down, left and right, and zoomed in. And remote control and other action command control devices. Using the gesture image of the user captured by the gesture camera to recognize the movement of the user's gestures up, down, left, right, front, and back, and to send the camera up, down, left, and right to the surveillance camera. To zoom in and out, without having to manipulate the mouse or joystick to release the above control message.

The aforementioned M382675 first uses a gesture camera to capture the user's hand image, and then uses the gesture displacement detection module to detect and recognize the user's hand position and trend, thereby calculating the direction of the camera to be controlled, and then transmitting The signal control monitors the lens of the camera to perform actions of going up, down, left, right, zooming in and out. The gesture displacement detecting module first calculates the position and area of the hand in the gesture camera, and serves as the origin of the gesture recognition. This part can also preset a gesture recognition on the screen of the gesture camera. The origin is used as the displacement detection reference point for subsequent gesture recognition, and defines the threshold value of the minimum area of a gesture, the threshold value of the minimum moving distance of a gesture, and the minimum area of a gesture. The threshold for change.

In the foregoing No. M382675, after the size of the gesture area of the user is greater than a preset minimum threshold value of the gesture, the device starts to activate the gesture recognition function, and uses the gesture displacement detection module to detect the displacement of the current gesture position and the origin of the gesture. the amount. When the displacement of the gesture movement is greater than a preset threshold value, the gesture movement is considered to occur. Conversely, when the displacement amount of the gesture movement is less than the threshold value, no gesture movement occurs; and the gesture displacement detection is performed. The module also detects the current gesture position and the direction of displacement of the gesture origin. If the gesture moves up, down, left or right, a control signal is triggered to drive the camera lens to move up, down, left or right. In addition, the gesture displacement detection module uses image recognition technology to determine the movement of the gesture in the range of motion; when the gesture moves forward, the area thereof is larger than the area threshold, and a control signal is triggered to drive the surveillance camera lens to zoom in. When the gesture moves backwards, its area will be smaller than the area threshold, triggering a control signal to drive the camera lens to zoom out.

Another application device for recognizing gesture image recognition, for example, the gesture recognition system of the Republic of China Patent No. I298461 and its method and invention patent, which discloses a gesture recognition system and a method thereof applied to an image capture device Notebook computer. The user can directly align the image capture device with a preset gesture, and the notebook computer will execute the corresponding application or function option of the gesture action to increase the user execution application or function options. The convenience of time.

Another application device for recognizing gesture image recognition, for example, the gesture recognition system of the Republic of China Patent Publication No. 201030630 and the method thereof, and a method for identifying a gesture recognition system comprising: a photographing device for obtaining a possible natural gesture An image is used to find the skin contour of the skin color portion from the image, and then classify the skin color contour into a plurality of contour fragments of different angles; an arithmetic engine has a plurality of parallel computing units and a plurality of different angles; Category of gesture template library, the number The parallel computing units respectively find the gesture template that is closest to the contour fragments in the gesture template library of different angle categories; an optimal template selection method, which is free from the several approximate gesture templates found by the plurality of parallel operation units. Selecting an optimal gesture template; and displaying an image of the best gesture template by the display terminal; thereby achieving the purpose of not recognizing the marker without using any marker.

Another conventional gesture image recognition related method and system, for example, the image-based motion gesture recognition method and system invention patent of the Republic of China Patent No. I431538, which discloses an image-based motion gesture recognition method and system. The method includes the steps of: receiving a plurality of image frames; performing a gesture detection according to the plurality of image images to obtain a first gesture, and if the first gesture conforms to a preset start gesture, according to the plurality of image frames The part position performs a movement tracking to obtain a movement gesture; during the execution of the movement tracking, performing gesture detection according to the plurality of image images to obtain a second gesture, if the second gesture meets a preset end Gesture, stop moving tracking.

Another conventional method and system for recognizing gesture image recognition, for example, the inventor proposes a gesture image recognition method using singular value decomposition to deal with complex backgrounds and a system for inventing the patent of the Republic of China Patent No. I444907, which discloses a singular value decomposition Handling image recognition methods for complex backgrounds. The method comprises: decomposing an original gesture image by using a singular value decomposition method to obtain a gain gesture image; removing a dark background from the gain gesture image to obtain at least one type of skin pixel block; and utilizing a skin color detection method The method performs skin color detection on the skin pixel block to remove the remaining background of the skin pixel block. The gesture image recognition system comprises an input unit, a calculation unit and an output unit. The input unit is configured to input the original gesture image, the calculation unit is configured to remove the dark background and the remaining background from the gain gesture image, and the output unit is configured to output a skin motion gesture image.

Another conventional method and system for recognizing gesture image recognition, for example, the inventor proposes a method for recognizing a gesture image using image direction alignment and a system for inventing the invention after the patent of the Republic of China Patent No. I444908, which discloses a method of correcting image orientation Handling gesture image recognition method. The method includes: inputting a skin color gesture image; calculating a global heart in the skin color gesture image; selecting a region of interest in the skin color gesture image; selecting a sub region using the region of interest; and calculating a subregion in the subregion Regional nature; use this global nature and regional nature to calculate a pair of positive angles. The gesture image recognition system comprises an input unit, a calculation unit and an output unit. The input unit is configured to input the skin color gesture image, and the calculation unit selects the region of interest and the sub-region in the skin color gesture image to calculate the global nature heart and the region property core, and then calculate the alignment angle, and the output is The unit is used to output the pair of positive angles.

Another conventional method and system for recognizing gesture image recognition, for example, the inventor proposes a method for recognizing a gesture image using singular value decomposition for gamma compensation, and a system for inventing the invention, which discloses a singular value The gesture image recognition method for decomposing the light compensation processing. The method comprises: inputting a gesture image; applying a singular value decomposition method to the gesture image; calculating at least one ray compensation coefficient by a ray compensation method; and performing ray compensation processing on the gesture image by using the ray compensation coefficient to obtain a gesture image Light compensated gesture image. The gesture image recognition system comprises an input unit, a calculation unit and an output unit. The input unit is configured to input the original gesture image, and the calculating unit calculates the light compensation coefficient by using the light compensation method, and performs light compensation processing on the gesture image by using the light compensation coefficient to obtain the light compensation gesture image, and the output is The unit is configured to output the light compensation gesture image.

In addition, related art techniques for conventional gesture image recognition have been disclosed in some U.S. patents, for example, U.S. Patent No. 7,702,130, User interface apparatus using hand gesture recognition and method After the Hand-gesture based interface apparatus, No. 6, 215, 890, Hand gesture recognizing device 〞, Hand gesture control system 第 No. 6, 002, 808 and Hand gesture machine control system 第 No. 5, 594, 469 The foregoing Japanese patents and U.S. patents are only for the purpose of the present invention, and are not intended to limit the scope of the present invention.

Although the aforementioned patents have disclosed related gesture image recognition techniques, they do not provide related techniques for three-dimensional gesture image recognition. In fact, in terms of gesture image recognition technology, it is necessary to simplify the complicated system architecture or omit complicated pre-processing procedures, otherwise it will affect the reliability of gesture image recognition. Therefore, the conventional gesture image recognition technology necessarily has the need to further provide three-dimensional gesture image recognition.

In view of the above, the present invention provides a three-dimensional gesture image recognition method and system thereof, which utilizes a light field photographing unit to take a gesture motion to obtain a 3D gesture image, and project the 3D gesture image to A predetermined recognition space is obtained to obtain a feature vector, and the feature vector is compared with a plurality of sample feature vectors to perform gesture image recognition, so as to improve the reliability of the conventional gesture image recognition technology.

The main purpose of the preferred embodiment of the present invention is to provide a three-dimensional gesture image recognition method and system thereof, which utilizes a light field photographing unit to take a gesture motion to obtain a 3D gesture image, and project the 3D gesture image to a predetermined The space is identified to obtain a feature vector, and the feature vector is compared with a plurality of sample feature vectors to achieve the purpose of improving the reliability of gesture image recognition.

In order to achieve the above object, a three-dimensional gesture image recognition method according to a preferred embodiment of the present invention includes: capturing a gesture by using a light field photographing unit to obtain a 3D gesture image; projecting the 3D gesture image to a predetermined recognition space to obtain a feature vector; and classifying the feature vector and the plurality of sample feature vectors to classify the 3D gesture image to identify the image The type of 3D gesture image.

In the preferred embodiment of the present invention, the 3D gesture image includes a plane image information and a depth information.

In the preferred embodiment of the present invention, the 3D gesture image is a 3D gesture contour image or a 3D gesture solid image.

In a preferred embodiment of the present invention, a 3D gesture solid RGB image is converted into a projected color space by using a projected color space method to obtain an R channel image information, a G channel image information, and a B channel image information.

In the preferred embodiment of the present invention, when the 3D gesture image is projected to the predetermined recognition space, the principal component analysis is used for projection.

In the preferred embodiment of the present invention, when the feature vector and the plurality of sample feature vectors are compared for classification, the nearest neighbor method is used for comparison and classification.

In order to achieve the above object, a three-dimensional gesture image recognition system according to a preferred embodiment of the present invention includes: a light field photographing unit that takes a gesture motion to obtain a 3D gesture image; and an arithmetic unit connected to the light field photographing unit Projecting the 3D gesture image to a predetermined recognition space to obtain a feature vector, and then classifying the feature vector and the plurality of sample feature vectors to classify the 3D gesture image to identify the 3D gesture image. And an output unit connected to the calculation unit to output the type of the 3D gesture image.

In the preferred embodiment of the present invention, the 3D gesture image is a 3D gesture contour image or a 3D gesture solid image.

In a preferred embodiment of the present invention, a 3D gesture solid RGB image is converted into a projected color space by using a projected color space method to obtain an R channel image information, a G channel image information, and a B channel image information.

In the preferred embodiment of the present invention, when the 3D gesture image is projected into the predetermined recognition space, the principal component analysis is used for projection, and when the feature vector and the plurality of sample feature vectors are compared for classification, the nearest neighbor method is adopted. Perform a comparison classification.

S1‧‧‧ steps

S2‧‧‧ steps

S3‧‧‧ steps

10‧‧‧Light field photography unit

20‧‧‧ calculus unit

30‧‧‧Output unit

FIG. 1 is a flow chart showing a method for recognizing a three-dimensional gesture image according to a preferred embodiment of the present invention.

2 is a block diagram of a three-dimensional gesture image recognition system in accordance with a preferred embodiment of the present invention.

3A and 3B are diagrams showing a three-dimensional gesture image recognition method and a system thereof for capturing nine 3D gesture contour images and nine 3D gesture solid images.

4A and 4B are diagrams showing a three-dimensional gesture image recognition method and a system thereof according to a preferred embodiment of the present invention, and using a variety of plane rotation variation conditions to respectively test a 3D gesture contour image and a 3D gesture solid image, the relationship between the recognition rate and the feature vector schematic diagram.

5A and 5B are diagrams showing a three-dimensional gesture image recognition method and a system thereof according to a preferred embodiment of the present invention, which use various depth rotation variation conditions to respectively test the relationship between the recognition rate and the feature vector after testing the 3D gesture contour image and the 3D gesture solid image. schematic diagram.

FIG. 6A is a schematic diagram of a three-dimensional gesture image recognition method according to another preferred embodiment of the present invention and a system for capturing the solid RGB images of nine 3D gestures. Figure.

FIG. 6B is a schematic diagram of a three-dimensional gesture image recognition method and a system thereof for performing projection color space conversion on a solid RGB image of a 3D gesture, and generating nine PCS projection images.

7(a) to 7(d): a three-dimensional gesture image recognition method and system thereof according to another preferred embodiment of the present invention, which is processed by an original 3D gesture solid RGB image to generate an R channel image, a G channel image, and A B channel image, and then a schematic diagram of a PCS projection image.

FIG. 8 is a schematic diagram showing a curve of a relationship between a recognition rate of a grayscale gesture image and a PCS projection image and a feature vector after performing a projection color space conversion according to another preferred embodiment of the present invention.

9(a) to 9(f): a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention, after testing with various Gaussian noise variations conditions, the original 3D gesture contour image and a series of test 3D A schematic diagram of the comparison of gesture contour images.

FIG. 10 is a schematic diagram showing the relationship between the recognition rate and the feature vector of a series of 3D gesture contour images after the test is performed by adding various Gaussian noise variations conditions according to the preferred embodiment of the present invention.

11(a) to 11(f): a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention, after adding various Gaussian noise variation conditions for testing, the original 3D gesture solid image and a series of test 3D A schematic diagram of the comparison of gestures and solid images.

12(a) to 12(f): a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention are tested by adding various Gaussian noise variations conditions, and performing projection color space conversion to project the original PCS A comparison of images and a series of test PCS projection images.

Figure 13: Three-dimensional gesture image recognition method in accordance with a preferred embodiment of the present invention The system and its system adopt a variety of Gaussian noise variation conditions to test, the relationship between the recognition rate of a series of 3D gesture contour images and the feature vector.

In order to fully understand the present invention, the preferred embodiments of the present invention are described in detail below, and are not intended to limit the invention.

The three-dimensional gesture image recognition method and system thereof according to the preferred embodiment of the present invention are applicable to various gesture recognition devices and related application devices, such as various types of computer systems, home appliance control systems (such as the Internet of Things), automated control systems, and medical care. The system or the security system is not intended to limit the scope of the invention.

FIG. 1 is a flow chart showing a method for recognizing a three-dimensional gesture image according to a preferred embodiment of the present invention. FIG. 2 is a block diagram showing a three-dimensional gesture image recognition system according to a preferred embodiment of the present invention, which corresponds to FIG. Referring to FIGS. 1 and 2, for example, a three-dimensional gesture image recognition system according to a preferred embodiment of the present invention includes a light field capturing unit 10, a calculation unit 20, and an output unit 30. The calculation unit 20 is suitably connected to the light field photographing unit 10, and the output unit 30 is suitably connected to the calculation unit 20.

Referring to FIG. 1 and FIG. 2 again, the method for recognizing a three-dimensional gesture image according to a preferred embodiment of the present invention includes the step S1: first, the light field imaging unit 10 or a light field image input unit having a similar function is used to take a gesture. To get a 3D gesture image. For example, the 3D gesture image includes a plane image information and a depth information. The 3D gesture image is a 3D gesture contour image or a 3D gesture solid image.

FIG. 3A is a schematic diagram showing a three-dimensional gesture image recognition method and a system for capturing nine 3D gesture contour images according to a preferred embodiment of the present invention, which includes zero to eight 3D gesture images. In contrast, Figure 3B reveals A three-dimensional gesture image recognition method and a system thereof for capturing nine solid images of 3D gestures in a preferred embodiment of the present invention.

Referring to FIG. 1 and FIG. 2 again, the three-dimensional gesture image recognition method according to the preferred embodiment of the present invention includes the step S2: then, the 3D gesture image is projected to a predetermined recognition space to obtain a feature vector. For example, the preferred embodiment of the present invention uses a principal component analysis (PCA) method to project features of the 3D gesture image into a relatively easily discernible space.

The principal component analysis can be applied to the dimensionality reduction of high-dimensional data, and the degree of variation of the data is preserved, because the one-dimensional PCA needs to straighten the dimensions of the input training image. Taking the image of ( m × n ) size as an example, when the covariation matrix is calculated, a matrix of ( m × n ) × ( m × n ) is generated, which requires a great calculation time for the step of calculating the feature vector. . Therefore i changed the original covariation matrix to the following formula:

Where C _i is the changed covariation matrix, L is the number of training samples, X _tr is the training image, and X is the average of all training images. In this way, the covariation matrix dimension is reduced to L × L and the time required to calculate the projection substrate is greatly reduced. The preferred embodiment of the present invention uses the singular value decomposition [SVD] to operate the covariation matrix as follows:

The formula obtained eigenvalue matrix Σ _i resulting eigenvalue matrix [Sigma same original covariance matrix C through singular value decomposition, and the eigenvector matrix shall be to (X _tr - ) * U _i square forms the eigenvector matrix U . Then, the original training data is projected into the PCA space by using the feature vector matrix to obtain the PCA processed training sample feature vector F _tr .

Referring to FIG. 1 and FIG. 2 again, the three-dimensional gesture image recognition method according to the preferred embodiment of the present invention includes the step S3: then, the feature vector and the plurality of sample feature vectors are compared and classified to classify the 3D gesture. Image to identify the type of 3D gesture image. For example, when the feature vector is compared with a plurality of sample feature vectors for comparison, the nearest neighbor method ( KNN , k- nearest neighbors) is used for comparison classification. The k training samples closest to the test image in the feature space are classified, the test image is projected into the PCA space, and compared with the training samples, so that the similarity is calculated as follows:

Where S _k is the calculated similarity matrix, k is the number of the nearest training samples, N is the maximum value of the feature roots, F _te and F _tr are the feature vectors of the test and training samples, and the similarity is selected according to the k value. The nearest k training samples determine which of the selected training samples is closest to the classification of the selected training samples for classification prediction.

FIG. 4A is a schematic diagram showing the relationship between the recognition rate and the feature vector after the 3D gesture contour image is tested by using various in-plane rotation variations conditions according to a preferred embodiment of the present invention. . Referring to FIG. 4A, the preferred embodiment of the present invention uses rotation angles of ±3, ±5, ±8, and ±10 degrees, respectively. Although the highest recognition rate of the 3D gesture contour image is 100% for k =1 and k =3, the trend of the recognition rate according to the eigenvector [eigen vector] rises faster with k =1. The degree of decline in the recognition rate of the two is also approximately equal. In this case, it is determined that k =1 is the k value that is most suitable for the plane rotation contour gesture, and the recognition rate of up to 100% can be achieved when the 17th root is taken.

In contrast, FIG. 4B discloses a three-dimensional gesture image recognition method according to a preferred embodiment of the present invention, and a system diagram of the relationship between the recognition rate and the feature vector after testing the solid image of the 3D gesture using various plane rotation variation conditions. Referring to FIG. 4B, the preferred embodiment of the present invention uses rotation angles of ±3, ±5, ±8, and ±10 degrees, respectively. The solid image of the 3D gesture has a large amplitude of oscillation when k =2, and also shows a tendency to decay after the recognition rate reaches the highest. Therefore, only k =1 and k = 3 are compared here, and when k = 3, the convergence is faster and the stability is maintained, so that k = 3 is determined to be the k value most suitable for the plane rotation contour gesture, and Up to 100% recognition rate can be achieved when the 7th root is taken.

FIG. 5A discloses a three-dimensional gesture image recognition method and a system thereof according to a preferred embodiment of the present invention. After using various out-of-plane rotation conditions to test a 3D gesture contour image, the relationship between the recognition rate and the feature vector is Schematic diagram of the curve. Referring to FIG. 5A, the preferred embodiment of the present invention uses rotation angles of ±15 and ±30 degrees, respectively. After adding the deep rotation variation condition, the recognition rate of the 3D gesture contour image is reduced to reach a maximum of 94.07% when k =1, and the highest recognition rate can be achieved when the feature is taken to the third root, and then the display is gradually slow. Downward trend.

In contrast, FIG. 5B discloses a three-dimensional gesture image recognition method according to a preferred embodiment of the present invention, and a system diagram of the relationship between the recognition rate and the feature vector after testing the solid image of the 3D gesture using various depth rotation variation conditions. Referring to FIG. 5B, the preferred embodiment of the present invention uses rotation angles of ±15 and ±30 degrees, respectively. The 3D gesture solid image has a significant decrease in the recognition rate of the deep rotation variation, and its maximum recognition rate is only 78.89% when k =1. Since the solid image of the 3D gesture is extracted only by the grayscale image of the depth, it cannot completely retain all the depth information. When rotated in depth, the spacing within the category is thus increased, and the characteristic distribution of the PCA projection is mixed with other types of images.

In order to solve the above problem, another preferred embodiment of the present invention uses the original RGB image of the solid image of the 3D gesture to project the image information of the RGB three channels into the same space by the projection color space [PCS] method, thereby retaining More image information.

In the case of the 3D gesture contour image, the influence of the depth change on the recognition rate is not as large as the solid gesture, because the depth change does not cause too much change to the contour line when the recognition is performed only by the contour information. On the contrary, in the case of the solid image of the 3D gesture, the variation of the depth rotation causes the spacing of the feature distributions between different depths in the same category to increase, resulting in a decrease in the recognition rate.

FIG. 6A is a schematic diagram showing a three-dimensional gesture image recognition method and a system for capturing nine solid 3D gesture RGB images according to another preferred embodiment of the present invention. FIG. 6B is a schematic diagram showing a method for recognizing a three-dimensional gesture image and a system thereof for performing projection color space conversion on a solid RGB image of a 3D gesture, and generating nine PCS projection images, which corresponds to FIG. 6A.

7(a) to 7(d) show a three-dimensional gesture image recognition method and system thereof according to another preferred embodiment of the present invention, which is processed by the original 3D gesture solid RGB image to generate an R channel image, a G channel image, and A B channel image, and then a schematic diagram of a PCS projection image. Referring to FIG. 7(a) to FIG. 7(d), the three-dimensional gesture image recognition method according to another preferred embodiment of the present invention selects to capture at least one 3D gesture solid RGB image, as shown in FIG. 7(a). Show. Then, the 3D gesture solid RGB image is obtained by the projection color space method to obtain RGB three-channel image information, as shown in FIGS. 7(b), 7(c), and 7(d). Then, the RGB three-channel image information is projected into the same space to obtain a PCS gesture solid projection image, as shown in FIG. 7(d).

FIG. 8 is a schematic diagram showing a curve of a relationship between a recognition rate of a grayscale gesture image and a PCS projection image and a feature vector after a projection color space conversion method according to another preferred embodiment of the present invention. Referring to FIG. 8 , the recognition rate of the solid projected image of the PCS gesture at the 14th root reaches 96.67%, and then the downward trend is stable, and the overall recognition rate is higher than the gray-scale solid gesture image. When the gray-scale solid gesture image is compared with the solid projection image of the PCS gesture, the recognition rate is significantly improved. Since the solid projected image of the PCS gesture retains the depth information of the three channels of RGB, no information is lost due to the depth change. For deep rotation variations, higher tolerances can be produced.

9(a) to 9(f) show a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention, and after adding various Gaussian noise variation conditions, the original 3D gesture contour image is A series of comparison diagrams for testing 3D gesture contour images. Referring to Figures 9(a) through 9(f), the average of the Gaussian noise added to the original 3D gesture contour image (as shown in Figure 9(a)) is 0, to 1%, 5 The %, 10%, 20%, and 30% variances were identified as shown in Figures 9(b) through 9(f).

FIG. 10 is a schematic diagram showing the relationship between the recognition rate and the feature vector of a series of 3D gesture contour images after the three-dimensional gesture image recognition method is performed by adding various Gaussian noise variations conditions according to the preferred embodiment of the present invention. Figures 9(b) to 9(f). Please refer to Figure 10 to add the original 3D gesture contour image (as shown in Figure 9(a)) to 1%, 5%, 10%, 20%, 30% Gaussian noise. From 9(b) to 9(f), the highest recognition rate of more than 90% is maintained until 5% of the noise is added, and the trend is relatively stable; when the added noise is higher than 10%, although the highest The degree of decline in the recognition rate is not very obvious, and there is still a maximum recognition rate of 82.22% when adding 30% Gaussian noise. However, when the number of selected features increases, the recognition rate decreases too fast, which can be displayed. Identification rate Less stable.

11(a) to 11(f) illustrate a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention, and after adding various Gaussian noise variations conditions, the original 3D gesture solid image and a series of test 3D are tested. A schematic diagram of the comparison of gestures and solid images. Referring to Figures 11(a) through 11(f), the average of the Gaussian noise added to the original 3D gesture solid image (as shown in Figure 11(a)) is 0, to 1%, 5 The %, 10%, 20%, and 30% variances were identified as shown in Figures 11(b) through 11(f).

12(a) to 12(f) illustrate a three-dimensional gesture image recognition method and system thereof according to a preferred embodiment of the present invention, which adopts adding various Gaussian noise variation conditions for testing, and performs projection color space conversion to project the original PCS. A comparison of images and a series of test PCS projection images, corresponding to Figures 11(a) through 11(f).

FIG. 13 is a schematic diagram showing the relationship between the recognition rate and the feature vector of a series of 3D gesture contour images after the test is performed by adding various Gaussian noise variations conditions according to the preferred embodiment of the present invention. Please refer to the 10th and 13th pictures to add 1% to 30% Gaussian noise to the original 3D gesture solid image (as shown in Figure 11(a)), and then process it in the projected color space, although the highest recognition The rate also showed a downward trend, but even with the addition of 30% Gaussian noise, the recognition rate remained volatile at more than 85%. Since the 3D gesture contour image (as shown in Figures 9(b) to 9(f)) retains only the outline information of the original gesture, it is not as solid as the 3D gesture in combating noise (eg, 11(b) As shown in Figure 11(f), the depth information advantage of full gestures is maintained.

The foregoing preferred embodiments are merely illustrative of the invention and the technical features thereof, and the techniques of the embodiments can be carried out with various substantial equivalent modifications and/or alternatives; therefore, the scope of the invention is subject to the appended claims. The scope defined by the scope shall prevail. The copyright limitation of this case is used for the purpose of patent application in the Republic of China.

Claims (10)

A three-dimensional gesture image recognition method includes: capturing a gesture by using a light field photographing unit to obtain a 3D gesture image, wherein the 3D gesture image is a 3D light field gesture image, and the 3D light field information includes the 3D light field information, and the The 3D light field gesture image is a 3D light field gesture contour RGB image or a 3D light field gesture solid RGB image; the 3D light field gesture image is projected to a predetermined recognition space to obtain a 3D light field feature vector; The 3D light field feature vector and the plurality of sample feature vectors are compared for classification to classify the 3D light field gesture image to identify the type of the 3D light field gesture image, wherein the 3D light field gesture profile is utilized by a projected color space manner RGB image or 3D light field gesture solid RGB image for projection color space conversion to obtain an R channel image information, a G channel image information and a B channel image information, and then project the 3D light field gesture image to the predetermined recognition space Principal component analysis is used for projection. The three-dimensional gesture image recognition method according to claim 1, wherein the 3D light field gesture image comprises a plane image information and a depth information. The three-dimensional gesture image recognition method according to claim 1, wherein the 3D light field feature vector is a feature number. According to the three-dimensional gesture image recognition method described in claim 1, wherein an RGB three-channel image information is projected into the same space to obtain a PCS gesture projection image. The three-dimensional gesture image recognition method according to claim 1, wherein when the 3D light field gesture image is projected to the predetermined recognition space, the principal component analysis is used for projection, and the 3D light field feature vector is When several sample feature vectors are used for comparison classification, the nearest neighbor method is used for comparison classification. The three-dimensional gesture image recognition method according to claim 1, wherein the 3D light field feature vector and the plurality of sample feature vectors are When performing the alignment classification, the nearest neighbor method is used for the comparison classification. A three-dimensional gesture image recognition system includes: a light field photographing unit that takes a gesture to obtain a 3D gesture image, and the 3D gesture image is a 3D light field gesture image, which includes 3D light field information, and The 3D light field gesture image is a 3D light field gesture contour RGB image or a 3D light field gesture solid RGB image; a calculation unit connected to the light field photography unit, and projecting the 3D light field gesture image to a predetermined identification Space, to obtain a 3D light field feature vector, and then classify the 3D light field feature vector and the plurality of sample feature vectors to classify the 3D light field gesture image to identify the type of the 3D light field gesture image And an output unit connected to the calculation unit for outputting the type of the 3D light field gesture image, wherein the 3D light field gesture contour RGB image or the 3D light field gesture solid RGB image is projected by the projection color space method Space conversion to obtain an R channel image information, a G channel image information, and a B channel image information, and then project the 3D light field gesture image into the predetermined recognition space. Principal component analysis is projected. The three-dimensional gesture image recognition system according to claim 7, wherein the 3D light field feature vector is a feature number. According to the three-dimensional gesture image recognition system described in claim 7, wherein an RGB three-channel image information is projected into the same space to obtain a PCS gesture projection image. The three-dimensional gesture image recognition system according to claim 7 , wherein when the 3D light field gesture image is projected to the predetermined recognition space, the principal component analysis is used for projection, and the 3D light field feature vector is When several sample feature vectors are used for comparison classification, the nearest neighbor method is used for comparison classification.