TW201137766A - Image based motion gesture recognition method and system thereof - Google Patents

Abstract

An image based motion gesture recognition method and system thereof are disclosed. In embodiment, a hand posture detection is performed according to the received image frames, to obtain a first hand posture. It is then determined whether the first hand posture matches a predefined starting posture or not. If the first hand posture matches said predefined starting posture, movement tracking is performed according to hand locations on image frames, to obtain a motion gesture. During said movement tracking, the hand posture detection is performed according to said image frames to obtain a second hand posture, and it is determined whether the second hand posture matches a predefined ending posture. If the second hand posture matches the predefined ending posture, the movement tracking is stopped. Therefore, reduce complexity motion gesture recognition can be reduced and the reliability in interaction can be improved.

Description

201137766 VI. Description of the Invention: [Technical Field] The present invention relates to a hand-detecting system, and an image-based action hand specially configured with a sensor [Prior Art]

:: For a fast-developing entertainment system, especially how the game system makes the interaction between the user and the computer more friendly is an important issue for Qin. Among them, the analysis of the user's actions through the computer ^ command has become the most promising interactive method in the future. $, the traditional # = solution often needs to be configured on the user's finger - the sensor, although this can increase the hand-made standard, but it also increases the burden on the user. In addition, the preferred method is to directly treat the hand of (4) as: the release device, analyzes the hand movement mode of the latter by image processing, and inputs the command 'control computer operating system or peripheral device. This traditional image analysis method is too complicated and not stable enough. • For example, the known US patent, its patent number MG2, followed by a method for quickly analyzing gestures to control a computer, 1 image vector calculation to determine the position of the user's hand, square (4) and large ^ then, through The way of image processing determines the gesture. For example, if there is a hole in the hand image of the exact ^, it means that the user's thumb touches the index finger and the OK gesture is displayed. In addition, this patent also discloses a screen display interface (0SD) that can be displayed by a hand control computer. This conventional technology is calculated as 201137766, which is easy to be in (4) lightly stable. For example, another known US patent, the patent number Am % Road, a gesture recognition system, is characterized by the position of a plurality of objects on the hands of the user θ, . Therefore, in the plurality of markers, the ff is recorded as a second marker group, and the first product is produced by you: the second component of the first-private group and the device #测第-产Γ°, and is used as a reference. And feel the user gesture in one step. , the movement of the object, can not be carried out by hand. 2 knowing the material to ask the user to wear the standard operation to interact with the hand gesture or the movement track and the retreat, the problem and the problem to be solved. SUMMARY OF THE INVENTION In view of the above-mentioned prior art, the present invention provides a method for improving the ease of use and reducing the computational complexity of the second method to achieve the object according to the present invention. The following steps are recorded in the action gesture of the image. The image is executed by a number of images: like a face, according to whether the first gesture is a sign, ', ' to pass a first gesture; the gesture conforms to the preset Start gesture; if this first-hand position, perform a movement tracking = this = image in the process of this movement tracking in the process of =- movement gesture; perform this hand according to this Wei Zhang image face 201137766 Measure to obtain a second gesture; determine whether the second gesture conforms to a preset end gesture; if the second gesture conforms to the preset end gesture 'stop the mobile tracking. If the second gesture does not meet the preset end gesture, the mobile tracking is continuously performed.

The step of performing the gesture detection further includes detecting whether the plurality of images of the plurality of images are present - the image of the hand is present; if the image of the hand exists, obtaining the hand image according to the hand image - the hand ^ For example, according to the hand contour image judgment - the direction of the hand and the direction of the finger and the number of the fingers identify the first hand = μ,, the step of your 籾, the step of the image contains the hand Image 2:: In the parent block; estimate the reduced motion vector between the end of the image area and the end of the image area. Wherein, the present invention records the plurality of motion vectors based on the image to obtain the H:potential recognition method to further move the trajectory to obtain the moving gesture. In the case of moving, the image is recognized by the step image of the hand direction, and the image 3 is touched according to the direction of the hand wheel. ...face-edge, to judge the hand, determine the number of the finger + position calculation to obtain the hand contour shadow ^ contains the execution - palm position of the palm of the hand contour image position; according to this heavy cutting hand Partial image; according to this already;:: cut 'to get a hand ~ like to judge the number of fingers. 201137766 Potential Discrimination: The purpose is to propose an image-based action-handling list: '匕 contains a storage unit, an image capture unit, - the first processing 7L, a comparison unit and a 篦 _ test _ storage - Preset start gesture and - preset end gesture? Shadow:: Two images inside. The first processing unit is rooted here; !~ like picture execution - gesture integration to get - first gesture. 3 I = Break this first—whether the gesture meets this preset start gesture. If the first gesture determines that the first gesture meets the preset start gesture, the second gesture conforms to the preset to determine the younger movement tracking. H Bay! This first-handling early element stops this end gesture when u does not conform to this preset for the second gesture. 匕The first step is to continue this movement tracking. The unit and the processing unit further include a - image processing unit. The processing unit detects the plurality of image objects = image -: image. The second image processing unit is based on a hand contour image. Gestures, such as taking a wait, judging a hand, M, and recognizing the 7° according to the hand contour image: hand brown and the number of fingers, and identifying the first gesture or the second gesture according to the number of the hand. 201137766 Mobile ίί single unit also contains - block detection unit and -

Id estimates at least the image block containing this hand image. Shifting </ br> is to estimate the multiple motion vectors between the complex image blocks. The second processor further includes a track recognition unit, and the plurality of motion vectors obtains a motion and recognizes the motion track to obtain the motion gesture. Touched: - 2:: Identification: The 兀 is based on the edge of the hand contour image of the shirt to determine the direction of the hand. ~ Among them, this gesture recognition unit performs one palm orientation to serve one of the hand contour images. The hand contour image performs a palm (four) set to = this f heart position for this image, A twisted L 〇 U takes one The hand image is cut and the number of fingers is determined based on the cut hand image. [Greed mode] The situation is determined by the image-based action image removal unit 12, the first-processing unit, and the second processing unit 15. The storage unit 11, for example, ends storing a preset start gesture 111 and a preset end potential 112. The shirt is like a capture unit (2). The image capturing unit 12 takes a picture of a plurality of shirt images. The first processing unit can output a continuous number of 121 images # 糸 糸 according to the number of image frames 131 ′ to obtain a first gesture 132. Alignment 7 201137766 Unit 14 determines this first gesture 111. Whether the compliance with the preset start gesture is performed. If the comparison unit 14 determines that the first gesture 132 conforms to the preset start gesture 11 , the second processing single magic 5 performs a movement according to the position of the hand in the plurality of image planes 121 . Track 151 to get a moving hand, 152. During the execution of the movement tracking 151, the first processing half 13 performs the gesture price measurement 131 according to the plurality of image images from 121, to continue to obtain the second gesture 133, if the comparison is performed. ^ 14 determines that the second gesture 133# coincides with the preset end trick 112, and the second processing unit 15 stops executing the motion tracking 151. When the comparison unit 14 determines that the second gesture 133 does not conform to the preset end gesture 112, the processing unit 15 continues to execute the movement tracking 151. Thereby, the system can prompt the user to preset the start gesture iu and the preset end gesture U2. When you want to input instructions or data by hand, the user can first put out a preset start gesture (1) to indicate that the command is to be started. After the system is successfully recognized, the user changes the gesture or moves the hand to operate. During the operation, the system continues to perform gesture recognition, and confirms the age of the person to be entered H (4) to confirm whether the user has placed the preset end gesture 112 to end the operation. Wherein, the preset start gesture 111 and the preset end gesture 112 can be a special and very clear gesture to ensure that the system is not easy to misjudge when the user performs the operation and changes the gesture; in addition, due to the clear separation between the start and the end, The system can also make the gesture recognition process simple, and the step-by-step operation is smoother, knowing: the possibility of real-time operation. 201137766 Please refer to the block diagram of the embodiment of the potential identification system of the present invention. In the figure, the two images ^ action hand memory 2 - camera 22, a first process = example includes - symbol U and a second processing unit 25. The first processing unit ::: first image: the processing unit 231, a second image processing unit, two::::33. The first image processing unit 231 detects any image of the plurality of shirt images 121 and writes the hand image of the plurality of shirt images 121. Figure 3: Figure 3: Binary 232 obtains the shirt image processing sheet according to the hand image 236. 3 image area 33). For example, the second image can be edge-detected by the first hand image, and the image is detected by the hand rim line 32 =: the image area 33 of the hand wheel is used as the hand gesture of the hand contour image 237 6 The identification unit 233 judges according to the direction of the hand and the number of the fingers (2) and the number of the fingers (four) like 2 3 7 - when the hand is broken, for example, according to the hand 2, the direction of the hand is 2 ...... Image touched by 237 = image area 33 is contact painting: direction; 3: right: as the upper edge of the third figure, the left edge of the hand surface (2) (four), the hand direction is defined as = tactile For the image painting, the number of fingers 239, the early 233 can perform a palm when the gesture of the shell is 237 - the center of gravity is different to obtain the hand contour image such as 'common two-dimensional according to the palm: Is] 9 201137766 Select a moment function J (X, y), then calculate the first-order moment and second-order moment Moo, Μ丨〇, Μ〇ι, Μ &quot;, M20 and Λ / according to this J (X, y) 02, as shown by the following equation: Μ〇° =ΣΣ7(^&gt;) xy Μί〇=ΣΣχ/(^&gt;0 χ yxy =ΣΣΛ^7(^,ν) χ y Μ2〇=ΣΣχ2 ^ Ν) χ &gt; · Μα2 = ΣΣ · ν% ^) x y 10 and M01 on the calculated position of the center of gravity (; CE, then '^ according to Choi, eight), as shown in the following equation:

The position of the center of gravity (Xc, VIII) such as ' y〇' Mqq ' λ/] and the width L are as shown in the fourth figure of Figure 4 at position 41. Then calculate the length of the hand rectangle according to Λ jyj 2() &amp; M02 a =

201137766 « ^ ^ ^ , Receiver with the center of gravity position 41 or the width of the L ^ type L2 a square in the seat wide 1 , the hand moment cut a round 蛣 w 仫 on the hand contour image 43 cut The body has been cut mrr (four) domain material - _ hand image - ^ hand "like 44 can be used to determine the number of fingers 2 dry orientation. If the hand image 44 is cut to indicate that the user's palm orientation is the default value, the domain distribution width is greater than the height, then Indicates the use: the direction of the area of the hand like 4 4; if the position of the (4) hand #1胄 is horizontal, the height of the user Mi is greater than the width, the orientation of the palm of the user is not vertical. Example::,5 In the exemplary diagram of determining the number of fingers in the present invention, a distance from the cut hand image 44 is first identified as w 45, and the calculated line 45 and the center of gravity position 41 are:::45 and then if d determines one / value (for example, (4)), then take the line segment PP' of -t //, and then calculate the segment PP of the cut hand image, the number of overlaps determines the number of fingers 239. Next, the gesture recognition unit 233 is further based on the hand The direction 238 and the number of fingers 239 identify the first gesture 131 or the second gesture 132. Applying the 'gesture recognition unit 233' can be compared with a database. Please continue the moxibustion: 6 figure 'the reason for the invention to identify the database of gestures' is not intended. In the figure, this database is a record The comparison information of three preset gestures, such as a fist gesture with a number of fingers of 0, a single-finger gesture of the number of &lt;1, and a palm gesture of the number of fingers; in addition, the database also compares these poor materials. Classified into four hand directions extending from East (E), West (W), South (S), and North (N); in addition, 'this database also compares these to rr τ 11

I 201137766

The data is classified into horizontal (H), vertical (V), and fist-type (magic and other three hand positions. The gesture recognition list S 233 can be queried according to the hand direction 238 and the hand 239 to obtain the corresponding gesture; for example The day identification unit 233 is based on the direction of the hand as the three pieces extending from the south. The three pieces of data can be obtained from the database: the inquiring side, the number of fingers is 丨 and the direction of the palm is: capsule m tL ^ ^ ^ Π Feng Shuiqian The two materials can query the gesture 62 in the shell library, which can indicate that the left-hand finger direction 238 is extended from the west, the number of fingers is $, and the horizontal data can be queried from the database. Gesture (7).Β

Mi 凡 25 can include a block detection unit, single 252 and a trace identification, uiyuan 253. The block image 236 to the two sides 121 includes a hand: a plurality of movements between the different directions to 58.: 孰 ,, 固 LI: Estimate the movement vector 25 of the general knowledge of this technical field The unit 253 records a plurality of captures of a mobile walker 2 259 to obtain a move gesture 152. Ai recognition mobile walks clear, see Fig. 7, which shows the production of the power generation identification method &quot;^ based on the action of the shirt figure Column step. ^= trash\action gesture recognition method includes

, 接收 Receive multiple images in writing. In the case of the cattle to 79 to P, according to this multiple images, the implementation of a hand of hot, the goal in step 72 roots. In step 73, it is judged that the value is 'to obtain a first gesture; if the second potential is not consistent with a preset start hand gesture to pay the preset start gesture, then in step 2011 12 201137766, according to the plurality of images 昼 φ The middle hand obtains a moving gesture; if not, the mobile tracking is performed by t, and the mobile tracking is performed. Perform this gesture price measurement in step 75 to get '^ this multiple image 昼=the second gesture is consistent with 1 setting: beam gesture at = :b6:, the gesture conforms to this preset end gesture, and the potential b If the second is no, proceed to step 75. #T止动移动追踪; The complexity of the gestures' and the recognition of the low tracking and identification movements. Please refer to Figure 8, which shows the flow chart of the present application. In the figure, the complex guess is detected in step 81. Whether there is a - hand image on the image side of the image. If the second:: two - - step 82 is based on the hand image - the image area 33 shown in the hand contour map. In the step like '3rd touch The image of the image is the edge-edge, and the direction of the hand image of the contour image image area 33 is judged as the hand/direction. For example, the step of calculating the palm head orientation to obtain the hand contour image is performed as shown in FIG. The center of gravity position 41. Connected to the ^ position, such as the first hand contour image performs a palm cut, j according to the center of gravity position image. In step 86, according to the cut hand = cut hand step 87 according to the hand direction and the number of fingers Refers to the number. If necessary, you can also judge the gesture according to the palm position. In addition, please refer to Figure 9, which shows the flow chart of the present invention. In the figure, in step 91, the following is performed: Really contains at least the shadow of the hand image The image has a plurality of moving directions between the image blocks; the ghost estimates the complex in step 92 to record the complex number in step 93. 201137766 =:; ΧΓ - moving the trajectory. For the sake of exemplification, the spirit and material of the present invention, 1,: the manufacturer, any unremoved, should be included in the attached application (4) ^ equivalent modification or change [simple description of the figure] Figure 1 Inventive image-based motion gesture recognition system 2nd image == image-based motion gesture recognition - =3 diagram is a schematic diagram of the hand wheel image of the invention; "picture system, the palm of the invention is cut Example diagram; =5 diagram is an example diagram for judging the number of fingers in the present invention; 6 is a schematic diagram of a database for identifying gestures according to the present invention, and is an image-based gesture recognition method of the present invention. The flowchart of the present invention is a flow chart for implementing the gesture detection of the present invention; and FIG. 9 is a flowchart for implementing the execution of the mobile tracking according to the present invention. 201137766 [Description of main component symbols] 11 : Storage unit 111 : Preset start gesture 112 : Preset end gesture 12 : Image capture unit 121 : Image screen 13 : First processing unit 131 : Gesture detection • 132 : First gesture 133: second gesture 14: comparison unit 15: second processing unit 151: mobile tracking 152: moving gesture 21: memory 22: camera • 23: first processing unit 231: first image processing unit 232: second Image processing unit 233: gesture recognition unit 236, 31: hand image 237, 43: hand contour image 238: hand direction 239: number of fingers 25: second processing unit 15 201137766 251: block detection unit 252: movement Vector unit 253: Track recognition unit 257: Image block 258: Motion vector 259: Motion trace 32: Hand contour 33: Image area 41: Center of gravity position 44: Cut hand image 45: Tip 61 to 63: Gesture 71~77: Steps flow 81~87 · Step flow 91~94: Step flow

Claims (1)

201137766 VII. Application for patents: 1. An image-based motion gesture recognition method, comprising receiving a plurality of image planes; performing a one-to-one first gesture according to the plurality of image planes; gesture detection, to determine the Whether the first gesture conforms to a preset start gesture; if the complex start score is the preset start gesture, the process is performed according to the hand position in the face-up to perform the movement tracking process to obtain a second gesture; whether the gesture conforms to a preset end gesture, such as performing the gesture detection x, determining the number according to the number of shadows, and 2. moving the second gesture to match the end gesture of the ship, stopping the fan=including The image-based action hand control method described in item 1 does not conform to the preset end gesture, and continues to perform the gesture detection step in the image-based action hand--such as the patent application fan 囹 flute identification method. More includes · In the two images of the two images - whether the image is stored IS} 17 201137766 « If the hand image exists, listen to the contour image of the hand; ~ image The number of servants is judged according to the image of the second-hand part of the porch - the direction of the hand and the direction of the finger or the root. The image-based action described in the fourth hand is identified according to the direction of the second-hand hand and the number of the finger. The step of performing the mobile tracking in the hand ^ , , the method further comprises: obtaining at least one image block of the n image in each of the plurality of images; and 匕 ... hand 5. _ remaining_moving vector of the block. In the middle of the square, the 44th-based image-based action player performs the move and the record of the plurality of movement vectors to obtain a mobile hold: to obtain the move track by 6. The move gesture. The second item of the second dimension is based on the two = edge of the action image of the action of the image, and the direction of the hand. (4) The image of the image is 7. The patent application scope 3rd potential identification method 'the' is the hand to perform the hand--the number of palms is more than ··································································· Position of the center of gravity; according to the position of the center of gravity, the vehicle is cut to obtain - two execution - the palm according to the number of fingers that have been cut. The number of fingers is 8. The image-based gesture recognition system includes: knot gesture and one Preset: the image capturing unit 'takes a plurality of image planes; the first processing unit is based on the plural number of cards, and the following is a gesture price measurement to obtain a first hand reading frame comparison In the early element, determining whether to start the gesture; whether the 1 gesture conforms to the pre-second processing unit, if the gesture conforms to the preset start gesture; and, determining the first two-dimensional processing unit in the first plurality of images According to the acquisition of a movement gesture; the execution of the movement tracking is performed, and the execution of the movement tracking unit is based on the plurality of shadows. The first gesture conforms to the preset end: the second movement tracking 1. A processing unit is stopped. 9. For example, the patent system has eight identification systems, wherein when the action gesture of the t 2 image is determined by the early 70, the second gesture is inconsistent with SJ 19 V V201137766 and the pre-beam (four), The second processing unit tracks β 々 the movement 10. The image-based potential recognition system as described in claim 8 wherein the first processing unit further includes a plurality of images. 5V is a hand image in the face; a second image processing unit is based on the contour image of the hand; and the image is: the direction of the hand and the number of the hand, and the number is identified according to the number of fingers - Gesture or the second gesture. Hai u.=Application of the image-based motion recognition system described in Item 1G, wherein the second processing unit further includes a face-to-face unit Obtaining at least one image block of the hand image in each of the plurality of image planes; and a motion vector unit estimating a plurality of motion vectors of the complex number. Between the red blocks 12.=C Image-based 11 The second processor further includes a trace identification. The trace recognition unit records the plurality of motion vectors to manually move the trace, and recognizes the movement track to obtain the movement as claimed in claim 10 The image-based action hand 20 201137766, wherein the gesture recognition unit determines the image-based action described in the patent application 帛 帛 1G according to the edge of the image face that is the hand of the hand Abundance identification system, in which the gesture hand The potential recognition system, where the image touches the direction of the hand. The position is calculated to obtain the image of the hand wheel image. The center of gravity is placed on the material part (4) f彡 pure line f, and the number of fingers is determined according to the hand cut image of the cut hand image. According to w has cut the hand image S S] 21