CN104808798B - A multi-screen interactive ethnic dance entertainment system based on Kinect - Google Patents

Abstract

A kind of multi-screen interactive traditional dance entertainment systems based on Kinect, the present invention is the technical field of the body-sensing traditional dance dance game system based on Kinect device.It is to lack the property immersed in order to solve existing dance game and lack the problem of being played on traditional dance.Its first projecting apparatus, the second projecting apparatus, the 3rd projecting apparatus and the 4th projecting apparatus are all disposed within the center in the square surrounded in order by the first curtain, the second curtain, act 3 cloth and act 4 cloth, Kinect device is arranged on the lower section before the first curtain, first audio amplifier and the second audio amplifier are separately positioned at act 3 cloth two ends, and the data output end of Kinect device and the data input pin of main frame are connected.Body-sensing Kinect device and China merging for traditional dance of classics of the present invention is studied and innovated, enable users to access accurately action by the correlation technique of action recognition and judgement to feed back, user is obtained whole-heartedly immersion experience by multi-screen projection.

Description

A multi-screen interactive ethnic dance entertainment system based on Kinect

technical field

The invention belongs to the technical field of the somatosensory folk dance game system based on Kinect equipment.

Background technique

In recent years, people's spiritual entertainment life has been continuously enriched, and dance activities have developed rapidly. Whether it is a computer dance game or a dance mat game, these dance games can enrich people's spiritual life and bring people a sense of rhythm, rhythm and beauty. Existing dance games on various platforms include dance mats, computer games, and mobile games. These devices can only accept hand movements or footsteps, which is far from the natural form of dance, and cannot fully participate in games. At the same time, the contents of these games are mainly modern western dances, and the folk dances with profound cultural heritage in our country rarely appear. On the one hand, it is due to its pursuit of interests, and on the other hand, it is also because the national dance of our country needs corresponding costumes and atmosphere. It is difficult for mobile phones, tablets, and dance mat games to immerse in the atmosphere of national dance. This system has carried out research and innovation on the integration of somatosensory equipment and our country's classic folk dance. Through the related methods of motion recognition and judgment, users can get accurate motion feedback, and through multi-screen projection, users can get a whole-body immersive experience.

Contents of the invention

The purpose of the present invention is to provide a Kinect-based multi-screen interactive ethnic dance entertainment system, in order to solve the lack of immersion in existing dance games and the lack of problems about ethnic dance games.

Described purpose is achieved by following scheme: described a kind of multi-screen interactive ethnic dance entertainment system based on Kinect, it comprises main frame 1, Kinect equipment 2, the first curtain 3, the second curtain 4, the third curtain 5 , the fourth curtain 6, the first projector 7, the second projector 8, the third projector 9, the fourth projector 10, the audio equipment 11, the first sound box 12, the second sound box 13;

The first projector 7, the second projector 8, the third projector 9 and the fourth projector 10 are all arranged on the first curtain 3, the second curtain 4, the third curtain 5 and the fourth curtain 6 in order surrounded by the center of a square, and the projected image of the first projector 7 is projected on the first curtain 3, the projected image of the second projector 8 is projected on the second curtain 4, and the third projection The projected image of instrument 9 is projected on the third curtain 5, the projected image of the fourth projector 10 is projected on the fourth curtain 6, the Kinect device 2 is arranged below the front of the first curtain 3, the first sound box 12 and the second Two sound boxes 13 are respectively arranged at the both sides of the third curtain 5, the drive input end of the first sound box 12 and the drive input end of the second sound box 13 are all connected with the drive output end of the audio equipment 11, and the data output of the Kinect equipment 2 end is connected with the data input end of host computer 1, the digital video signal input end of the first projector 7, the digital video signal input end of the second projector 8, the digital video signal input end of the third projector 9 and the fourth projector The digital video signal input ends of 10 are all respectively connected with the multi-channel digital video output ends of the main frame 1, and the multi-channel digital audio input ends of the audio equipment 11 are respectively connected with the multi-channel digital audio output ends of the main frame 1; start the main frame 1, Kinect device 2, first projector 7, second projector 8, third projector 9, fourth projector 10 and audio equipment 11, initialize Kinect device 2, Kinect device 2 obtains image data, including color flow ColorImageSteam, depth Stream DepthImageSteam and skeleton stream SkeletonStream to generate the Iplmage image in OpenCV; host 1 judges the number of players participating in the game through Kinect device 2, and selects the player with the smallest depth value, that is, the player closest to the visible field of view of the device for bone tracking; connects the player bone node with the model bone node Bind and map the coordinates to the OpenCV drawing coordinate system; obtain 121 key points of the player's face, separate the player's face image, and assign the player's face image to the model through key point matching; the host 1 uses the Kinect device 2 to determine the player's action , after the action is matched, the next level of the game can be played; a large amount of dance action data is stored in the action library in advance, represented by the relative position between the joint points; after the player makes an action, the relative position and action of the player's bone node The relative position in the library is matched, and it is judged whether the matching within a certain threshold of the spatial position of the action within a specific time is satisfied.

Research and innovation have been carried out on the integration of the somatosensory Kinect device of the present invention and the classic folk dance of our country. Through the relevant methods of motion recognition and judgment, users can get accurate motion feedback, and through multi-screen projection, users can get a whole-body immersive experience.

Description of drawings

Fig. 1 is a schematic diagram of the overall structure of the present invention.

detailed description

Specific embodiment one: in conjunction with shown in Figure 1, it comprises main frame 1, Kinect device 2, first curtain 3, second curtain 4, the third curtain 5, the fourth curtain 6, first projector 7, the second projector 8. The third projector 9, the fourth projector 10, the audio equipment 11, the first speaker 12, and the second speaker 13;

The first projector 7, the second projector 8, the third projector 9 and the fourth projector 10 are all arranged on the first curtain 3, the second curtain 4, the third curtain 5 and the fourth curtain 6 in order surrounded by the center of a square, and the projected image of the first projector 7 is projected on the first curtain 3, the projected image of the second projector 8 is projected on the second curtain 4, and the third projection The projected image of instrument 9 is projected on the third curtain 5, the projected image of the fourth projector 10 is projected on the fourth curtain 6, the Kinect device 2 is arranged below the front of the first curtain 3, the first sound box 12 and the second Two sound boxes 13 are respectively arranged at the both sides of the third curtain 5, the drive input end of the first sound box 12 and the drive input end of the second sound box 13 are all connected with the drive output end of the audio equipment 11, and the data output of the Kinect equipment 2 end is connected with the data input end of host computer 1, the digital video signal input end of the first projector 7, the digital video signal input end of the second projector 8, the digital video signal input end of the third projector 9 and the fourth projector The digital video signal input ends of 10 are all respectively connected with the multi-channel digital video output ends of the main frame 1, and the multi-channel digital audio input ends of the audio equipment 11 are respectively connected with the multi-channel digital audio output ends of the main frame 1; start the main frame 1, Kinect device 2, first projector 7, second projector 8, third projector 9, fourth projector 10 and audio equipment 11, initialize Kinect device 2, Kinect device 2 obtains image data, including color flow ColorImageSteam, depth Stream DepthImageSteam and skeleton stream SkeletonStream to generate the Iplmage image in OpenCV; host 1 judges the number of players participating in the game through Kinect device 2, and selects the player with the smallest depth value, that is, the player closest to the visible field of view of the device for bone tracking; connects the player bone node with the model bone node Bind and map the coordinates to the OpenCV drawing coordinate system; obtain 121 key points of the player's face, separate the player's face image, and assign the player's face image to the model through key point matching; the host 1 uses the Kinect device 2 to determine the player's action , after the action is matched, the next level of the game can be played; a large amount of dance action data is stored in the action library in advance, represented by the relative position between the joint points; after the player makes an action, the relative position and action of the player's bone node The relative position in the library is matched, and it is judged whether the matching within a certain threshold of the spatial position of the action within a specific time is satisfied.

Working principle: The principle of free dance is to obtain the user's bone point information and assign it to the model. The working principle of teaching dance includes two parts, one is to quantify the data storage of standard dance movements, and the other is to compare user movement data with standard movement data. Through research, it is found that folk dance movements mostly use changes in elbow joints, knee joints and wrist joints to make various forms of dance postures. Therefore, firstly, the movement data of the elbow joint, knee joint and wrist joint in standard dance movements are quantified and stored. In the motion judgment, the motion of the elbow joint, knee joint and wrist joint is judged first, and then the motion judgment of other parts is performed. The judging method is as follows: firstly establish the xyz coordinate system of the space three-dimensional coordinate system, the space position of the player’s bone node is represented by the point (x, y, z) in the space three-dimensional coordinate system, and the error range of the action angle is measured by experiment when the value is α Most reasonably, α is π/36. Taking the judgment of the shape of the elbow joint as an example, three straight lines can be constructed in space by using the positions of the wrist node, elbow node and shoulder node. Take the right arm as an example, the position of the shoulder center is (x0, y0, z0), the position of the right shoulder is (x1, y1, z1), the position of the right elbow is (x2, y2, z2), and the position of the right wrist is (x3, y3, z3) ). The straight line formed by the right shoulder and the right wrist is L13, the straight line formed by the right shoulder and the right elbow is L12, the straight line formed by the right elbow and the right wrist is L23, and the straight line formed by the center of the shoulder and the right shoulder is L01. The shape is related to the rotation of the shoulder and elbow, so the relative position of the shoulder, elbow, and wrist joints is used as the judgment standard, and the user's arm length is not considered but the joint rotation angle is used as the judgment standard. First, determine whether the angle between L13 and L01 satisfies the interval [K013-α, K013+α], K013 is the angle of the right wrist relative to the right shoulder in the standard dance movements stored in the movement library. The angle between L13 and L01 can be obtained by using the space vector angle formula, and the angle calculation formula is:

Similarly, the formula for calculating the included angle between the straight line L12 and the straight line L23 is:

Judging whether the angle meets the standard angle range [K123-α, K123+α] stored in the action library.

When the angles of all relevant nodes match the angles in the action library, the action matching is successful. You can learn the next dance move.

Claims (1)

1. A multi-screen interactive ethnic dance entertainment system based on Kinect is characterized in that it comprises a main frame (1), Kinect equipment (2), the first curtain (3), the second curtain (4), the third curtain (5 ), fourth curtain (6), first projector (7), second projector (8), third projector (9), fourth projector (10), audio equipment (11), first speaker (12), the second speaker (13); The first projector (7), the second projector (8), the third projector (9) and the fourth projector (10) are all arranged on the first curtain (3), the second curtain (4) , the center of a square enclosed by the third curtain (5) and the fourth curtain (6) in order, and project the projected image of the first projector (7) on the first curtain (3), The projected image of the second projector (8) is projected on the second curtain (4), the projected image of the third projector (9) is projected on the third curtain (5), and the projected image of the fourth projector (10) The projected image is projected on the fourth curtain (6), the Kinect device (2) is arranged under the front of the first curtain (3), and the first sound box (12) and the second sound box (13) are respectively arranged on the third curtain ( 5) at both sides, the drive input of the first speaker (12) and the drive input of the second speaker (13) are connected to the drive output of the audio device (11), and the data output of the Kinect device (2) The terminal is connected to the data input terminal of the host computer (1), the digital video signal input terminal of the first projector (7), the digital video signal input terminal of the second projector (8), and the digital video signal input terminal of the third projector (9). The signal input end and the digital video signal input end of the fourth projector (10) are all respectively connected with the multi-channel digital video output end of the host (1), and the multi-channel digital audio input ends of the audio equipment (11) are respectively connected with the host ( 1) Multi-channel digital audio output connection; start the host (1), Kinect device (2), the first projector (7), the second projector (8), the third projector (9), the fourth The projector (10) and audio equipment (11), initialize the Kinect device (2), and the Kinect device (2) obtains image data, including color stream ColorImageSteam, depth stream DepthImageSteam, and skeleton stream SkeletonStream, to generate the Iplmage image in OpenCV; the host ( 1) Use the Kinect device (2) to judge the number of players participating in the game, and select the player with the smallest depth value, that is, the player closest to the visible field of view of the device, for bone tracking; bind the player bone node to the model bone node and map the coordinates to the OpenCV drawing coordinate system ; Obtain 121 key points of the player's face, separate the player's facial image, and assign the player's facial image to the model through key point matching; the host (1) judges the player's action through the Kinect device (2), and the next step can be performed after the action is matched. Close the game; store a large amount of dance action data in the action library in advance, represented by the relative position between joint points; after the player makes an action, match the relative position of the player's bone node with the relative position in the action library, Judging whether the matching of actions within a certain threshold in the spatial position within a specific time is satisfied.