WO2013123640A1 - Motion interaction system and method - Google Patents

Description

 Motion interaction system and method

 The invention relates to a motion interaction system and method.

Background technique

 At present, people are paying more and more attention to sub-health or diseases caused by non-sports, so sports-based fitness games are endless.

 For example, the Japanese Nintendo game console provides motion sensing, and players can use the Wii handle to make actions similar to real-life sports, such as swinging tennis rackets, throwing bowling balls, and so on. However, the game player needs to operate through the handle, does not achieve the effect of full body exercise, and can not interact with other players in the same place in real time, which may make the exercise fun and efficiency less.

 A method for operating two or more mobile terminals in an interactive session, wherein the mobile terminal is used to provide one or more sensors, is disclosed in US 2006/0061468 A1. The sensor is used to obtain the user's heart rate, ambient temperature, humidity, etc., and calculate the user's PEL (personal excitability value). The data captured from the sensor is wirelessly connected to other mobile terminals participating in the interactive session, and data received from other mobile terminals is separately transmitted to the mobile terminal user through the user interface. The above-mentioned U.S. patents only enable the user to simply perform an interactive exercise and share only the physiological parameters such as the heart rate, ambient temperature, humidity, etc. of the user obtained during exercise between the mobile terminals.

Summary of the invention

 The present invention provides a new motion interaction system and method, which enables a user to obtain user action related information, such as action type information, in real time in various sports during the exercise process, and fully perform the server and other users. The data interaction, real-time exercise and sports interaction, greatly enhance the fun of sports.

 Specifically, the present invention provides a motion interaction server, including

 Means for providing motion interaction scene data to the client;

 Means for coordinating the transmission of the motion interaction scene data between a plurality of clients.

 among them,

The motion interaction scene data includes at least motion information generated by the client; and further includes one of a motion recognition algorithm model, a motion interaction scene, a motion interaction scene identifier list, or a combination thereof. among them,

 Means for acquiring motion information of the user according to an action signal of the user;

 Means for outputting a list of alternative motion interaction scene identifiers;

 Means for storing a motion interaction scenario corresponding to the motion interaction scene identifier list;

 Means for the user to select an identifier of the motion interaction scene to be entered by the user through the motion interaction scene identification list;

 Means for transmitting the user's motion information to the client in the same motion interaction scenario through coordinated communication by the server;

 The means for outputting the list of selectable motion interaction scenes is also used to output the motion information of the associated client.

 among them,

 The device for storing the motion interaction scenario corresponding to the motion interaction scenario identifier list is further configured to store a motion recognition algorithm model corresponding to each of the motion interaction scenarios.

 The present invention further provides a sports interaction system including a plurality of clients and at least one interactive server, wherein

 Each of the clients is configured to output a motion interaction scene identifier list, so that the user selects an identifier of the motion interaction scene to be entered from the motion interaction scene identifier list, and provides the motion interaction scenario, and is used to acquire the The motion information of the action performed by the user in the selected motion interaction scenario; and,

 Each of the clients in the same motion interaction scenario performs the transmission of the motion information through coordinated communication of the interaction server, and outputs the motion information of the corresponding client in the same motion interaction scenario;

 The interaction server also provides each client with a list of alternative motion interaction scene identifiers and corresponding motion interaction scenarios.

 among them,

 Each of the related clients is also used to output a motion interaction result.

 among them,

The interaction server is further configured to store a motion recognition algorithm model corresponding to the motion interaction scenario, and provide the motion recognition algorithm model to the client. among them,

 The interaction server further provides the mobile interaction scene group and the corresponding scene group related information to the client based on each of the motion interaction scenarios.

 among them,

 The motion recognition algorithm model includes motion classification algorithm parameters.

 among them,

 The motion recognition algorithm model also includes an action classification algorithm.

 among them,

 The motion recognition algorithm model includes a cheat action classification algorithm parameter.

 among them,

 The motion recognition algorithm model further includes a cheat action classification algorithm.

 among them,

 Providing a plurality of motion interaction scenarios and an optional motion interaction scenario identifier list, wherein the user selects a motion interaction scenario to be entered in the motion interaction scenario;

 Providing a corresponding motion interaction scenario to the user according to the motion interaction scenario identifier selected by the user; acquiring action information of the user in the selected motion interaction scenario;

 Each of the clients in the same motion interaction scenario performs the transmission of the action information; and the related client in the same motion interaction scenario outputs the action information of the relevant client user in the motion interaction scenario.

 among them,

 Each of the clients also outputs a motion interaction result.

 among them,

 The motion information of the user includes a type of motion of the user;

 And providing a corresponding motion recognition algorithm model to the client according to the motion interaction scenario; the client identifying the action type of the user according to the motion recognition algorithm model.

 Other features, objects, and advantages of the present invention will become more apparent and understood from

DRAWINGS

1 is a structural diagram of a motion interaction system according to an embodiment of the present invention; 2 is a schematic structural diagram of a client according to a first embodiment of the present invention;

 FIG. 3 is a schematic diagram of a list of motion interaction identifiers provided by the client shown in FIG. 2;

 4 is a schematic diagram of grouping of various motion interaction scenarios provided by the motion interaction server of the present invention; FIG. 5 is a schematic structural diagram of a client according to a second embodiment of the present invention;

 FIG. 6 shows the flow of the motion interaction method of the present invention.

 In all of the above figures, the same reference numerals are used to indicate the same or similar features or functions. detailed description

 Referring to Figure 1, a block diagram of a motion interaction system 10 in accordance with an embodiment of the present invention.

 The plurality of clients 11A, 11B, 11C and the interaction server 12 are included, and the clients 11A-11C communicate with the interaction server 12 through the communication network 13, wherein the communication network 13 can be a local area network (LAN), a wide area network (WAN), The personal area network (PAN) or the Internet (INTERNET), etc., the communication interface can be wired or wireless. It can be understood by those skilled in the art that the number of clients in the motion interaction system 10 of the present invention is not limited to three, and FIG. 1 is only a schematic diagram illustrating the networking architecture of the motion interaction system 10.

 The clients 11A-11C in the kinematic interaction system 10 may further include devices and terminals, respectively. Taking the client 11A as an example, the device 110A and the terminal 111A may be included, as shown in FIG. 2.

 The terminal 111A may be a terminal device of a type such as a computer, a notebook computer, a laptop computer, a PDA, a mobile phone, or the like. It may include an output device 111a, an input device 111b, a processor 111c, and a communication module (not shown), wherein the communication module may perform data transmission by a wired manner, such as USB, or wireless, such as Bluetooth, in the field. The skilled person can understand that the communication module of the present invention is not limited to the above manner, and may be transmitted by any other known communication method.

 FIG. 6 shows the flow of the motion interaction method of the present invention. The operation of the motion interactive system 10 of the present invention will now be described in detail with reference to Figs. 2 and 6.

 First, proceeding to step 100, the user can select the motion interaction scene identifier to be entered in the plurality of motion interaction scene identifier lists outputted by the output device 111b of the terminal 111A through the input device of the terminal 111A, and the motion interaction scene provided by the terminal 111A. The user enters the selected motion interaction scenario; wherein the motion interaction scenario may be a virtual motion interaction scenario, or may be a motion interaction performed in a real motion scenario;

The terminal 111A may further include a storage device 111d for storing related one or more motion interaction scenarios. The motion interaction scenario is used to provide a motion interaction interface display, multimedia playback, and motion interaction. Control and other functions;

 The motion interaction scene may include, but is not limited to, skipping, running, push-ups, golf, table tennis, etc., as shown in FIG. 3; wherein the output device 111a may be a display, such as a liquid crystal display, or an audio output device, for example The number and name of the interactive interaction scene are output by voice; the user can select the motion interaction scene identifier to be entered by the input device 111b, for example, "1, skipping rope";

 Next, in step 101, the device 110A identifies the action type of the user in the selected and entered sports interactive scene and extracts action related information;

 Wherein, device 110A can include sensor 110a, as shown in FIG. It can be one or a combination of a variety of sensors known in the art including, but not limited to, accelerometers, gyroscopes, angular velocity sensors, height sensors, infrared sensors, image sensors, etc., for detecting motion related signals of the user. The sensor 110a in the present invention is preferably a 3D acceleration sensor for sampling a series of motion signals in different directions (three axes);

 And a microprocessor 110b for receiving an action signal transmitted from the sensor 110a and acquiring motion information of the user according to the action signal transmitted by the sensor 110a, wherein the motion information may include the action type and/or action related information of the user;

 The microprocessor 110b further includes a motion recognition unit (not shown), and may first perform motion recognition processing according to the motion signal transmitted by the sensor 110a; wherein the motion recognition processing may first perform feature extraction and the like on the detected motion signal. Then, the microprocessor 110b performs action classification processing according to the action classification algorithm, thereby identifying the type of action of the user. The action classification algorithm may use a classification algorithm well known in the art;

 It can be known that, for different motion interaction scenarios, different classification algorithms may be used for motion classification, such as a Gaussian mixture model, a Bayesian network model, an artificial neural network model, etc., which are well known in the art, and may also be adopted for different motion interaction scenarios. The same classification algorithm performs action classification by setting the same or different classification algorithm parameters;

 Preferably, the device 110 further includes a storage device 110c for storing a motion recognition algorithm model corresponding to each motion interaction scenario, where the motion recognition algorithm model may further include an action classification algorithm and/or an action classification corresponding to each motion interaction scenario. Algorithm parameters, taking the Gaussian mixture model as an example, the parameters that can be set include, the mixed weight value, the mean vector, and the covariance matrix;

Those skilled in the art will appreciate that various motion interactions can be set by training various different motion models. The appropriate classification method in the scene, taking the Gaussian classification algorithm as an example. In the skipping motion scene, the motion is classified into, for example, static, one-legged jump, two-legged jump, fast jump, very slow jump, etc.; In the sports scene, the Gaussian model can be trained to classify the action as, for example, swing, hitting, etc. In the yoga scene, the Gaussian model can be trained to classify the action as raising the hand, stretching the arm, stretching the leg, lowering the waist, etc.; In the ball scene, the action is classified into, for example, a ball cutting, an upper spin ball, a lower spin ball, a push ball, a lifting ball, a ball attack, etc. by training a Gaussian model;

 In particular, in a sports scene in which an action standard level can be set as an interactive scene rule, such as a sports scene such as golf or yoga, the Gaussian model can be further trained to classify one type of action into a standard level 1 action and a standard level 2 action. , standard level 3 action, etc.;

 The microprocessor nob can search for the corresponding motion recognition algorithm model according to the motion interaction scene selected by the user to perform motion recognition processing, thereby accurately identifying the type of motion of the user;

 Preferably, the motion recognition algorithm model of the embodiment of the present invention may further include a placement location information, and the device 110 prompts the user to place the device to the corresponding location according to the placement location information corresponding to the motion recognition algorithm model stored in the storage device 110c. In order to accurately monitor the user's actions in the motion interaction scene of their choice. For example, if the user selects a push-up interaction scene, the user may be prompted to wear the device to the waist; if the user selects an interactive scene such as a ball, the user may be prompted to wear the device to, for example, the wrist; if the user selects to run The interactive scene may prompt the user to place the device, for example, on the ankle;

 Preferably, after identifying the type of action of the user, the microprocessor 110b may further extract action related information through a certain algorithm according to signals detected by different sensors, which may include motion duration, motion intensity, and motion speed in the field. , frequency, angle, angular velocity, direction, etc.;

 Then, proceeding to step 102, through the terminal 111A, under the coordinated communication of the interaction server 12, the acquired user motion information is transmitted to other clients in the same motion interaction scenario, such as the client 11B-11C; wherein, each client The transmission of motion information can be performed between the coordinated communication by the interaction server 12. For example, the client 11A-11C can upload the respective motion information to the interaction server 12, collect it by the interaction server 12, and then transmit the motion information to other clients, such as the client.

11A-11C; or the client 11A-11C can also perform point-to-point data transmission through the interactive server 12;

Next, in step 103, an output device in the client, such as the output device 11 la, outputs the motion The action type and/or action related information of the related client user in the interaction scenario. The related client may be a client to which other users closely watched by the local client user. For example, in the running scene, the related information of the top three running clients and/or the related clients located in the front and back of the user may be displayed;

 Relevant action related information may be output according to rules of each motion interaction scenario. For example, according to the action frequency calculation, the number of user skipping ropes can be output in the skipping rope scene; according to the motion duration and the motion speed calculation, the number of kilometers the user runs can be output in the running scene; according to the motion frequency calculation, the user can be outputted in the push-up scene The number of push-ups; in the golf scene, the angle, intensity, direction, etc. of the hitting ball can be output according to the motion signal of the sensor, or the standard level of the user's swing and the action of hitting the ball; Output the direction, angle, intensity, speed, etc. of the user's shot, and further output whether the hitting user receives the ball and/or concedes the ball. As described above, the above action related information can be output by the displayed mode or audio;

 Preferably, the interaction server 12 can also store one or more motion interaction scenarios and provide them to the terminal 111A through the communication network 13; for example, it can be transmitted to the terminal 111A through the network 13 according to the request sent by the terminal 111A;

 The client 11A-11C can log in to the interactive server 12 through the communication network 13, and can then access the motion interaction scenario in the interaction server 12 through the WEB browser, or download and run the motion interaction scenario software in the interaction server 12 through the communication network 13. Those skilled in the art can know that any client in the network can serve as an interactive server 12 to provide motion interaction scenarios to other clients, such as point-to-point connections;

 Preferably, the interaction server 12 is further configured to store a motion recognition algorithm model database, including a motion recognition algorithm model corresponding to each motion interaction scenario, and provide the same to the client 11A; for example, the client 11A-11C may send the acquisition to the interaction server as needed. The action recognition algorithm model corresponding to the motion interaction scenario requests the interaction server 12 to send the motion recognition algorithm model corresponding to the motion interaction scenario to the corresponding client, and the client identifies the user action according to the downloaded motion recognition algorithm model. The motion recognition algorithm model is as described above and will not be described again.

Preferably, the interaction server 12 can also group the motion interaction scenarios. As shown in FIG. 4, the capacity of each group of users can be set according to the number of users required for the motion interaction scenario. After the user of the client 11A-11C selects the motion interaction scene identifier, the interaction server 12 can provide each client 11A-11C. The group information in the motion interaction scenario and the related user information in the group are sent; the user selects the motion interaction scene group to enter through the terminal according to the group information sent by the interaction server 12 and the related user information in the group, for example, the terminal 111A.

 Further, in each of the motion interaction scenarios, respective motion interaction scene rules may be set, and the winning and losing results are evaluated according to the motion interaction scene rules according to the extracted motion related information. For example, in a skipping rope scenario, a user who first reaches 100 skipping ropes can be set as a winner, and according to the number of skipped ropes of each extracted user, when one of the users reaches 100, the user can be the first name; In the dance mat scene, the user's stepping rhythm and direction can be determined according to the footstep rhythm and direction set by the dance mat scene; in the scene where table tennis can be played in real time, the number of catching and conceding the ball can be set to determine the winning or losing; In golf, yoga, etc., you can set the level of the action specification to determine whether to win or lose. Each client 11A-11C can also judge the interaction win and win and output the result according to the motion interaction rules.

 Preferably, the motion recognition algorithm model in the motion interaction system 10 of the present invention further includes a cheat action classification algorithm and/or a cheat action algorithm parameter to identify the cheating action. For example, the user may pretend to exercise only by shaking device 110A in the motion interaction scene that has entered rather than actually exercising. The cheat action model can be trained, including, for example, the sway action 1 model, the sway action 2 model, etc., to identify various types of cheating actions.

 Preferably, when the client 11A recognizes that the action of the user is a cheating action, the corresponding information is output according to the rules of the different interactive scenes, for example, the cheat action is not output or the warning message is output.

 It should be noted that the client 11A may also include only the terminal device, as shown in FIG. The sensor 110a' can detect the motion related signal of the user and then transmit it to the processor 110c' for motion recognition processing. Components in Figure 5 that have the same or similar functions as in Figure 2 are not described again.

 The following describes the flow of the motion interaction method of the present invention by taking a table tennis interactive scene as an example.

 Initially, after the user selects the table tennis interaction scene, the client may prompt the user to wear the device 110 on the wrist;

 Then, in step 200, the user enters the ping-pong scene provided by the interactive server, and the client outputs the group information in the ping-pong ball scene and the related information in the group;

 Next, in step 201, the user selects a table tennis scene group to enter;

Step 202: The user starts to move, and the client detects the motion signal of the wrist part and identifies the algorithm model according to the corresponding action, for example, the action identifier stored locally on the client or downloaded from the interaction server. The method model identifies the action type of the user and extracts the action related information according to the interaction scene rule, and sends the action related information to the interaction server; wherein the action related information that can be extracted includes the direction, angle, intensity, speed, etc. of the user hitting the ball, thereby combining according to the information Identifying the type of action of the user to accurately determine the user's action, and calculating the direction, speed, drop point, etc. of the table tennis ball, and further determining whether the hit user receives the ball and/or concedes the ball;

 Step 203: After receiving the action type and/or action related information, the interaction server sends the action type and/or the action related information to other clients in the same scenario or in the same scenario group.

 Step 204: According to the motion interaction rule of the ping-pong scene, the client displays the action type and/or action-related information of the related client user, for example, user A catches 1 ball, user B loses 1 ball, etc.; returns to step 202;

 Assume that the user who sets the motion interaction rule to the first time reaches 11 points is the winner, and the client of the user who first reaches 11 points displays the victory message, and the motion interaction scene ends.

 The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It is considered as the scope of protection of the present invention.

Claims

Claim  1. A motion interaction server, comprising:  Means for providing motion interaction scene data to the client;  Means for coordinating the transmission of the motion interaction scene data between a plurality of clients.  2. The sports interaction server according to claim 1, wherein:  The motion interaction scene data includes at least motion information generated by the client; and further includes one of a motion recognition algorithm model, a motion interaction scene, a motion interaction scene identifier list, or a combination thereof.  3. A sports interaction client, including:  Means for acquiring motion information of the user according to an action signal of the user;  Means for outputting a list of alternative motion interaction scene identifiers;  Means for storing a motion interaction scenario corresponding to the motion interaction scene identifier list;  Means for the user to select an identifier of the motion interaction scene to be entered by the user through the motion interaction scene identification list;  Means for transmitting the user's motion information to the client in the same motion interaction scenario through coordinated communication by the server;  The means for outputting the list of selectable motion interaction scenes is also used to output the motion information of the associated client.  4. The sports interaction client according to claim 3, wherein:  The device for storing the motion interaction scenario corresponding to the motion interaction scenario identifier list is further configured to store a motion recognition algorithm model corresponding to each of the motion interaction scenarios.  A motion interaction system, comprising: a plurality of clients and at least one interaction server, wherein: each of the clients is configured to output a motion interaction scene identifier list, so that the user selects a candidate from the motion interaction scene identifier list An identifier of the entered motion interaction scenario, and providing the motion interaction scenario, and motion information for acquiring the action performed by the user in the selected motion interaction scenario; and  Each of the clients in the same motion interaction scenario performs the transmission of the motion information through coordinated communication of the interaction server, and outputs the motion information of the corresponding client in the same motion interaction scenario; The interaction server also provides each client with a list of selectable motion interaction scene identifiers and phases. The appropriate motion interaction scenario.  6. The exercise interaction system of claim 5, wherein:  Each of the related clients is also used to output a motion interaction result.  7. The athletic interaction system of claim 5, wherein:  The interaction server is further configured to store a motion recognition algorithm model corresponding to the motion interaction scenario, and provide the motion recognition algorithm model to the client.  8. The athletic interaction system of claim 5, wherein:  The interaction server further provides the mobile interaction scene group and the corresponding scene group related information to the client based on each of the motion interaction scenarios.  9. The athletic interaction system of claim 7, wherein:  The motion recognition algorithm model includes motion classification algorithm parameters.  10. The action interaction system according to claim 9, wherein:  The motion recognition algorithm model also includes an action classification algorithm.  11. The athletic interaction system of claim 9 wherein:  The motion recognition algorithm model includes a cheat action classification algorithm parameter.  12. The athletic interaction system of claim 11 wherein:  The motion recognition algorithm model further includes a cheat action classification algorithm.  13. A method of motion interaction, including:  Providing a plurality of motion interaction scenarios and an optional motion interaction scenario identifier list, wherein the user selects a motion interaction scenario to be entered in the motion interaction scenario;  Providing a corresponding motion interaction scenario to the user according to the motion interaction scenario identifier selected by the user; acquiring action information of the user in the selected motion interaction scenario;  Each of the clients in the same motion interaction scenario performs the transmission of the action information; and the related client in the same motion interaction scenario outputs the action information of the relevant client user in the motion interaction scenario.  14. The method of motion interaction according to claim 13, wherein:  Each of the clients also outputs a motion interaction result.  15. The method of motion interaction according to claim 13, wherein: The motion information of the user includes a type of motion of the user; And providing a corresponding motion recognition algorithm model to the client according to the motion interaction scenario; the client identifying the action type of the user according to the motion recognition algorithm model.