HK1221045B - Method for realizing data interaction between intelligent display terminal and somatosensory device - Google Patents

Description

A method and device for realizing data interaction between intelligent display terminal and somatosensory device

Technical Field

This application relates to the field of intelligent display terminal technology, and more specifically, to a method for implementing data interaction between an intelligent display terminal and a somatosensory device. This application also relates to an apparatus for implementing data interaction between an intelligent display terminal and a somatosensory device, somatosensory device middleware for implementing data interaction between an intelligent display terminal and a somatosensory device, and a software development kit for implementing data interaction between an intelligent display terminal and a somatosensory device.

Background Art

Smart TV terminals refer to devices such as smart TVs or TV boxes (such as Apple TV, Xiaomi Box, LeTV TV, and Alibaba's Tmall Magic Box) that are equipped with smart operating systems (such as iOS, Android, and Alibaba TV OS). While users enjoy regular TV content, they can also install and uninstall various applications and games to expand and upgrade the TV's functionality. With the rapid development of smart TVs in recent years, the variety of applications used in smart TVs has also grown rapidly, especially in the diversification and enrichment of games. Furthermore, to meet users' gaming needs on smart TVs, a variety of peripherals specifically adapted for smart TV applications have emerged, such as controllers, remote controls, mice, and motion sensors (devices that detect user movements or status through built-in sensors).

Under the existing technology, the operating system of smart TV terminals is derived from the operating system of smart phones. Generally speaking, the operating system of smart TV terminals only supports one set of motion sensing devices. In order to realize multiplayer games, multiplayer motion sensing games are first developed, which require their own specific game controllers (for example: multiplayer games of Sports Plus and Subor). The game distinguishes different game controllers by directly reading the data with different tags transmitted by their game controllers, thereby supporting the multiplayer mode of the game.

The above-mentioned prior art methods for managing motion sensing devices on smart TV terminals have significant drawbacks. Under the existing technology, software or games installed on smart TV terminals can only interact with one or several specific motion sensing devices, resulting in poor compatibility. Furthermore, software developers can only develop one or a series of games based on a specific motion sensing device, and user-developed games are only compatible with specific motion sensing devices, resulting in inconsistent interfaces. Users, upon purchasing a motion sensing device, can only play the one or a series of games corresponding to that device. Once a game becomes outdated, users must purchase a new device to play another game, which is then discarded, resulting in a waste of resources.

Summary of the Invention

This application provides a method for enabling data interaction between a smart display terminal and a somatosensory device to address the issues of poor compatibility and inconsistent interfaces in existing methods. This application also provides an apparatus for enabling data interaction between a smart display terminal and a somatosensory device, somatosensory device middleware for enabling data interaction between a smart display terminal and a somatosensory device, and a software development kit for enabling data interaction between a smart display terminal and a somatosensory device.

This application provides a method for realizing data interaction between an intelligent display terminal and a somatosensory device, comprising:

Read device data uploaded by the somatosensory device based on the smart display terminal;

Converting the device data into standardized somatosensory data;

The application program reads the standardized body sensing data.

Optionally, the smart TV terminal reads the device data uploaded by the somatosensory device and receives the data through an interface driver module;

Wherein, the interface driver module includes: USB driver, WIFI driver and/or Bluetooth driver.

Optionally, converting the device data into standardized somatosensory data includes:

Obtaining a data format standard adopted by a somatosensory device corresponding to the device data;

Parsing the device data according to the data format standard to generate somatosensory data;

The somatosensory data is converted into standardized somatosensory data based on a preset conversion algorithm.

Optionally, the conversion algorithm is formulated for the somatosensory data format based on the relationship between the somatosensory data format and the standardized somatosensory data format.

Optionally, before the application reads the standardized somatosensory data, the application may include:

Packaging the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet;

The motion sensing data packet is sent to the application.

Optionally, the application reads the standardized somatosensory data, including:

The application receives the somatosensory data packet;

The application parses the body sensing data packet according to the private protocol to obtain standardized body sensing data.

Optionally, the application receives the somatosensory data packet, specifically, the application receives the somatosensory data packet through an application programming interface.

Optionally, the intelligent display terminal sends instructions to the somatosensory device through the application programming interface.

The present application further provides a device for realizing data interaction between an intelligent display terminal and a somatosensory device, comprising:

A device data reading unit, used to read device data uploaded by the somatosensory device based on the smart display terminal;

A data conversion unit, configured to convert the device data into standardized somatosensory data;

The data calling unit is used to enable the application to read the standardized body sensing data.

Optionally, the data conversion unit includes:

A data standard acquisition subunit, configured to acquire a data format standard adopted by a somatosensory device corresponding to the device data;

a device data parsing subunit, configured to parse the device data according to the data format standard to generate somatosensory data;

The standardized somatosensory data generating subunit is configured to convert the somatosensory data into standardized somatosensory data based on a preset conversion algorithm.

Optionally, the device for realizing data interaction between the intelligent display terminal and the somatosensory device includes:

a data packaging unit, configured to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet;

A data sending unit is used to send the somatosensory data packet to an application.

Optionally, the data calling unit includes:

A data receiving subunit, wherein the application receives the somatosensory data packet;

The data parsing subunit is configured such that the application parses the somatosensory data packet according to the private protocol to obtain standardized device data.

The present application also provides a somatosensory device middleware for realizing data interaction between an intelligent display terminal and a somatosensory device, comprising: a driver docking unit, a data standardization unit, a somatosensory device management unit, and a data communication unit;

The driver docking unit is used to read the device data uploaded by the somatosensory device based on the smart TV terminal, and send commands downloaded by the application;

The data standardization unit is used to convert the device data into standardized body sensing data;

The somatosensory device management unit is used to manage the somatosensory devices connected to the intelligent display terminal and the device data corresponding to the somatosensory devices;

The data communication unit performs data communication with the application based on inter-process communication.

Optionally, the data standardization unit includes:

A data format standard acquisition subunit, configured to acquire a data format standard adopted by a somatosensory device corresponding to the device data;

a data parsing subunit, configured to parse the device data according to the data format standard to generate somatosensory data;

The data conversion subunit is used to convert the somatosensory data into standardized somatosensory data based on a preset conversion algorithm.

Optionally, the somatosensory device management unit includes:

an analysis subunit, configured to analyze the standardized somatosensory data to obtain device feature information;

The storage subunit is configured to store the standardized somatosensory data in a data storage area corresponding to the device characteristic information.

Optionally, the data communication unit includes:

a somatosensory data packet generating subunit, configured to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet;

a somatosensory data packet sending subunit, configured to send the somatosensory data packet to the application; and

An instruction data packet receiving subunit, configured to receive an instruction data packet sent by the application;

The instruction data packet parsing subunit is used to parse the instruction data packet into standardized instruction data according to a private protocol.

Optionally, the inter-process communication includes: a message queue, a shared memory or a Socket.

The present application also provides a software development kit for realizing data interaction between an intelligent display terminal and a somatosensory device, comprising: a data communication unit, an event management unit, a somatosensory device management unit, and an interface encapsulation unit;

The data communication unit performs data communication with the somatosensory device middleware based on inter-process communication;

The event management unit implements operation control of the application based on various preset somatosensory events;

A somatosensory device management unit, configured to manage the somatosensory devices connected to the intelligent display terminal and device data corresponding to the somatosensory devices;

The interface providing unit is used to provide an application program interface to the application program.

Optionally, the data communication unit includes:

A somatosensory data packet receiving subunit, configured to receive a somatosensory data packet sent by the somatosensory device middleware;

A somatosensory data packet parsing subunit, configured to parse the somatosensory data packet according to a private protocol to obtain standardized somatosensory data;

An instruction data packet generating subunit, configured to package the instruction data according to a private protocol to obtain an instruction data packet;

The instruction data packet sending subunit is used to send the instruction data packet to the somatosensory device middleware.

Optionally, the somatosensory device management unit includes:

A somatosensory event parsing subunit, configured to parse the somatosensory event to obtain device feature information and standardized somatosensory data;

a somatosensory device acquisition subunit, configured to acquire, from a somatosensory device list, a somatosensory device corresponding to the device characteristic information according to the device characteristic information;

The data storage subunit is used to store the standardized somatosensory data in a data storage area corresponding to the device characteristic information.

Optionally, the interface providing unit includes:

An interface implementation subunit, which implements the application programming interface based on the somatosensory device management unit;

The interface encapsulation subunit is used to package and compile the application programming interface into a library file and output it.

Compared with the prior art, this application has the following advantages:

The method for realizing data interaction between the intelligent display terminal and the somatosensory device provided in this application has few limitations and saves resources.

The method provided in the present application for realizing data interaction between a smart display terminal and a somatosensory device includes: reading device data uploaded by the somatosensory device based on the smart display terminal; converting the device data into standardized somatosensory data; and reading the standardized somatosensory data by an application.

The above-mentioned method provided in the present application can convert the device data collected by somatosensory devices produced by different device manufacturers into standardized somatosensory data with a unified data format standard, so that the application installed on the smart display terminal can be compatible with somatosensory devices with different data format standards produced by different device manufacturers, with good compatibility; in addition, the above-mentioned method provided in the present application provides a unified interface to software developers based on the standardized somatosensory data.

The present application further provides a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device, comprising a driver docking unit, a data standardization unit, a somatosensory device management unit, and a data communication unit; the driver docking unit is used to read device data uploaded by the somatosensory device based on the smart TV terminal, and to send commands downloaded by the application; the data standardization unit is used to convert the device data into standardized somatosensory data; the somatosensory device management unit is used to manage the somatosensory device connected to the smart display terminal and the device data corresponding to the somatosensory device; the data communication unit performs data communication with the application based on inter-process communication.

The somatosensory device middleware provided in this application converts the device data uploaded by somatosensory devices with different data format standards produced by different device manufacturers into standardized somatosensory data with a unified data format standard for application calls, so that the application can be compatible with somatosensory devices with different data format standards produced by different device manufacturers, and has good compatibility.

The present application also provides a software development toolkit for realizing data interaction between a smart display terminal and a somatosensory device, comprising a data communication unit, an event management unit, a somatosensory device management unit, and an interface encapsulation unit; the data communication unit performs data communication with the somatosensory device middleware based on inter-process communication; the event management unit realizes operation control of the application based on various preset somatosensory events; the somatosensory device management unit is used to manage the somatosensory device connected to the smart display terminal and the device data corresponding to the somatosensory device; and the interface providing unit is used to provide an application interface to the application.

The software development kit provided in this application provides software developers with a unified application programming interface based on standardized somatosensory data with a unified data format standard, thereby avoiding the need for software developers to develop corresponding software or games for the same software or game for different data format standards, thereby saving resources.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a processing flow chart of a method for realizing data interaction between a smart display terminal and a somatosensory device provided in the first embodiment of the present application.

FIG2 is a schematic diagram of a device for realizing data interaction between a smart display terminal and a somatosensory device provided in a second embodiment of the present application.

FIG3 is a schematic diagram of a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device, provided in a third embodiment of the present application.

FIG4 is a schematic diagram of a software development kit for realizing data interaction between a smart display terminal and a somatosensory device, provided in a fourth embodiment of the present application.

DETAILED DESCRIPTION

The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

The present application provides a method for realizing data interaction between a smart display terminal and a somatosensory device. The present application also provides an apparatus for realizing data interaction between a smart display terminal and a somatosensory device, a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device, and a software development kit for realizing data interaction between a smart display terminal and a somatosensory device.

The following describes in detail the method for realizing data interaction between the intelligent display terminal and the somatosensory device of the present application with reference to the accompanying drawings and embodiments.

Example 1

1 , which shows a processing flow chart of a method for implementing data interaction between an intelligent display terminal and a somatosensory device provided in a first embodiment of the present application.

The method for implementing data interaction between an intelligent display terminal and a somatosensory device described in this embodiment includes the following steps:

S101: Read device data uploaded by the somatosensory device based on the smart display terminal.

The smart display terminal refers to a device terminal used for playback and display functions, and can install and run smart operating systems and applications, such as: smart TV terminals and PCs; the somatosensory devices include somatosensory handles, smart mobile terminals and wearable devices; the device data includes data collected by sensors built into the somatosensory devices. In addition, the device data also includes device information data (device ID, device manufacturer ID, device name, sensor model, type, etc. of the somatosensory device).

In this embodiment, a smart TV terminal is used as an example for explanation. Other types of smart display terminals are similar to smart TV terminals. The following method for realizing data interaction between a smart display terminal and a somatosensory device is provided with reference to this embodiment, which is not listed one by one in this embodiment.

Before the smart display terminal reads the device data uploaded by the somatosensory device in this step, the process of the somatosensory device uploading the device data includes the following two steps: 1) the somatosensory device collects device data, and the somatosensory device collects the device data of the current somatosensory device through the built-in sensor; 2) the somatosensory device uploads the device data to the smart TV terminal.

After the somatosensory device uploads the device data, the smart display terminal reads the device data uploaded by the somatosensory device. The smart TV terminal reads the device data uploaded by the somatosensory device via an interface driver module; wherein the interface driver module includes: a USB driver, a WIFI driver, and a Bluetooth driver.

After completing the step of reading the device data uploaded by the somatosensory device based on the intelligent display terminal, the step of converting the device data into standardized somatosensory data is performed.

S102: Convert the device data into standardized body sensing data.

The converting of device data into standardized somatosensory data includes:

1) Obtaining the data format standard adopted by the somatosensory device corresponding to the device data;

The device data includes data collected by the sensors built into the motion sensing device and device information data, including the device ID, manufacturer ID, device name, and sensor model and type of the motion sensing device. Motion sensing devices with different device IDs, manufacturer IDs, device names, and sensor models and types may have different data format standards for the corresponding device data.

In this step, a data format standard corresponding to the somatosensory device provided by the somatosensory device manufacturer or a data format standard corresponding to the sensor built into the somatosensory device provided by the sensor manufacturer is obtained in advance, and the data format standard is stored.

2) parsing the device data according to the data format standard to generate somatosensory data;

The somatosensory data refers to data that can characterize the motion characteristics of the somatosensory device; the purpose of this step is to convert the data collected by the built-in sensor of the somatosensory device in the device data into somatosensory data that can characterize the motion characteristics of the somatosensory device or the user's somatosensory movements through analytical calculation.

3) Converting the somatosensory data into standardized somatosensory data based on a preset conversion algorithm.

The standardized data refers to the somatosensory data that characterizes the motion characteristics of the somatosensory device or the user's somatosensory movements based on a unified standard; the purpose of this step is to convert somatosensory data with different data format standards into somatosensory data with a unified data format standard, that is, standardized somatosensory data; the conversion algorithm is formulated based on the relationship between the data format standard of the somatosensory data and the data format standard of the standardized somatosensory data.

After the device data is converted into standardized somatosensory data according to the above three steps, the standardized somatosensory data is sent to the application.

S103: Pack the standardized body sensing data according to a private protocol to obtain a body sensing data packet.

Before sending the standardized somatosensory data to the application, the standardized somatosensory data is packaged according to a proprietary protocol to obtain a somatosensory data packet. After obtaining the somatosensory data packet, the somatosensory data packet is sent to the application. The packaging of the standardized somatosensory data according to the proprietary protocol described in this step is not the focus of this embodiment and is not detailed here.

S104: Send the somatosensory data packet to the application.

The somatosensory data packet is sent to the application program based on an inter-process communication method, which includes a message queue, shared memory, and a socket. In addition, the somatosensory data packet can be sent to the application program using methods other than this embodiment, which are not limited here.

S105: The application receives the somatosensory data packet.

The application receives the somatosensory data packet sent through the inter-process communication mode, and after receiving the somatosensory data packet, parses the somatosensory data packet into standardized device data.

S106: The application parses the body sensing data packet according to a private protocol to obtain standardized body sensing data.

For data communication between the somatosensory device middleware and the application based on inter-process communication, in order to ensure the consistency and validity of data between the somatosensory device middleware and the application, a unified data standard is reached between the somatosensory device middleware and the application, namely: a private protocol.

Establishing a private protocol between the somatosensory device middleware and the application to ensure that the application can identify data information sent by the somatosensory device middleware to the application, and that the somatosensory device middleware can identify instruction information sent by the application to the somatosensory device middleware;

In addition, a private protocol is established between the somatosensory device middleware and the application to ensure that the application can obtain the complete information contained in the data information based on the private protocol, and the somatosensory device middleware can obtain the complete information contained in the instruction information based on the private protocol, thereby avoiding data loss.

After the application program in step S105 receives the somatosensory data packet, the application program parses the somatosensory data packet according to the private protocol to obtain standardized device data.

Example 2

In the above-mentioned embodiment, a method for realizing data interaction between an intelligent display terminal and a somatosensory device is provided. Correspondingly, the present application also provides an apparatus for realizing data interaction between an intelligent display terminal and a somatosensory device.

2 , which shows a schematic diagram of a device for implementing data interaction between a smart display terminal and a somatosensory device according to a second embodiment of the present application.

Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment. The device embodiment described below is only illustrative.

The device for realizing data interaction between the intelligent display terminal and the somatosensory device described in this application includes:

The device data reading unit 201 is used to read the device data uploaded by the somatosensory device based on the intelligent display terminal;

A data conversion unit 202 is used to convert the device data into standardized body sensing data;

The data calling unit 205 is used to enable the application to read the standardized body sensing data.

Optionally, the data conversion unit 202 includes:

The data standard acquisition subunit 202-1 is used to acquire the data format standard adopted by the somatosensory device corresponding to the device data;

The device data parsing subunit 202-1 is configured to parse the device data according to the data format standard to generate somatosensory data;

The standardized body sensing data generating subunit 202 - 3 is configured to convert the body sensing data into standardized body sensing data based on a preset conversion algorithm.

Optionally, the device for realizing data interaction between the intelligent display terminal and the somatosensory device includes:

A data packaging unit 203 is configured to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet;

The data sending unit 204 is configured to send the motion sensing data packet to an application.

Optionally, the data calling unit 205 includes:

The data receiving subunit 205-1, the application receives the somatosensory data packet;

Data parsing subunit 205 - 2 , the application parses the somatosensory data packet according to the private protocol to obtain standardized device data.

Example 3

In the above-mentioned embodiment, a method for realizing data interaction between a smart display terminal and a somatosensory device is provided. In addition, the present application also provides a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device, which is used in the above-mentioned method for realizing data interaction between a smart display terminal and a somatosensory device to realize the step of converting the device data into standardized somatosensory data; and to realize the step of the smart display terminal sending instructions to the somatosensory device through the application programming interface.

3 , which shows a schematic diagram of a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device, provided in a third embodiment of the present application.

In this application, the function of the somatosensory device middleware is to convert the device data uploaded by the somatosensory device into standardized somatosensory data with a unified data format. Without the somatosensory device middleware, even if the application can directly read the device data uploaded by the somatosensory device based on the smart display device, it cannot recognize the device data with different data format standards, let alone call it. The application cannot exchange data with the somatosensory device, and the method for realizing data interaction between the smart display terminal and the somatosensory device cannot be realized. In summary, the somatosensory device middleware plays an indispensable role in the implementation of the method for realizing data interaction between the smart display terminal and the somatosensory device.

The somatosensory device middleware used for realizing data interaction between the intelligent display terminal and the somatosensory device includes: a driver docking unit 301 , a data standardization unit 302 , a somatosensory device management unit 303 and a data communication unit 304 .

The driver docking unit 301 is used to read the device data uploaded by the somatosensory device based on the smart TV terminal and send commands downloaded by the application;

The device data includes data collected by the sensors built into the motion sensing device and device information data, including the device ID, manufacturer ID, device name, and sensor model and type of the motion sensing device. Motion sensing devices with different device IDs, manufacturer IDs, device names, and sensor models and types may have different data format standards for the corresponding device data.

The driver docking unit 301 acts as an interface between the somatosensory device and the intelligent display terminal. The interface is implemented through an interface driver, which includes: USB driver, WIFI driver and Bluetooth driver; in addition, it can also include other drivers, such as infrared driver, etc., which are not limited here.

If the motion sensing device is connected to the smart display terminal via a USB interface, data communication between the motion sensing device and the smart display terminal is provided via a USB driver. In this embodiment, the motion sensing device uploads its collected device data to the smart display terminal via the USB interface. Furthermore, the process of uploading device data to the smart display terminal via the Wi-Fi driver and the Bluetooth driver is similar to the process of uploading device data to the smart display terminal via the USB interface, and is not further described here.

The data standardization unit 302 is used to convert the device data into standardized body sensing data;

After receiving device data uploaded by the somatosensory device via the driver docking unit 301, the somatosensory device middleware reads the device data via the data standardization unit 302 and converts the device data into standardized somatosensory data. Standardized somatosensory data refers to data generated by converting device data of different data formats corresponding to different somatosensory devices into a unified data format.

The data standardization unit 302 includes: a data format standard acquisition subunit 302-1, a data parsing subunit 302-2 and a data conversion subunit 302-3;

The data format standard acquisition subunit 302-1 is used to acquire the data format standard adopted by the somatosensory device corresponding to the device data;

For the data standardization unit 302 to convert the device data uploaded by the somatosensory devices into standardized somatosensory data, a crucial prerequisite is that the data standardization unit 302 first recognizes the device data uploaded by different somatosensory devices in different data format standards. In this embodiment, the data format standard acquisition subunit 302-1 within the data standardization unit 302 acquires and stores the data format standards corresponding to somatosensory devices provided by different somatosensory device manufacturers. Only after acquiring the data format standards corresponding to the somatosensory devices provided by these different somatosensory device manufacturers can the data standardization unit 302 recognize the device data uploaded by the somatosensory devices, completing the data preparation prerequisite for converting the device data into standardized somatosensory data.

The data parsing subunit 302-2 is configured to parse the device data according to the data format standard to generate somatosensory data;

The somatosensory data refers to data that can characterize the movement characteristics of the somatosensory device (i.e., the user's somatosensory movements). The data parsing subunit 302-2 parses the device data based on the data format standard corresponding to the somatosensory device provided by the somatosensory device manufacturer obtained by the above-mentioned data format standard acquisition subunit 302-1, and converts the parsed device data into somatosensory data through analysis and calculation. It should be noted that here, the somatosensory data and the data format standard correspond one to one, that is, one somatosensory device corresponds to somatosensory data of one data format standard, and the data format standards of the somatosensory data corresponding to different somatosensory devices are different from each other.

The data conversion subunit 302-3 is configured to convert the somatosensory data into standardized somatosensory data based on a preset conversion algorithm. Standardized somatosensory data refers to somatosensory data that characterizes the motion characteristics of a somatosensory device or the somatosensory actions of a user based on a unified standard. Specifically, the somatosensory data obtained by the data parsing subunit 302-2 in different data formats is converted into somatosensory data based on a unified standard.

In this embodiment, the data conversion sub-unit 302 - 3 converts the somatosensory data into somatosensory data of a unified standard, that is, standardized somatosensory data, based on a preset conversion algorithm.

The conversion algorithm is formulated based on the relationship between the data format standard of the somatosensory data and the data format standard of standardized somatosensory data.

The somatosensory device management unit 303 is used to manage the somatosensory device connected to the intelligent display terminal or the device data corresponding to the somatosensory device;

The somatosensory device management unit 303 includes: an analysis subunit 303-1 and a storage subunit 303-2.

The parsing subunit 303-1 performs parsing based on the standardized somatosensory data to obtain device feature information;

The device data includes data collected by the sensors built into the motion sensing device and device information data. Furthermore, the standardized motion sensing data also includes data collected by the sensors and device information data. The device information data includes the device ID, device manufacturer ID, device name, and sensor model and type of the motion sensing device. Device feature information refers to device data that can identify the motion sensing device, such as the device ID of the motion sensing device.

The storage subunit 303 - 2 is configured to store the standardized somatosensory data in a data storage area corresponding to the device characteristic information.

The storage subunit 303 - 2 is used to store different standardized somatosensory data in different data storage areas; and to keep the standardized somatosensory data stored in the data storage area corresponding to the same device characteristic information updated.

For the same somatosensory device, when new standardized somatosensory data is generated, the standardized somatosensory data stored in the data storage area is updated with the new standardized somatosensory data to achieve data update.

In this embodiment, the data storage area corresponds to the device ID of the somatosensory device, and one device ID corresponds to one data storage area. The data storage area can be a storage area in a database or other data storage unit, which is not limited here.

The data communication unit 304 performs data communication with the application based on inter-process communication.

The motion sensing device middleware runs in an independent process; similarly, the application runs in an independent process. To enable data communication between the motion sensing device middleware and the application, inter-process communication (IPC) is used, thereby enabling data exchange between the two. This IPC method includes message queues, shared memory, and sockets, but other methods may also be used, which are not limited here.

The data communication unit 304 includes: a body sensing data packet generating subunit 304-1, a body sensing data packet sending subunit 304-2, an instruction data packet receiving subunit 304-3 and an instruction data packet parsing subunit 304-4;

The somatosensory data packet generating subunit 304-1 is configured to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet;

For the data communication between the somatosensory device middleware and the application based on inter-process communication, in order to ensure the consistency and validity of the data between the somatosensory device middleware and the application, a unified data standard is reached between the somatosensory device middleware and the application, namely: a private protocol.

Establishing a private protocol between the somatosensory device middleware and the application to ensure that the application can identify data information sent by the somatosensory device middleware to the application, and that the somatosensory device middleware can identify instruction information sent by the application to the somatosensory device middleware;

In addition, a private protocol is established between the somatosensory device middleware and the application to ensure that the application can obtain the complete information contained in the data information based on the private protocol, and the somatosensory device middleware can obtain the complete information contained in the instruction information based on the private protocol, thereby avoiding data loss.

After the somatosensory device middleware and the application program reach a private agreement, the somatosensory device middleware packages the standardized data according to the private agreement and generates a somatosensory data packet;

After the somatosensory device middleware obtains the somatosensory data packet, it sends the somatosensory data packet to the application program;

After receiving the somatosensory data packet, the application parses the somatosensory data packet according to the private protocol, obtains standardized somatosensory data and performs relevant operations;

Similarly, after the somatosensory device middleware and the application program reach a private agreement, the application program packages the instruction data according to the private agreement and generates an instruction data packet;

After the application obtains the instruction data packet, it sends the instruction data packet to the somatosensory device middleware;

After receiving the instruction data packet, the somatosensory device middleware parses the instruction data packet according to the private protocol to obtain standardized instruction data.

A somatosensory data packet sending subunit 304-2 is configured to send the somatosensory data packet to the application;

Please refer to the description of the above-mentioned body sensing data packet generating subunit 304-1, which will not be repeated here.

The instruction data packet receiving subunit 304-3 is used to receive the instruction data packet sent by the application;

Please refer to the description of the above-mentioned body sensing data packet generating subunit 304-1, which will not be repeated here.

The instruction data packet parsing subunit 304 - 4 is configured to parse the instruction data packet into standardized instruction data according to a proprietary protocol.

The body sensing data packet sending subunit 304 - 2 , the instruction data packet receiving subunit 304 - 3 and the instruction data packet parsing subunit 304 - 4 refer to the description of the body sensing data packet generating subunit 304 - 1 , which will not be repeated here.

Under the existing technology, applications are not compatible with somatosensory devices. For a certain model of somatosensory device provided by a certain device manufacturer, only the corresponding one or a series of games are compatible with the somatosensory device. In addition, other applications or third-party applications are not compatible with the somatosensory device. For example, Nintendo and Subor game consoles can only install game programs specially developed for the corresponding game consoles, and most game programs on the market cannot be installed or cannot be implemented. The somatosensory device middleware provided in this embodiment obtains the data format standard adopted by the device data corresponding to the somatosensory devices of different device manufacturers, and then converts the device data into somatosensory data that characterizes the motion characteristics of the somatosensory device according to the corresponding data format standard, and converts the somatosensory data into standardized somatosensory data with a unified data format according to a preset conversion algorithm, thereby unifying the data transmission format of the somatosensory device. Finally, the standardized somatosensory data is provided to the application, so that the application can be compatible with somatosensory devices produced by different manufacturers.

Example 4

In the above-mentioned embodiment, a method for realizing data interaction between a smart display terminal and a somatosensory device, and a somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device are provided, which are used in the above-mentioned method for realizing data interaction between a smart display terminal and a somatosensory device, to realize the step of converting the device data into standardized somatosensory data; and to realize the step of the smart display terminal sending instructions to the somatosensory device through the application programming interface; in addition, the present application also provides a software development kit for realizing data interaction between a smart display terminal and a somatosensory device, which is used to match the above-mentioned somatosensory device middleware for realizing data interaction between a smart display terminal and a somatosensory device.

In this application, the role of the somatosensory device middleware is to convert the device data uploaded by the somatosensory device into standardized somatosensory data with a unified data format; the role of the software development kit is to provide software developers with development standards and basis for software (applications and games) that match the somatosensory device, and provides an interface tool for the software to call the standardized somatosensory data.

4 , which shows a schematic diagram of a software development kit for implementing data interaction between a smart display terminal and a somatosensory device according to a fourth embodiment of the present application.

The software development kit includes a data communication unit 401, an event management unit 402, a somatosensory device management unit 403 and an interface encapsulation unit 404;

In this embodiment, the software development kit refers to a set of application development interfaces (APIs) for monitoring, reading, operating, and managing the somatosensory devices and the corresponding data of the somatosensory devices on the smart display terminal. The actual form provided is a number of application library files, and applications (including games) can call the application development interfaces provided by the software development kit.

It should be noted that in this embodiment, the software development kit itself does not run as an independent process, but only within the calling application process. The somatosensory device middleware runs in an independent process, and data communication between the somatosensory device middleware and the software development kit is achieved through inter-process communication (IPC), which includes message queues, shared memory, and sockets. Other implementation methods are also possible and are not limited here.

The data communication unit 401 performs data communication with the somatosensory device middleware based on inter-process communication;

The motion sensing device middleware runs in an independent process; similarly, the application runs in an independent process. To enable data communication between the motion sensing device middleware and the application, inter-process communication (IPC) is used, thereby enabling data exchange between the two. This IPC method includes message queues, shared memory, and sockets, but other methods may also be used, which are not limited here.

The data communication unit 401 includes: a body sensing data packet receiving subunit 401-1, a body sensing data packet parsing subunit 401-2, an instruction data packet generating subunit 401-3 and an instruction data packet sending subunit 401-4;

The somatosensory data packet receiving subunit 401-1 is configured to receive a somatosensory data packet sent by the somatosensory device middleware;

For the data communication between the somatosensory device middleware and the application based on inter-process communication, in order to ensure the consistency and validity of the data between the somatosensory device middleware and the application, a unified data standard is reached between the somatosensory device middleware and the application, namely: a private protocol.

Establishing a private protocol between the somatosensory device middleware and the application to ensure that the application can identify data information sent by the somatosensory device middleware to the application, and that the somatosensory device middleware can identify instruction information sent by the application to the somatosensory device middleware;

In addition, a private protocol is established between the somatosensory device middleware and the application to ensure that the application can obtain the complete information contained in the data information based on the private protocol, and the somatosensory device middleware can obtain the complete information contained in the instruction information based on the private protocol, thereby avoiding data loss.

After the somatosensory device middleware and the application program reach a private agreement, the somatosensory device middleware packages the standardized data according to the private agreement and generates a somatosensory data packet;

After the somatosensory device middleware obtains the somatosensory data packet, it sends the somatosensory data packet to the application program;

After receiving the somatosensory data packet, the application parses the somatosensory data packet according to the private protocol, obtains standardized somatosensory data and performs relevant operations;

Similarly, after the somatosensory device middleware and the application program reach a private agreement, the application program packages the instruction data according to the private agreement and generates an instruction data packet;

After the application obtains the instruction data packet, it sends the instruction data packet to the somatosensory device middleware;

After receiving the instruction data packet, the somatosensory device middleware parses the instruction data packet according to the private protocol, obtains instruction data and performs relevant operations.

The somatosensory data packet parsing subunit 401-2 is configured to parse the somatosensory data packet according to a private protocol to obtain standardized somatosensory data;

Please refer to the description of the above-mentioned body sensing data packet receiving subunit 401-1, which will not be repeated here.

The instruction data packet generating subunit 401 - 3 is used to package the instruction data according to a private protocol to obtain an instruction data packet;

Please refer to the description of the above-mentioned body sensing data packet receiving subunit 401-1, which will not be repeated here.

The instruction data packet sending subunit 401 - 4 is configured to send the instruction data packet to the somatosensory device middleware.

The body sensing data packet parsing subunit 401 - 2 , the instruction data packet generating subunit 401 - 3 and the instruction data packet sending subunit 401 - 4 refer to the description of the body sensing data packet receiving subunit 401 - 1 , which will not be repeated here.

The event management unit 402 implements operational control of the application based on various preset somatosensory events. Events are a mechanism for communication between objects. There are many types of events, and commonly used events include mouse click events, mouse movement events, and button click events. The purpose of events is to help users complete what they want to accomplish; for example:

For a mouse click event, after clicking the mouse, some action will definitely occur, such as exiting the window, refreshing the page, or submitting data. At this time, the mouse click operation triggers the mouse click event. Events are all corresponding. For a mouse click event, only after the mouse click operation is performed will the mouse click event be triggered and the methods or properties in the mouse click event be executed.

In this embodiment, the event management unit 402 implements the following steps to control the operation of the application based on various preset somatosensory events:

1) Different types of somatosensory events are defined according to the data type of the standardized somatosensory data; the somatosensory events include: key events, sensor events and/or air mouse events.

2) The somatosensory event is monitored through an event listener; if the somatosensory event is triggered, an operation response corresponding to the somatosensory event is executed to implement operational control of the application.

In addition, it should be noted that the event management unit 402 may also be implemented using event implementation methods other than those in this embodiment, such as implementation of C# events, which is not limited here.

A somatosensory device management unit 403 is configured to manage the somatosensory devices connected to the intelligent display terminal or device data corresponding to the somatosensory devices;

The somatosensory device management unit 403 includes a somatosensory event parsing subunit 403 - 1 , a somatosensory device acquiring subunit 403 - 2 and a data storage subunit 403 - 3 .

The somatosensory event parsing subunit 403-1 is used to parse the somatosensory event to obtain device feature information and standardized somatosensory data;

For various somatosensory events implemented in the event management unit 402 , if a body sense event is triggered among the somatosensory events, the somatosensory event parsing subunit 403 - 1 parses the somatosensory event and simultaneously obtains device feature information and standardized somatosensory data corresponding to the somatosensory event.

The somatosensory device acquisition subunit 403-2 is configured to acquire, from a somatosensory device list, a somatosensory device corresponding to the device characteristic information according to the device characteristic information;

The somatosensory device list refers to a somatosensory device list consisting of all somatosensory devices connected to the smart display terminal.

The above-mentioned somatosensory event parsing subunit 403-1 obtains device characteristic information and standardized somatosensory data by parsing the somatosensory event, and the somatosensory device acquisition subunit 403-2 queries and obtains the somatosensory device corresponding to the device characteristic information from the somatosensory device list based on the device characteristic information; in this embodiment, the somatosensory device acquisition subunit 403-2 queries the somatosensory device corresponding to the ID from the somatosensory device list based on the somatosensory device ID.

The data storage subunit 403 - 3 is configured to store the standardized somatosensory data in a data storage area corresponding to the device characteristic information.

The data storage subunit 403 - 3 is used to store different standardized somatosensory data in different data storage areas; and to keep the standardized somatosensory data stored in the data storage area corresponding to the same device feature information updated.

For the same somatosensory device, when a new somatosensory event is triggered, the somatosensory event is parsed to obtain new standardized somatosensory data, and the standardized somatosensory data stored in the data storage area is updated with the new standardized somatosensory data to achieve data update.

In this embodiment, the data storage area corresponds to the device ID of the somatosensory device, and one device ID corresponds to one data storage area. The data storage area can be a storage area in a database or other data storage unit, which is not limited here.

The data storage subunit 403-3, the somatosensory device acquisition subunit 403-2 and the somatosensory event parsing subunit 403-1 jointly maintain the somatosensory device list;

The somatosensory event parsing subunit 403-1 first parses the somatosensory event triggered in the event management unit 402 to obtain device feature information and standardized somatosensory data corresponding to the somatosensory event;

The somatosensory device acquisition subunit 403-2 searches and acquires the somatosensory device corresponding to the device characteristic information from the somatosensory device list based on the device characteristic information;

The data storage subunit 403 - 3 stores the standardized somatosensory data in a data storage area corresponding to the device characteristic information. The data storage area may be a storage area in a database; or other data storage units, which is not limited here.

The interface providing unit 404 is configured to provide an application program interface to the application program.

The interface providing unit 404 includes an interface implementing subunit 404-1 and an interface encapsulating subunit 404-2;

An interface implementation subunit 404 - 1 implements the application programming interface based on the somatosensory device management unit 403 ;

It should be noted that the application programming interface can be implemented based on different programming languages, such as Java, C, and C++, etc. The interface packaging subunit 404-2 is used to package and compile the application programming interface into a library file and output it.

The software development kit performs interface encapsulation on the standardized somatosensory data and device feature information provided by the somatosensory device management unit 403 according to different application language environments, typical of which are Java, C, and C++. Furthermore, the software development kit is ultimately provided to the outside in the form of an application library file.

For example:

The process of interface encapsulation based on Java language is as follows:

1) Define the somatosensory device class, somatosensory device management class, and somatosensory event class in Java language;

2) Define the application programming interface (API) that needs to be exposed to the public for the somatosensory device class, somatosensory device management class, and somatosensory event class, and implement the application programming interface (API) according to the somatosensory device management unit 403; In addition, the somatosensory device management unit 403 is written in C/C++, so it is encapsulated in Java and requires Java JNI calls;

3) The application programming interface (API) is packaged and compiled into a library file for external output, and a software development kit manual document is simultaneously output for use by application developers.

In addition, other programming languages may also be used to implement the function of the interface encapsulation unit 404 for providing an application program interface encapsulated in multiple languages for application software, which is not limited in this embodiment.

Under existing technologies, applications developed by a particular manufacturer or developer are only compatible with one or more specific motion sensing devices for which the application was developed, and are not compatible with other motion sensing devices. In this embodiment, a unified application development interface is provided through a software development kit for developers to use, and all applications developed by developers using the software development kit are compatible with the motion sensing devices.

Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims (17)

1. A method for data interaction between an intelligent display terminal and a motion-sensing device, characterized in that it includes: Based on reading device data uploaded by motion-sensing devices using a smart display terminal; The data format standard adopted by the motion-sensing device corresponding to the acquired device data; The device data is parsed according to the data format standard to generate motion data; The somatosensory data is converted into standardized somatosensory data based on a preset conversion algorithm; The application reads the standardized somatosensory data. 2. The method for data interaction between a smart display terminal and a motion-sensing device according to claim 1, characterized in that the step of reading device data uploaded by the motion-sensing device based on the smart display terminal is received through an interface driver module; The interface driver module includes: USB driver, WIFI driver and/or Bluetooth driver. 3. The method for data interaction between an intelligent display terminal and a motion-sensing device according to claim 1, wherein the conversion algorithm is formulated for the motion-sensing data format based on the relationship between the motion-sensing data format and the standardized motion-sensing data format. 4. The method for data interaction between a smart display terminal and a motion-sensing device according to claim 1, characterized in that, before the application reads the standardized motion-sensing data, it includes: The standardized somatosensory data is packaged according to a proprietary protocol to obtain a somatosensory data package; The motion-sensing data packet is sent to the application. 5. The method for data interaction between a smart display terminal and a motion-sensing device according to claim 4, characterized in that the application reads the standardized motion-sensing data, comprising: The application receives the somatosensory data packets; The application parses the somatosensory data packets according to the proprietary protocol to obtain standardized somatosensory data. 6. The method for data interaction between a smart display terminal and a motion-sensing device according to claim 5, wherein the application receives the motion-sensing data packet, specifically, the application receives the motion-sensing data packet through an application programming interface. 7. The method for data interaction between a smart display terminal and a motion-sensing device according to claim 6, wherein the smart display terminal sends instructions to the motion-sensing device through the application programming interface. 8. A device for data interaction between an intelligent display terminal and a motion-sensing device, characterized in that it comprises: The device data reading unit is used to read device data uploaded by the motion sensing device based on the smart display terminal. A data conversion unit is used to convert the device data into standardized somatosensory data; the data conversion unit includes: a data standard acquisition subunit, a device data parsing subunit, and a standardized somatosensory data generation subunit; The data standard acquisition subunit is used to acquire the data format standard adopted by the somatosensory device corresponding to the device data; The device data parsing subunit is used to parse the device data according to the data format standard and generate motion data. The standardized somatosensory data generation subunit is used to convert the somatosensory data into standardized somatosensory data based on a preset conversion algorithm; The data retrieval unit is used to enable the application to read the standardized somatosensory data. 9. The apparatus for data interaction between an intelligent display terminal and a motion-sensing device according to claim 8, characterized in that it comprises: A data packaging unit is used to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data package; A data sending unit is used to send the motion-sensing data packet to the application. 10. The apparatus for data interaction between an intelligent display terminal and a motion-sensing device according to claim 9, characterized in that the data retrieval unit comprises: The data receiving subunit allows the application to receive the motion-sensing data packets; The data parsing subunit is used by the application to parse the motion data packets according to the private protocol to obtain standardized device data. 11. A motion-sensing device middleware for data interaction between a smart display terminal and a motion-sensing device, characterized in that it comprises: a drive docking unit, a data standardization unit, a motion-sensing device management unit, and a data communication unit; The drive docking unit is used to read device data uploaded by the motion sensing device based on the smart display terminal, and to send commands from the application. The data standardization unit is used to convert the device data into standardized haptic data; The data standardization unit includes: a data format standard acquisition subunit, a data parsing subunit, and a data conversion subunit; The data format standard acquisition subunit is used to acquire the data format standard adopted by the somatosensory device corresponding to the device data; The data parsing subunit is used to parse the device data according to the data format standard and generate motion data. The data conversion subunit is used to convert the somatosensory data into standardized somatosensory data based on a preset conversion algorithm; The motion sensing device management unit is used to manage the motion sensing devices connected to the smart display terminal and the device data corresponding to the motion sensing devices. The data communication unit communicates with the application based on inter-process communication. 12. The motion-sensing device middleware for data interaction between a smart display terminal and a motion-sensing device according to claim 11, characterized in that the motion-sensing device management unit comprises: The parsing subunit is used to parse the standardized somatosensory data to obtain device feature information; The storage subunit is used to store the standardized haptic data into the data storage area corresponding to the device feature information. 13. The motion-sensing device middleware for data interaction between a smart display terminal and a motion-sensing device according to claim 11, characterized in that the data communication unit comprises: The somatosensory data packet generation subunit is used to package the standardized somatosensory data according to a private protocol to obtain a somatosensory data packet. A motion-sensing data packet sending subunit is configured to send the motion-sensing data packet to the application; and The instruction data packet receiving subunit is used to receive instruction data packets sent by the application. The instruction data packet parsing subunit is used to parse the instruction data packet into standardized instruction data according to a private protocol. 14. The motion-sensing device middleware for data interaction between a smart display terminal and a motion-sensing device according to claim 11, wherein the inter-process communication includes: message queues, shared memory, or sockets. 15. A software development kit for data interaction between a smart display terminal and a motion-sensing device, characterized in that it comprises: a data communication unit, an event management unit, a motion-sensing device management unit, and an interface encapsulation unit; The data communication unit performs data communication with the motion sensing device middleware based on inter-process communication. The data communication unit includes: a somatosensory data packet receiving subunit, a somatosensory data packet parsing subunit, an instruction data packet generating subunit, and an instruction data packet sending subunit; The somatosensory data packet receiving subunit is used to receive somatosensory data packets sent by the somatosensory device middleware; The somatosensory data packet parsing subunit is used to parse the somatosensory data packet according to a private protocol to obtain standardized somatosensory data; The instruction data packet generation subunit is used to package the instruction data according to a private protocol to obtain an instruction data packet. The instruction data packet sending subunit is used to send the instruction data packet to the motion sensing device middleware; The event management unit controls the operation of the application based on various preset haptic events; A motion sensing device management unit is used to manage motion sensing devices connected to the smart display terminal and the device data corresponding to the motion sensing devices. The interface provider unit is used to provide application interfaces to applications. 16. The software development kit for data interaction between a smart display terminal and a motion-sensing device according to claim 15, wherein the motion-sensing device management unit comprises: The somatosensory event parsing subunit is used to parse the somatosensory events to obtain device feature information and standardized somatosensory data; A motion sensing device acquisition subunit is used to acquire the motion sensing device corresponding to the device feature information from the motion sensing device list based on the device feature information. The data storage subunit is used to store the standardized haptic data into the data storage area corresponding to the device feature information. 17. The software development kit for data interaction between a smart display terminal and a motion-sensing device according to claim 15, wherein the interface providing unit comprises: The interface implementation subunit implements the application programming interface based on the motion sensing device management unit; the interface encapsulation subunit is used to package and compile the application programming interface into a library file and output it.