TWI762321B - Federated data exploration systems and interoperable data exploration apparatuses - Google Patents

Abstract

A federated data exploration system includes a first and a second interoperable data exploration apparatuses. The first interoperable data exploration apparatus executes a data process engine and has a first data object. The data process engine enables a display screen to display a visualization user interface, which includes a first and second display areas respectively for the first and second data object. The second data object belongs to the second interoperable data exploration apparatus. The data process engine explores the second data object by issuing a sequence of queries in response to a sequence of operations on the second display area to narrow down a target of interest. The data process engine receives a first set of identifiers corresponding to the target of interest from the second data object and produces a target dataset by querying the first data object according to the first set of identifiers.

Description

Federal Data Exploration System and Interoperable Data Exploration Device

The present invention relates to a federated data exploration system and an interoperable data exploration device. In particular, the present invention relates to a federated data discovery system with high precision access control and an interoperable data discovery device, and data sharing between the interoperable data discovery devices is a relationship between data ownership and The right to use the data is differentiated.

With the advent of the era of big data, more and more users (such as: individuals, enterprises, organizations) will collect various data for analysis, and then make decisions based on the results of the analysis. However, high-quality decision-making often requires reference to external data (eg, open data) and third-party market data. Since the breadth of user-owned data is quite limited, it is an inevitable trend to share data among users to make more accurate decisions.

In order to achieve data sharing, the conventional technology mainly provides files for users to download, or provides an Application Programming Interface (API) for users to access data, but these methods have many disadvantages. First, since the user who downloads the data set will own the data set, and even share the downloaded data set with others, the data owner may lose the unique data ownership. Therefore, the vast majority of data owners are hesitant to share data. Second, because data owners may lose unique data ownership after sharing data, they tend to provide sharable data from their own perspective rather than the perspective of others. As a result, users expecting to find useful information may end up with data that is not ready or unusable, and it takes a lot of time and data processing skills to convert the raw data into usable data. Third, it is very difficult for users to perform cross-dataset analysis without integrating datasets of different data owners into the same database and then aligning attributes for cross-reference. Extracting and processing multiple data sets from different data sources is also an extremely time-consuming and labor-intensive task. That is, cross-analysis across multiple data sets belonging to different data owners is difficult to achieve with today's means of sharing data (ie, downloading files, providing APIs). Fourth, in the file-based data sharing mechanism, the user's access restriction is set at the file level, so the user has no choice but to download the entire file. Loading a data file with very many dimensions (ie, fields) is time-consuming and requires sufficient local storage to retain the downloaded file, even if the user's application uses only a tiny fraction of the downloaded data It is still needed.

Furthermore, for today's data applications, high-precision multi-dimensional data access (eg: any combination of dimensions and values can be arbitrary conditions, such as "state is New York") is indispensable, especially For data stream analysis. It is necessary for users to be able to request the granularity of data access, and for the data owner to determine the granularity of data access that can be granted. When data exploration and high-precision data access control are available, data owners can charge users for data usage.

To sum up, in order to promote the use of data, there is an urgent need in the technical field of the present invention to have a high-precision access control, distinguish data ownership and data use rights, and have a user-friendly interface to quickly and easily find at any level Interoperable data exploration device for data of interest.

An object of the present invention is to provide an interactive data exploration device. The interoperable data exploration device includes a storage device, a transceiver interface and a processor, wherein the processor is electrically connected to the storage device and the transceiver interface. The processor installs a data object package software, wherein the data object package software includes a data processing engine, a database and a visual user interface. The processor executes the data processing engine, the data processing engine continuously inputs a data stream into the database to obtain a first data object, and the storage device retains the first data object.

The data processing engine causes a display screen to display the visual user interface, wherein the visual user interface includes a first display area and a second display area. The first display area is used for the first data object. The data processing engine establishes a connection with another data exploration device through the transceiving interface, so that the second display area is used for a second data object belonging to the other data exploration device. In response to a first series of operations performed on the second display area, the other interoperable data exploration device issues a first series of queries to perform a first data exploration on the second data object tasks, thereby gradually narrowing down to a data set of interest. Each of the queries in the first series of queries is executed against a result data set of a corresponding previous query in the first series of queries, and a first query in the first series of queries is Executed for the second data object. The data processing engine receives a first plurality of identification codes corresponding to the data set of interest from the second data object through the connection. In response to a second series of operations performed on the first display area according to the first set of identifiers, the data processing engine issues a second series of queries to execute on the first data object A second data exploration task, thereby generating a target data set.

Another object of the present invention is to provide an interactive data exploration device. The interoperable data exploration device includes a storage device, a transceiver interface and a processor, wherein the processor is electrically connected to the storage device and the transceiver interface. The processor installs a data object package software, wherein the data object package software includes a data processing engine, a database and a visual user interface. The processor executes the data processing engine, the data processing engine continuously inputs a data stream into the database to obtain a first data object, and the storage device retains the first data object.

The data processing engine causes a display screen to display the visual user interface, wherein the visual user interface includes a first display area and a second display area. The first display area is used for the first data object. The data processing engine establishes a connection with another data exploration device through the transceiving interface, so that the second display area is used for a second data object belonging to the other data exploration device. In response to a first series of operations performed on the first display area, the data processing engine issues a first series of queries to perform a first data exploration task on the first data object, thereby scaling down to a data set of interest. Each of the queries in the first series of queries is executed against a result data set of a corresponding previous query in the first series of queries, and a first query in the first series of queries is Executed for the first data object. The data processing engine receives a first plurality of identifiers corresponding to the data set of interest from the first data object, and the data processing engine transmits the first plurality of identifiers to the other through the connection Interoperable data exploration device. In response to a second series of operations performed on the second display area according to the first set of identifiers, the other interoperable data exploration device issues a second series of queries to A second data exploration task is executed on the second data object, thereby generating a target data set.

Another object of the present invention is to provide a federated data exploration system. The federated data discovery system includes a central management device, a first interoperable data discovery device and a second interoperable data discovery device. The first interoperable data exploration device and the second interoperable data exploration device are individually installed with a data object package including a data processing engine, a database and a visual user interface. The first interoperable data exploration device continuously inputs a first data stream into the corresponding database to obtain a first data object, retains the first data object, and transmits a first meta information ) to the central management device to register the first data object, wherein the first metadata describes a plurality of first identifiers and a plurality of first attributes. The second interoperable data exploration device continuously inputs a second data stream into the corresponding database to obtain a second data object, retains the second data object, and transmits a second metadata to the central The management device registers the second data object, wherein the second metadata describes a plurality of second identification codes and a plurality of second attributes. The first interoperable data discovery device sends a message to the central management device to subscribe to the second data object.

The first interactive data exploration device causes a display screen to display the corresponding visual user interface, wherein the corresponding visual user interface includes a first display area and a second display area. The first display area is used for the first data object. The first interoperable data exploration device makes the second display area available for the second data object by establishing a connection with the second data exploration device. In response to a first series of operations performed on the second display area, the second interoperable data exploration device issues a first series of queries to perform a first data exploration on the second data object tasks, thereby gradually narrowing down to a data set of interest. Each of the queries in the first series of queries is executed against a result data set of a corresponding previous query in the first series of queries, and a first query in the first series of queries is Executed for the second data object. The second interoperable data discovery device transmits a first plurality of identifiers corresponding to the data set of interest to the central management device, and the central management device transmits a first plurality of identifiers corresponding to the first set of identifiers A second set of identification codes to the first interoperable data discovery device. In response to a second series of operations performed on the first display area according to the second set of identifiers, the first interoperable data exploration device issues a second series of queries to A second data exploration task is executed on the first data object, thereby generating a target data set.

Another object of the present invention is to provide a federated data exploration system. The federated data discovery system includes a central management device, a first interoperable data discovery device and a second interoperable data discovery device. The first interoperable data exploration device and the second interoperable data exploration device are individually installed with a data object package including a data processing engine, a database and a visual user interface. The first interoperable data exploration device continuously inputs a first data stream into the corresponding database to obtain a first data object, retains the first data object, and transmits a first metadata to the central The management device registers the first data object, wherein the first metadata describes a plurality of first identification codes and a plurality of first attributes. The second interoperable data exploration device continuously inputs a second data stream into the corresponding database to obtain a second data object, retains the second data object, and transmits a second metadata to the central The management device registers the second data object, wherein the second metadata describes a plurality of second identification codes and a plurality of second attributes. The first interoperable data discovery device sends a message to the central management device to subscribe to the second data object.

The first interactive data exploration device causes a display screen to display the corresponding visual user interface, wherein the corresponding visual user interface includes a first display area and a second display area. The first display area is used for the first data object. The first interoperable data exploration device makes the second display area available for the second data object by establishing a connection with the second data exploration device. In response to a first series of operations performed on the first display area, the first interoperable data exploration device issues a first series of queries to perform a first data discovery on the first data object tasks, thereby gradually narrowing down to a data set of interest. Each of the queries in the first series of queries is executed against a result data set of a corresponding previous query in the first series of queries, and a first query in the first series of queries is Executed for the first data object. The first interoperable data exploration device transmits a first plurality of identifiers corresponding to the data set of interest to the central management device, and the central management device transmits a first set of identifiers corresponding to the first plurality of identifiers A second set of identification codes to the second interoperable data discovery device. In response to a second series of operations performed on the second display area according to the second set of identifiers, the second interoperable data exploration device issues a second series of queries to A second data exploration task is executed on the second data object, thereby generating a target data set.

As can be seen from the above description, the data exploration technology provided by the present invention distinguishes data ownership and data use rights, and has a user-friendly visual user interface that allows users to quickly and easily find the data of interest (that is, the sense of interest data set). From the perspective of the data owner (e.g. the second interoperable data exploration device), it still retains data ownership because the data user (e.g. the first interoperable data exploration device) can only acquire data from the data The user obtains a plurality of identification codes or uses a plurality of identification codes to explore the data object of the data owner. Since the data user does not own the data owner's data objects, the data owner feels protected. From a data user's point of view, since a plurality of identifiers obtained from one interoperable data discovery device can be used to discover data objects of another interoperable data discovery device, many Cross-analysis of data sets (that is, multiple data objects). Exchanging a set of identifiers rather than exchanging or sharing entire data sets can be viewed as efficient and efficient interaction between multiple data sets (ie, multiple data objects).

The interoperable data exploration device provided by the present invention can adopt the object-oriented technology, thereby encapsulating the data and the code together to achieve the purpose of protecting the ownership of the data when sharing the data. Another indispensable condition for data sharing is the interoperability between interoperable data exploration devices, which allow commercial users or data analysts to visualize between data sets provided by different data planets. way of exploration and analysis. As data owners feel protected and benefit from sharing data, data ecosystems such as data marketplaces, supply chain collaboration platforms, industrial data platforms, etc. can be positively developed in the market. Therefore, the data discovery technology provided by the present invention can facilitate data sharing among users.

The detailed technology and embodiments of the present invention are described below with reference to the drawings, so that those with ordinary knowledge in the technical field to which the present invention pertains can understand the technical features of the claimed invention.

The following will explain the federated data discovery system and the interoperable data discovery device provided by the present invention through embodiments. However, these embodiments are not intended to limit the implementation of the present invention in any environment, application or manner as described in these embodiments. Therefore, the description about the following embodiments is only for the purpose of explaining the present invention, and not for limiting the scope of the present invention. It should be understood that, in the following embodiments and drawings, elements not directly related to the present invention have been omitted and not shown. In addition, the size of each element in the drawings and the size ratio among the elements are only for convenience of illustration and description, and are not intended to limit the scope of the present invention.

The key concept of the present invention is to provide some operational mechanism, allowing data owners to package and provide structured data in an interoperable data exploration device, and allowing data users (IT personnel or non-IT personnel) to download the data Interactively explore data objects in an unplanned, unstructured, but controlled manner to discover primitive patterns, features, and matters of interest, prior to in-depth analysis. The federated data exploration system provided by the present invention allows users to visually explore and analyze multiple data sets of different interoperable data exploration devices, so as to track and search for interesting data sets. For the data results obtained from the interoperable data exploration devices, the data owners only share the identification codes associated with them, thereby protecting the data they own from being completely owned by others. A Non-Fungible Token (NFT) or other suitable mechanism may be employed to secure ownership of an interoperable data exploration device. Therefore, the present invention can achieve the technical effect of providing interactivity, data exploration or/and interactive analysis for data sets of different data owners.

The first embodiment of the present invention is a federated data exploration system 1 , the schematic diagram of which is depicted in FIG. 1A . The federated data exploration system 1 includes interoperable data exploration devices 11 and 12 owned by data owners O1 and O2, respectively. The interoperable data exploration devices 11 and 12 can share data in response to a series of operations performed by the user. It should be noted that the interoperable data exploration devices 11 and 12 share data in a manner of distinguishing data ownership and data usage rights. That is, the user U2 of the interoperable data exploration device 12 can be allowed to explore the data objects retained in the interoperable data exploration device 11 without sacrificing the data ownership of the data owner O1 (ie, data owned by the data owner O1). Similarly, the user U1 of the interoperable data exploration device 11 can be allowed to explore the data objects retained in the interoperable data exploration device 12 without sacrificing the data ownership of the data owner O2 (ie, data owned by data owner O2). The technical details of how to distinguish between data ownership and data use rights are described later.

The interoperable data exploration devices 11 and 12 are each an electronic computing device, such as a desktop computer, a notebook computer, and a tablet computer. FIG. 1B depicts a schematic structural diagram of the data exploration devices 11 and 12 that can operate interoperably. The interoperable data exploration device 11 includes a processor 111 , a transceiver interface 113 and a storage device 115 , wherein the processor 111 is electrically connected to the transceiver interface 113 and the storage device 115 . Similarly, the interoperable data exploration device 12 includes a processor 121 , a transceiver interface 123 and a storage device 125 , wherein the processor 121 is electrically connected to the transceiver interface 123 and the storage device 125 . The processors 111 and 121 can each be various processors, a central processing unit (CPU), a microprocessor (Microprocessor Unit, MPU), a digital signal processor (DSP), or the technical field to which the present invention pertains. have other computing devices known to those of ordinary skill. The transceiver interfaces 113, 123 can each be a wired interface or a wireless interface, which are used to connect to a network (eg, the Internet, a local area network) to send signals and receive data through the network. The storage devices 115 and 125 can each be a hard disk (eg, a solid-state hard disk) or other non-transitory computer-readable storage media or devices known to those of ordinary skill in the art to store various digital data. .

In this embodiment, the interoperable data exploration devices 11 and 12 can perform similar operations, have similar functions, and achieve similar technical effects. Therefore, the following description will focus on an interoperable data exploration device, For example: an interactive data exploration device 11 .

The interoperable data exploration device 11 receives a data object from the other device through the transceiving interface 113 before sharing data with the other interoperable data exploration device or/and before exploring the data in the other interoperable data exploration device Package software PK (eg: download data object package software PK from a server). The data object package software PK includes a data processing engine EN, a database DB and a visual user interface VI. The database DB provided by the data object package PK can be any database that allows the data owner to manage the data in a structured manner, such as relational database, graph database, and NoSQL database. In some embodiments, the storage device 115 of the interoperable data exploration device 11 may already store the data object package PK, so that there is no need to receive the data object package PK from another device.

The processor 111 installs the data object package software PK and executes the data processing engine EN. The data processing engine EN continuously inputs a data stream 110 into the database DB to obtain a data object 112 . It should be noted that, in some embodiments, the data owner O1 can operate on the visual user interface VI1 displayed on the display screen 117 to determine how to input the data stream 110 to the database DB (eg, the data stream Which attribute in 10 corresponds to which dimension in the database DB) for the data processing engine EN to operate accordingly. The storage device 115 retains the data object 112 . The interoperable data exploration device 12 performs the same operation to obtain the data object 122 from the data stream 120 . Similarly, the data owner O2 can operate on the visual user interface VI2 displayed on the display screen 127 to determine how to input the data stream 120 to the database DB. Due to the data processing engine, visual user interface and data objects, the interoperable data exploration devices 11, 12 are each independently a self-contained data platform that can provide IT personnel and/or non-IT personnel in multiple Visual exploration, analysis and interactive analysis are performed between interactive data exploration devices, and information technology personnel or programmers are provided to access data through application programming interfaces. The details of its operation are detailed below.

The data processing engine EN executed by the interoperable data exploration device 11 causes a display screen 117 to display the visual user interface VI1. The visual user interface VI1 includes a display area D1 and a display area D2, wherein the display area D1 is used for the data objects owned by the interactive data exploration device 11, and the display area D2 is used for other interactive data exploration The data objects owned by the device (ie, the data objects that the interoperable data exploration device 11 is about to explore). In order for the display area D2 to display the data objects owned by another data exploration device that can operate interactively, the data processing engine EN executed by the data exploration device 11 that can interact with each other establishes a connection with the other data exploration device through the transceiver interface 113 . A connection, where the connection is a connection on a network. In this embodiment, the display area D1 is used for the data object 112 , and the display area D2 is used for the data object 122 possessed by the interactive data exploration device 12 . In order to display the data object 122 in the display area D2, the data processing engine EN executed by the interoperable data exploration device 11 establishes a connection between the interoperable data exploration device 11 and the interoperable data exploration device 12.

The user U1 may perform a first series of operations (not shown) on the display area D2 to mine useful information from the data object 122 . In some embodiments, if the data object 122 is a data set (not shown) having a plurality of dimensions (not shown), the first series of operations and the plurality of operations performed by the user U1 on the display area D2 Related to a subset of key dimensions, where the key dimensions are contained within the dimensions. The first series of operations performed on the display area D2 are related to selecting an ordered subset of the dimensions according to the plurality of values, wherein each operation of the first series of operations is based on the corresponding operation of the previous operation The ordered subset of is followed by any of the dimensions, and each of the first series of queries generates a temporary set of identifiers for each of the key dimensions. In response to the first series of operations performed by the user U1 on the display area D2, the data processing engine EN executed by the interoperable data exploration device 12 issues a first series of queries to execute on the data object 122 A first data exploration task, thereby gradually narrowing down to a data set of interest, such as: users who have purchased a certain product in a certain city, vehicles with a speed of 60-80 km/h in Taipei City last month . It should be noted that each of the aforementioned queries is executed against a result data set (not shown) of a corresponding previous query in the first series of queries, and a first query in the first series of queries Executed for data object 122 . Afterwards, the data processing engine EN executed by the interoperable data exploration device 11 receives a first corresponding data set of interest from the data object 122 through the connection between the interoperable data exploration devices 11 and 12 The plurality of identification codes S1 are set so that the data processing engine EN executed by the interoperable data exploration device 11 can use the first plurality of identification codes S1 for further analysis.

In some embodiments, in the process of gradually narrowing down to the data set of interest, the user U1 can determine whether the current result data set displayed in the display area D2 is the data set of interest. If the result of the judgment is positive, the user U1 can perform a confirmation operation in the display area D2 (for example, click a confirmation button in the display area D2), so that the interactive data exploration device 12 can assign the interest accordingly. An identification code (not shown) of the data set, wherein the identification code corresponds to the first group of plural identification codes S1. Then, the data processing engine EN executed by the interoperable data discovery device 11 receives a notification message M2 through the connection between the interoperable data discovery devices 11 and 12, wherein the notification message M2 is about the identification code. produce. The data processing engine EN executed by the interoperable data exploration device 11 transmits a request signal M3 to the interoperable data exploration device 12 through the connection between the interoperable data exploration devices 11 and 12, wherein the request signal M3 is used to request the first group of plural identification codes S1. Afterwards, the data processing engine EN executed by the interoperable data discovery device 11 receives the first plurality of identification codes S1 through the connection between the interoperable data discovery devices 11 and 12 .

In some embodiments, the data processing engine EN executed by the interoperable data exploration device 11 also receives the first set of plural numbers from the data object 122 through the connection between the interoperable data exploration devices 11 and 12 At least one attribute of each of the identification codes S1.

Then, the user U1 can mine useful information from the data object 112 based on the first plurality of identification codes S1. Specifically, the user U1 can perform a second series of operations (not shown) on the display area D1 based on the first plurality of identification codes S1 to mine useful information from the data object 112 . In some embodiments, if the data object 112 is a data set (not shown) having a plurality of dimensions (not shown), the second series of operations performed by the user U1 on the display area D1 are related to the A subset of equal dimensions are correlated, wherein each of the operations of the second series of operations is followed by any one of the dimensions based on a selected dimension combination corresponding to the previous operation. In response to the second series of operations performed by the user U1 on the display area D1, the data processing engine EN executed by the interoperable data exploration device 11 sends out a second series of complex numbers according to the first plurality of identification codes S1 A query to perform a second data exploration task on the data object 112, thereby generating a target data set (not shown).

In some embodiments, if the data processing engine EN executed by the interoperable data exploration device 11 also receives from the data object 122 at least one attribute of each of the first plurality of identifiers S1, the interactive The data processing engine EN executed by the operating data exploration device 11 issues the second series of queries to execute the first query on the data object 112 according to the first set of the plurality of identifiers S1 and the received attributes. Two data exploration tasks, thereby generating the target data set (not shown).

In some implementations, the data object 112 and the data object 122 may use different encoding mechanisms for the same item (eg, attribute, dimension). In these embodiments, after receiving the first plurality of identification codes S1, the data processing engine EN executed by the interoperable data exploration device 11 converts the first plurality of identification codes S1 to one-to-one A second group of plural identification codes. Next, the data processing engine EN executed by the interoperable data exploration device 11 issues the second plurality of operations in response to the second series of operations performed on the display area D1 according to the second plurality of identification codes. A series of queries are performed to perform the second data exploration task on the data object 112 .

In certain embodiments, the aforementioned data exploration mechanisms may be reversed. The user U1 can perform a first series of operations (not shown) on the display area D1 to mine useful information from the data object 112 . In response to the first series of operations performed by the user U1 on the display area D1, the data processing engine EN executed by the interoperable data exploration device 11 issues a first series of queries to execute a query on the data object 112. The first data exploration task, whereby gradually narrowing down to a data set of interest. Next, the data processing engine EN executed by the interoperable data exploration device 11 receives from the data object 112 a first plurality of identification codes S1 corresponding to the interested data set. Afterwards, the data processing engine EN executed by the interoperable data discovery device 11 transmits the first set of plural identification codes S1 to the interoperable data discovery through the connection between the interoperable data discovery devices 11 and 12 device 12 . The data processing engine EN executed by the interoperable data exploration device 12 issues a second series of queries in response to a second series of operations performed on the display area D2 according to the first plurality of identification codes S1 A second data exploration task is performed on the data object 122, thereby generating a target data set.

In some embodiments, in the process of gradually narrowing down to the data set of interest, the user U1 can determine whether the current result data set displayed in the display area D1 is the data set of interest. If the result of the judgment is affirmative, the user U1 can perform a confirmation operation in the display area D1 (for example, click a confirmation button in the display area D1), so that the data processing engine executed by the interactive data exploration device 11 The EN assigns an identification code (not shown) to the data set of interest in response to the confirmation operation on the display area D1 , wherein the identification code corresponds to the first plurality of identification codes S1 . Then, the data processing engine EN executed by the interoperable data exploration device 11 transmits a notification message to the interoperable data exploration device 12 through the connection between the interoperable data exploration devices 11 and 12, wherein the The notification message is about the generation of the identifiable code. Afterwards, the data processing engine EN executed by the interoperable data exploration device 11 receives a request signal from the interoperable data exploration device 12 through the connection between the interoperable data exploration devices 11 and 12, wherein the The request signal is used to request the first group of the plurality of identification codes S1. Next, the data processing engine EN executed by the interoperable data discovery device 11 transmits the first set of plural identification codes S1 to the interoperable data discovery through the connection between the interoperable data discovery devices 11 and 12 device 12 . Based on the foregoing description, those skilled in the art to which the present invention pertains can understand all the operation details in this application situation, so no further description will be given.

As can be seen from the above description, the user of the interactive data exploration device 11 can explore the data objects 122 stored in the interactive data exploration device 12 through the visual user interface VI. This mechanism is very fast and easy. After finding the data set of interest from the data object 122 , the interoperable data exploration device 11 only obtains the first plurality of identification codes S1 corresponding to the data set of interest, but not the data corresponding to the data set of interest. From the perspective of the data owner (ie, the interoperable data exploration device 12 ), since the data requester (ie, the interoperable data exploration device 11 ) only obtains the first plurality of identification codes S1 , Therefore, the data owner still has unique data ownership. From the point of view of the data requester, since the first group of plural identifiers S1 are related to the data objects 122 of other interoperable data exploration devices, and carry information that the data requester does not have, the first group of plural identification codes S1 are related to the data objects 122 of other interoperable data exploration devices The identifier S1 is of great value to the data requester. Therefore, the data requester can query its own data objects according to the first plurality of identification codes S1 to generate a target data set.

It can be seen from the above description that the federated data exploration system 1 distinguishes data ownership and data use rights, and has a user-friendly visual user interface that allows users to quickly and easily find interesting data (that is, the data of interest). set). By providing a visual user interface, IT personnel or non-IT personnel can perform a series of multiple operations in a display area to realize interactive data exploration and thus find data sets of interest. From the perspective of the data owner (eg, the interoperable data discovery device 12 ), it still retains data ownership because the data user (eg, the interoperable data discovery device 11 ) can only get the data from the data owner A set of identifiers or a data object that uses a set of identifiers to discover the owner of the data. Since the data user does not own the data owner's data objects, the data owner feels protected. From a data user's point of view, since a set of identifiers obtained from a data object of an interoperable data discovery device can be used to discover a data object of another interoperable data discovery device, Cross-analysis of multiple data sets (ie, multiple data objects) can be achieved. Exchanging a set of identifiers rather than exchanging or sharing entire data sets can be viewed as efficient and efficient interaction between multiple data sets (ie, multiple data objects). The federated data exploration system 1 achieves cross-analysis among a plurality of interoperable data exploration devices by exchanging a plurality of identifiers, thereby producing more synergistic results and revealing unpredictable insights and/or facts. Therefore, the federated data discovery system 1 can facilitate data sharing among multiple users.

The second embodiment of the present invention is a federated data exploration system 2 , the schematic diagram of which is depicted in FIG. 2 . The federated data discovery system 2 includes interoperable data discovery devices 11 , 12 and a central management device 15 . The central management device 15 can be any electronic computing device capable of exchanging messages with other electronic computing devices, such as a cloud server. It should be noted that, in other embodiments, the number of interoperable data exploration devices included in the federated data exploration system 2 may be more than two.

Compared with the federated data exploration system 1, the federated data exploration system 2 can also perform the same operations, have the same functions, and achieve the same technical effects. Therefore, the following description will focus on the differences between Federated Data Exploration System 2 and Federated Data Exploration System 1.

In this embodiment, the central management device 15 is a hub, which can also be understood as an exchange. The central management device 15 allows the interoperable data exploration devices 11, 12 to register and subscribe, maintains the meta information of the interoperable data exploration devices 11, 12 for retrieval, and targets different data sets of interest. Different sets of multiple identification codes provide a switching mechanism. For example, metadata for an interoperable data discovery device may include name, URL, sharing policy, and access control policy.

If an interoperable data discovery device wants to join the federated data discovery system 2 to share data with other interoperable data discovery devices or/and explore the data of other interoperable data discovery devices, it must first report to the central management device 15 Register. Specifically, the interoperable data exploration device 11 transmits the metadata 114 of the data object 112 to the central management device 15 through the transceiver interface 113 to register the data object 112 . The metadata 114 describes a plurality of first identifiers and a plurality of first attributes. Similarly, the interoperable data exploration device 12 transmits the metadata 124 of the data object 122 to the central management device 15 through the transceiver interface 123 to register the data object 122 . The metadata 124 describes a plurality of second identifiers and a plurality of second attributes.

In this embodiment, if a user of an interoperable data discovery device is interested in data objects of other interoperable data discovery devices, the interoperable data discovery device must send a message to the central management device 15 To subscribe to that interoperable data discovery device's data objects. In some implementations, the message for subscribing to a data object may include an access control permission for controlling the accessibility of a subset of the data object. Here is a concrete example. The user of the interoperable data exploration device 11 is interested in the data objects 122 of the interoperable data exploration device 12 . The interoperable data exploration device 11 has to send a message M1 to the central management device 15 through the transceiver interface 113 to subscribe to the data object 122 . After subscribing, the interoperable data exploration device 11 has the right to explore the data object 122 . In some implementations, the message M1 used to subscribe to the data object 122 includes an access control authority, and the access control authority is used to control the accessibility of a subset of the data object 122 . If the data object 122 is a data set with a plurality of dimensions, the access control authority controls which dimensions the interoperable data exploration device 11 can access.

In some embodiments, the visual user interface VI1 displayed on the display screen 117 includes not only the display areas D1 and D2 but also a display area D3, as shown in FIG. 2 . Specifically, the display area D3 is used to display a plurality of registered data objects 116 and 126 that have been registered in the central management device 15 . The registration data objects 116 and 126 each have a registration metadata in the central management device 15 , and each of the registration metadata and the metadata 114 has at least one common identifier description. In this way, a user of the interoperable data exploration device 11 can browse the registered data objects 116, 126 and find data objects of interest.

In this embodiment, after the data processing engine EN executed by the interoperable data exploration device 11 is gradually reduced to a data set of interest, the central management device 15 processes the plurality of identification codes corresponding to the data set of interest exchange operation. Specifically, the interoperable data exploration device 12 transmits the first group of plural identification codes S1 corresponding to the interested data set to the central management device 15, and the central management device 15 then sends the first group of plural identification codes S1 to a One-to-one conversion to the second plurality of identification codes S2 , and then the central management device 15 transmits the second plurality of identification codes S2 to the interoperable data exploration device 11 .

In some embodiments, in the process of gradually narrowing down to the data set of interest, the user U1 can determine whether the current result data set displayed in the display area D2 is the data set of interest. If the result of the judgment is positive, the user U1 can perform a confirmation operation in the display area D2, so that the interoperable data exploration device 12 assigns an identification code (not shown) to the interested data set, wherein the identification The code corresponds to the first group of plural identification codes S1. Then, the data processing engine EN executed by the interoperable data discovery device 11 receives a notification message M2 through the connection between the interoperable data discovery devices 11 and 12, wherein the notification message M2 is about the identification code. produce.

Afterwards, the data processing engine EN executed by the interoperable data exploration device 11 transmits a request signal M3 to the interoperable data exploration device 12 through the connection between the interoperable data exploration devices 11 and 12, wherein the request The signal M3 is used to request the first group of identification codes S1. In this embodiment, the central management device 15 handles the exchange operation of the plurality of identification codes corresponding to the data set of interest. Therefore, the interoperable data exploration device 12 transmits the first plurality of identification codes S1 corresponding to the identifiable codes to the central management device 15 , and the central management device 15 then assigns the first plurality of identification codes S1 one-to-one After switching to the second plurality of identification codes S2 , the central management device 15 transmits the second plurality of identification codes S2 to the interoperable data exploration device 11 . Thereafter, the data processing engine EN executed by the interoperable data exploration device 11 can issue a first plurality of operations in response to a second series of operations performed on the display area D1 according to the second plurality of identification codes S2. A second series of queries is performed to perform a second data exploration task on the data object 112, thereby generating a target data set.

In some embodiments, the federated data discovery system 2 may perform the aforementioned data discovery mechanism in the opposite direction. Based on the foregoing description, those skilled in the art to which the present invention pertains can understand all the operation details in this application situation, so no further description will be given.

It should be noted that in the patent specification and the scope of the patent application of the present invention, certain terms (including: interoperable data exploration device, data object and display area) are prefixed with "first", "second" or "" "Third", such "first", "second" and "third" are used to distinguish these terms from each other. In addition, some terms (including: plural queries, plural operations) are prefixed with "first series" or "second series", and these "first series" and "second series" are used to distinguish The terms that follow belong to a different series. Furthermore, some terms (including: multiple identification codes) are prefixed with "first group" or "second group", and these "first group" and "second group" are used to distinguish the following of such terms belong to different groups.

As can be seen from the above description, since the federated data exploration system 2 deploys the central management device 15, it has more advantages. If an interoperable data discovery device wants to join the federated data discovery system 2 to share data with other interoperable data discovery devices or/and explore the data of other interoperable data discovery devices, it must first report to the central management device 15 Register. Since the central management device 15 maintains the interpretation information of all the interoperable data exploration devices, the users of the registered interoperable data exploration devices can browse the interpretation information through the visual user interface VI to find out the interesting information. the interoperable data exploration device, and then subscribe to the data objects of the interoperable data exploration device. Furthermore, since an interoperable data exploration device must first subscribe to data objects reserved by other interoperable data exploration devices before it can be explored, different levels of access control rights can be set. Therefore, the federated data exploration system 2 can achieve cross-analysis among multiple interoperable data exploration devices in a more user-friendly and safer manner, thereby promoting data sharing among multiple users.

The above-mentioned embodiments are used to illustrate some embodiments of the present invention and illustrate the technical features of the present invention, but are not used to limit the protection scope and scope of the present invention. Any changes or equivalent arrangements that can be easily accomplished by those with ordinary knowledge in the technical field to which the present invention pertains belong to the claimed scope of the present invention.

1, 2: Federal data exploration system 11, 12: Interoperable data exploration device 110, 120: Data streaming 111, 121: Processor 112, 122: Data Objects 113, 123: Transceiver interface 114, 124: Interpretation of information 115, 125: Storage equipment 116, 126: Registration data objects 117, 127: Display screen 15: Central management device D1, D2, D3: Display area DB:Database en:DATA PROCESSING ENGINE M1: message M2: Notification message M3: request signal O1, O2: data owner PK: Data Object Package Software S1: The first group of plural identification codes S2: The second group of plural identification codes U1, U2: User VI, VI1, VI2: Visual User Interface

FIG. 1A depicts a schematic structural diagram of the federated data exploration system 1 in the first embodiment of the present invention.

FIG. 1B depicts a schematic structural diagram of an interoperable data exploration device in the federated data exploration system.

FIG. 2 depicts a schematic structural diagram of the federated data exploration system 2 in the second embodiment of the present invention.

2: Federal data exploration system

11, 12: Interoperable data exploration device

112, 122: Data Objects

114, 124: Interpretation of information

116, 126: Registration data objects

117, 127: Display screen

15: Central management device

D1, D2, D3: Display area

M1: message

M2: Notification message

M3: request signal

O1, O2: data owner

S1: The first group of plural identification codes

S2: The second group of plural identification codes

U1, U2: User

VI, VI1, VI2: Visual User Interface

Claims (16)

An interoperable data exploration device, comprising: a storage device; a transceiver interface; and a processor, which is electrically connected to the storage device and the transceiver interface, and installs a data object package software, wherein the data object package software Including a data processing engine, a database and a visual user interface, the processor executes the data processing engine, wherein the data processing engine continuously inputs a data stream into the database to obtain a first data object , the storage device retains the first data object, wherein the data processing engine causes a display screen to display the visual user interface, the visual user interface includes a first display area and a second display area, the first A display area is used for the first data object, and the data processing engine establishes a connection with another interoperable data exploration device through the transceiving interface, so that the second display area is used for belonging to the other data exploration device. a second data object of an interoperable data exploration device, wherein in response to a first series of operations performed on the second display area, the other interoperable data exploration device emits a first series of querying to perform a first data exploration task on the second data object, thereby narrowing down to a data set of interest, wherein each query in the first series of queries is for the first A result data set of a corresponding previous query in a series of queries is executed, and a first query in the first series of queries is executed against the second data object, wherein the data processing engine passes the The connection receives a first plurality of identification codes corresponding to the data set of interest from the second data object, and in response to the first plurality of identification codes in the first plurality of identification codes A second series of operations performed on a display area, the data processing engine issues a second series of queries to perform a second data exploration task on the first data object, thereby generating a target data set . The interoperable data exploration device of claim 1, wherein the data processing engine converts the first plurality of identifiers one-to-one to a second plurality of identifiers, wherein the target data set is generated according to the second set of the plurality of identification codes. The interoperable data exploration device of claim 1, wherein the second data object is a data set having a plurality of dimensions, a subset of the dimensions includes a plurality of key dimensions, and the The first series of operations performed on the second display area are related to selecting an ordered subset of the dimensions according to the plurality of values, and each operation of the first series of operations is based on the operation corresponding to the previous operation. The ordered subset follows any of the dimensions, and each of the queries of the first series of queries generates a temporary set of identifiers for each of the key dimensions. The interoperable data exploration device of claim 1, wherein in the process of gradually narrowing down to the interested data set, the other interoperable data exploration device responds to a confirmation operation on the second display area and assigning an identifiable code to the interested data set on the second data object, the identifiable code corresponds to the first set of a plurality of identification codes, and the data processing engine further performs the following operations: receiving a a notification message, wherein the notification message is related to the generation of the identifiable code, and a request signal is sent to the other interoperable data exploration device through the connection, and the request signal is used to request the first plurality of identifiers , and receiving the first plurality of identification codes from the other interoperable data exploration device through the connection. The interoperable data exploration device of claim 1, wherein the data processing engine further receives from the second data object through the connection at least one attribute ( attribute), the data processing engine executes on the first display area according to the at least one attribute of the first plurality of identification codes and each of the identification codes of the first plurality of identification codes A second series of operations issue the second series of queries to perform the second data exploration task on the first data object, thereby generating the target data set. An interoperable data exploration device, comprising: a storage device; a transceiver interface; and a processor, which is electrically connected to the storage device and the transceiver interface, and installs a data object package software, wherein the data object package software Including a data processing engine, a database and a visual user interface, the processor executes the data processing engine, wherein the data processing engine continuously inputs a data stream into the database to obtain a first data object , the storage device retains the first data object, wherein the data processing engine causes a display screen to display the visual user interface, the visual user interface includes a first display area and a second display area, the first A display area is used for the first data object, and the data processing engine establishes a connection with another interoperable data exploration device through the transceiving interface, so that the second display area is used for the other interoperable data object. a second data object of the data exploration device, wherein, in response to a first series of operations performed on the first display area, the data processing engine issues a first series of queries on the first data object performing a first data exploration task, thereby gradually narrowing down to a data set of interest, wherein each query in the first series of queries is executed against a result data set of a corresponding previous query in the first series of queries, and a first query in the first series of queries is executed against the first data object, wherein the The data processing engine receives a first set of plural identifiers corresponding to the interested data set from the first data object, and the data processing engine transmits the first plural identifiers to the other interactable through the connection An operational data exploration device, the other interoperable data exploration device issues a second series of operations in response to a second series of operations performed on the second display area according to the first plurality of identification codes A plurality of queries are used to perform a second data exploration task on the second data object, thereby generating a target data set. The interoperable data exploration device of claim 6, wherein the data processing engine converts the first plurality of identifiers one-to-one to a second plurality of identifiers, wherein the target data set is generated according to the second set of the plurality of identification codes. The interoperable data exploration device of claim 6, wherein the first data object is a data set having a plurality of dimensions, a subset of the dimensions includes a plurality of key dimensions, and in the The first series of operations performed on the first display area are related to selecting an ordered subset of the dimensions according to the plurality of values, and each operation of the first series of operations is based on the operation corresponding to the previous operation. The ordered subset follows any of the dimensions, and each of the queries of the first series of queries generates a temporary set of identifiers for each of the key dimensions. The interoperable data exploration device of claim 6, wherein in the process of gradually narrowing down to the data set of interest, the data processing engine searches the first data in response to a confirmation operation on the first display area An identifiable code is assigned to the interested data set on the object, and the identifiable code corresponds to the first set of plural identification codes, and the data processing engine further performs the following operations: sending a notification message to the other through the connection Interoperable data exploration device, wherein the The notification message is about the generation of the identifiable code, receiving a request signal from the other interoperable data exploration device through the connection, the request signal is used to request the first plurality of identification codes, and through the connection The line transmits the first plurality of identification codes to the other interoperable data exploration device. The interoperable data exploration device of claim 6, wherein the data processing engine further transmits at least one attribute of each of the identifiers of the first plurality of identifiers through the connection from the first data object to the other interoperable data exploration device, the other interoperable data exploration device is responsive to the at least one of the identification codes according to the first plurality of identification codes and the first plurality of identification codes attribute and the second series of operations performed on the second display area, issue the second series of queries to perform the second data exploration task on the second data object, thereby generating the target data set. A federated data discovery system, comprising: a central management device; and a first interoperable data discovery device and a second interoperable data discovery device, wherein the first interoperable data discovery device and the The second interoperable data exploration device individually installs a data object package including a data processing engine, a database and a visual user interface, wherein the first interoperable data exploration device continuously updates the A first data stream is input to the corresponding database to obtain a first data object, the first data object is retained, and a first meta information is sent to the central management device to register the first data object , the first interpretation information describes a plurality of first identifiers and a plurality of first attributes, Wherein, the second interoperable data exploration device continuously inputs a second data stream into the corresponding database to obtain a second data object, retains the second data object, and transmits a second interpretation information to The central management device registers the second data object, the second metadata describes a plurality of second identifiers and a plurality of second attributes, wherein the first interoperable data exploration device sends a message to the central management the device subscribes to the second data object, wherein the first interoperable data exploration device causes a display screen to display the corresponding visual user interface, the corresponding visual user interface includes a first display area and a second display area, the first display area is used for the first data object, the first interoperable data exploration device establishes a connection with the second interoperable data exploration device to make the first Two display areas are used for the second data object, wherein, in response to a first series of operations performed on the second display area, the second interoperable data exploration device issues a first series of queries to A first data exploration task is performed on the second data object, thereby gradually narrowing down to a data set of interest, wherein each query in the first series of queries is for a corresponding query in the first series of queries and a first query of the first series of queries is executed against the second data object, wherein the second interoperable data exploration device transmits the A first set of multiple identification codes corresponding to the data set of interest is sent to the central management device, and the central management device transmits a second set of multiple identification codes corresponding to the first set of multiple identification codes to the first possible an interoperable data exploration device, and the first interoperable data exploration device issues a first in response to a second series of operations performed on the first display area according to the second plurality of identifiers Two series of plural queries starting with A second data exploration task is performed on the first data object, thereby generating a target data set. The federated data exploration system of claim 11, wherein the corresponding visual user interface displayed on the display screen further comprises a third display area, and the third display area is used for displaying a plurality of registered data objects, The central management device has a registered metadata of each of the registered data objects, and each of the registered metadata and the first metadata has at least one common identifier description. The federated data discovery system of claim 11, wherein the central management device converts the first plurality of identifiers to the second plurality of identifiers on a one-to-one basis. A federated data discovery system, comprising: a central management device; and a first interoperable data discovery device and a second interoperable data discovery device, wherein the first interoperable data discovery device and the The second interoperable data exploration device individually installs a data object package including a data processing engine, a database and a visual user interface, wherein the first interoperable data exploration device continuously updates the A first data stream is input to the corresponding database to obtain a first data object, retains the first data object, and transmits a first metadata to the central management device to register the first data object. The interpretation information describes a plurality of first identifiers and a plurality of first attributes, wherein the second interoperable data exploration device continuously inputs a second data stream into the corresponding database to obtain a second data object , keep the second data object, and send a second metadata to the central management device to register the second data object, the second metadata describes a plurality of second identifiers and a plurality of second attributes, Wherein, the first interoperable data exploration device sends a message to the central management device to subscribe to the second data object, wherein the first interoperable data exploration device causes a display screen to display the corresponding visualization a user interface, the corresponding visual user interface includes a first display area and a second display area, the first display area is used for the first data object, and the first interactive data exploration device is provided by establishing a connection with the second interoperable data exploration device so that the second display area is used for the second data object, wherein in response to a first series of operations performed on the first display area, The first interoperable data exploration device issues a first series of queries to perform a first data exploration task on the first data object, thereby progressively narrowing down to a data set of interest, wherein the first series Each of the queries of the plurality of queries is executed against a result data set of a corresponding previous query of the first series of queries, and a first query of the plurality of queries of the first series is performed against the first query The data object is executed, wherein the first interoperable data exploration device transmits a first plurality of identification codes corresponding to the data set of interest to the central management device, and the central management device transmits the corresponding to the first A second set of identification codes of the plurality of identification codes is sent to the second interoperable data exploration device, and the second interoperable data exploration device is responsive to the second set of identification codes in the second plurality of identification codes. A second series of operations performed on the second display area issues a second series of queries to perform a second data exploration task on the second data object, thereby generating a target data set. The federated data exploration system of claim 14, wherein the corresponding visual user interface displayed on the display screen further includes a third display area, the third display area is used for Displaying a plurality of registered data objects, the central management device has a registered metadata of each of the registered data objects, and each of the registered metadata and the first metadata has at least one common identification code description. The federated data discovery system of claim 14, wherein the central management device converts the first plurality of identifiers to the second plurality of identifiers on a one-to-one basis.

Images