CN115168504A - A method and device for determining functional dependencies - Google Patents

Abstract

One or more embodiments of the present specification provide a method and an apparatus for determining function dependence, including: acquiring an original data set, wherein the original data set comprises a plurality of tuples, and each tuple comprises field values corresponding to at least two fields; generating a corresponding partition set aiming at each field respectively, wherein the partition set comprises at least one tuple set, and the values of the tuples in each tuple set on the fields corresponding to the partition set are the same; sampling all tuple sets to obtain tuple pairs, wherein each tuple pair comprises two tuples belonging to the same tuple set; generating a corresponding invalid function dependency for each tuple group and adding the invalid function dependency into an invalid function dependency set to obtain all invalid function dependencies corresponding to the sampled tuple groups; and inverting the invalid function dependence set to obtain an effective function dependence set, wherein the effective function dependence set comprises effective function dependence corresponding to the original data set.

Description

A method and device for determining functional dependencies

technical field

One or more embodiments of this specification relate to the field of data processing, and in particular, to a method and apparatus for determining functional dependencies.

Background technique

With the increasing number of data assets, the data processing process has become more and more complex, and the way of manual data processing has been difficult to cope with the explosive growth of enterprise data in the era of big data. When faced with a dataset with a huge number of fields and a wide variety of data types, it is often necessary to determine the relationship between the fields first, and then further process the dataset. Using functional dependencies to characterize the relationship between fields is the mainstream method at this stage.

Although some methods for determining functional dependencies based on data sets are proposed in the related art, they are often inefficient and increase computational and time overhead.

SUMMARY OF THE INVENTION

In view of this, one or more embodiments of the present specification provide a method and apparatus for determining functional dependencies, which can solve the deficiencies in the related art.

To achieve the above purpose, one or more embodiments of this specification provide the following technical solutions:

According to a first aspect of one or more embodiments of the present specification, a method for determining functional dependencies is proposed, and the method includes:

Obtaining an original data set, the original data set includes a plurality of tuples, and each tuple includes field values corresponding to at least two fields;

A corresponding partition set is generated for each field, the partition set includes at least one tuple set, and the tuples in each tuple set have the same field value on the corresponding field of the partition set;

Sampling all tuple sets to obtain tuple pairs, each tuple pair contains two tuples belonging to the same tuple set;

A corresponding invalid functional dependency is generated for each tuple pair and added to the invalid functional dependency set to obtain all invalid functional dependencies corresponding to the sampled tuple pair;

Inverting the invalid functional dependency set to obtain a valid functional dependency set, where the valid functional dependency set includes valid functional dependencies corresponding to the original data set.

According to a second aspect of one or more embodiments of the present specification, a method for reasoning based on a sensitive field of functional dependencies is proposed, and the method includes:

obtaining a functional dependency generated based on a data set, where the functional dependency is used to characterize the relationship of fields contained in the data set;

determine the sensitive fields that the user has flagged in the fields of the dataset;

Determine objective functional dependencies from the functional dependencies, and the fields contained in the left part set of the objective functional dependencies belong to the sensitive fields;

If the field contained in the right part set of the objective functional dependency is different from the sensitive field, it is determined that the field contained in the right part set of the objective functional dependence is a potential sensitive field.

According to a third aspect of one or more embodiments of the present specification, an apparatus for determining functional dependencies is provided, the apparatus comprising:

The first obtaining unit: obtains an original data set, the original data set includes a plurality of tuples, and each tuple includes field values corresponding to at least two fields;

A division unit: respectively generate a corresponding division set for each field, the division set includes at least one tuple set, and the tuples in each tuple set have the same field value on the corresponding field of the division set;

Sampling unit: sampling all tuple sets to obtain tuple pairs, each tuple pair contains two tuples belonging to the same tuple set;

Generating unit: generate corresponding invalid functional dependencies for each tuple pair and add them to the invalid functional dependency set to obtain all invalid functional dependencies corresponding to the sampled tuple pairs;

Inversion unit: inverting the invalid functional dependency set to obtain a valid functional dependency set, where the valid functional dependency set includes valid functional dependencies corresponding to the original data set.

According to a fourth aspect of one or more embodiments of the present specification, an inference device based on a sensitive field of functional dependencies is proposed, the device comprising:

The second obtaining unit: obtains a functional dependency generated based on the data set, the functional dependency is used to characterize the relationship of the fields contained in the data set;

the fourth determining unit: determining the sensitive fields marked by the user in the fields of the data set;

The fifth determination unit: determine the objective functional dependency from the functional dependency, and the fields contained in the left part set of the objective functional dependency belong to the sensitive field;

Determining unit: if the fields included in the right part set of the objective functional dependency are different from the sensitive fields, determine that the fields included in the right part set of the objective functional dependence are potential sensitive fields.

According to a fifth aspect of one or more embodiments of the present specification, an electronic device is proposed, including:

processor;

memory for storing processor-executable instructions;

Wherein, the processor implements the method according to the first aspect or the second aspect by executing the executable instructions.

According to a sixth aspect of one or more embodiments of the present specification, there is provided a computer-readable storage medium having computer instructions stored thereon, the instructions, when executed by a processor, implement the method according to the first aspect or the second aspect A step of.

It can be seen from the above technical solutions that the method for determining functional dependencies provided by one or more embodiments of this specification first obtains an original data set containing multiple tuples, and the tuples in the original data set contain field values corresponding to at least two fields, Next, a division set containing at least one tuple set is generated for each field, and then all tuple sets are sampled to obtain a tuple pair, which contains two tuples belonging to the same tuple set, for each tuple set The corresponding invalid functional dependencies are generated and added to the invalid functional dependency set to obtain all invalid functional dependencies corresponding to the sampled tuple pairs. Since the tuples contained in the same tuple set have the same field value on a certain field, invalid functional dependencies must be generated based on the tuple pairs obtained by sampling. Compared with directly comparing the tuples of the original data set, generating the partition set first can reduce the number of tuple pairs, thereby reducing the computational and time overhead of generating invalid function dependencies through tuple pairs later. The valid functional dependency set is obtained by inverting the invalid functional dependency set, and the valid functional dependency set includes the valid functional dependencies corresponding to the original data set. Compared with directly verifying whether a functional dependency is established, this specification generates functional dependencies to be verified through invalid functional dependency inversion, and uses the invalid functional dependency set to determine valid functional dependencies, which not only reduces the number of functional dependencies that need to be verified, but also simplifies the verification. steps of functional dependencies, thereby improving the efficiency of determining functional dependencies.

Description of drawings

FIG. 1 is a system architecture diagram of a method for determining functional dependencies provided by an exemplary embodiment.

FIG. 2 is a flowchart of a method for determining functional dependencies provided by an exemplary embodiment.

FIG. 3 is a schematic diagram of generating a partition set according to an exemplary embodiment.

FIG. 4 is a schematic diagram of sequential sampling provided by an exemplary embodiment.

FIG. 5 is a schematic diagram of generating an invalid functional dependency according to an exemplary embodiment.

FIG. 6 is a schematic diagram of constructing a binary search tree provided by an exemplary embodiment.

FIG. 7 is a schematic diagram of generating effective functional dependencies according to an exemplary embodiment.

FIG. 8 is a schematic diagram of a sliding serial port sampling provided by an exemplary embodiment.

FIG. 9 is a flowchart of a method for reasoning based on a sensitive field of functional dependencies provided by an exemplary embodiment.

Fig. 10 is a schematic structural diagram of a device provided by an exemplary embodiment.

Fig. 11 is a block diagram of an apparatus for determining functional dependencies provided by an exemplary embodiment.

FIG. 12 is a block diagram of an inference apparatus based on a sensitive field of functional dependencies provided by an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatus and methods consistent with some aspects of one or more embodiments of this specification, as recited in the appended claims.

It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step in other embodiments. describe.

To further illustrate one or more embodiments of this specification, the following examples are provided:

FIG. 1 is a system architecture diagram of a method for determining functional dependencies provided by an exemplary embodiment. As shown in FIG. 1, the system architecture diagram includes: a database 11 and a server 12; the database 11 is used for storing and providing data sets, The server 12 is used to process the data set provided by the database 11 to determine functional dependencies.

Among them, functional dependency means that when one attribute set can determine another attribute set, the other attribute set is said to depend on the attribute set. Introduce functional dependencies from a mathematical point of view: let R(U) be a relational schema on an attribute set U, X and Y are subsets of U, if for any two possible relations r1, r2 of R(U), If r1[x]=r2[x], then r1[y]=r2[y], or if r1[y] is not equal to r2[y], then r1[x] is not equal to r2[x], which is called X decision Y, or Y depends on X.

Taking the scenario of the student table as an example, the name of a student can be determined by the student number of a student, and the student number of the student can also be determined by the name of a student. This situation is called the name attribute depends on the student number attribute, or The number attribute depends on the name attribute, and these two functional dependencies are represented by "{student number}→{name}" and "{name}→{student number}". The name of a student cannot be determined by the age of the student, but the age of the student can be determined by the name of a student. This situation is called the name attribute determines the age attribute, or the age attribute depends on the name attribute, with "{name}→ {age}" indicates the functional dependency. A student's school cannot be determined from a student's age, nor can the student's course be determined from a student's school. In this case, there is no functional dependency between the age attribute and the school attribute.

It can be seen that the relationship pattern between attribute set X and attribute set Y corresponds to functional dependency: when there is a one-to-one relationship between X and Y, such as a one-to-one relationship between student ID and name, there is a one-to-one relationship between X and Y. Functional dependency "X→Y" or "Y→X"; when there is a one-to-many relationship between X and Y, such as a one-to-many relationship between age and name, there is a functional dependency "Y→X"; if There is a many-to-many relationship between X and Y. For example, there is a many-to-many relationship between students and courses, and there is no functional dependency between X and Y.

In the technical solution of this specification, the database 11 can be any type of relational database, and the data set maintained by the database 11 is carried in a table; wherein, each record in the data set is recorded as a row of data in the table, that is, a tuple. For example, in the scenario of the above student table, information such as the name, student ID, age and other information of each student is recorded in a row in the student table, forming a corresponding tuple. At the same time, the columns of the table are fields, and each row of data contains field values corresponding to each field. For example, the information of each student contains field values corresponding to fields such as name, student ID, age, etc., that is, the corresponding student in these fields. Attributes on the dimension represented.

The server 12 may be a physical server comprising an independent host, or a virtual server hosted by a cluster of hosts. During the running process, the server 12 obtains the data set from the database 11, and processes the data set through the technical solution of this specification, thereby determining the functional dependency. Wherein, although in the embodiment shown in FIG. 1, the database 11 is shown as being independent of the server 12, but in some cases, the database 11 can also be deployed in the local storage space of the server 12, which is not limited in this specification .

In fact, the technical solution of the present specification can analyze and obtain corresponding functional dependencies efficiently and quickly by processing the data set, so as to realize further processing purposes according to the obtained functional dependencies. For example, in the scenario of sensitive field analysis, by determining the functional dependency, it can be used to analyze the known sensitive fields, so as to infer potential sensitive fields. The technical solutions of the present specification will be described below with reference to the embodiments.

FIG. 2 is a flowchart of a method for determining functional dependencies provided by an exemplary embodiment. As shown in Figure 2, the method may include the following steps:

Step 201: Obtain an original data set, where the original data set includes multiple tuples, and each tuple includes field values corresponding to at least two fields.

A dataset can refer to a collection of data, usually in tabular form. Each column represents a specific field, each row corresponds to a tuple, and each tuple corresponds to a field value on the field. The original data set can refer to the unprocessed data set, which corresponds to the replacement data set. The process of converting the original data set into the replacement data set after processing will be described in detail in the subsequent steps, and will not be repeated here.

Tuple sequence number Name gender age blood pressure Medication 1 little red Female 11 normal DrugX 2 noob male 20 Low Drug B 3 little black male 65 normal DrugX 4 little blue male 35 high Drug A 5 little green Female twenty four high Drug A 6 little yellow male twenty four normal DrugX

Table 1

Table 1 is used as an example to introduce the method for determining functional dependencies in detail below. Table 1 is a data set provided by an exemplary embodiment. As shown in Table 1, the original data set includes 6 tuples, corresponding to 6 tuples respectively. The tuple serial number "1, 2, 3, 4, 5, 6" also includes 5 different fields, namely "name, gender, age, blood pressure, medication". The 6 tuples have field values corresponding to these 5 fields, for example: a tuple whose tuple serial number is "1", the corresponding field value in the "Name" field is "Xiaohong", and in the "Gender" field The corresponding field value is "female", the corresponding field value on the "age" field is "11", the corresponding field value on the "blood pressure" field is "normal", and the corresponding field value on the "medication" field is "Drug X". The field values of different tuples on the same field can be the same or different. For example, the field value of the tuple with the tuple number "3" on the "Name" field is "Little Black", and the field value on the "Blood Pressure" field. The field value is "Normal", the tuple and the tuple whose tuple sequence number is "1" have different field values in the "Name" field, and the same field value in the blood pressure field. In the subsequent steps, the corresponding invalid functional dependencies are generated by comparing the differences in the field values of each tuple on the same field.

Step 202: Generate a corresponding partition set for each field, the partition set includes at least one tuple set, and the tuples in each tuple set have the same field value on the corresponding field of the partition set.

FIG. 3 is a schematic diagram of generating a partition set provided by an exemplary embodiment. As shown in FIG. 3 , corresponding five partition sets are generated for the five fields in Table 1, and each partition set includes multiple tuple sets. , the tuples in each tuple set have the same field value on the corresponding field of the division set, for example: for the "gender" field, there are only two field values, namely "male" and "female", and in the "gender" field, there are only two field values. The tuple number corresponding to the tuple whose field value is "male" on the "" field is "2, 3, 4, 6", and the tuple number corresponding to the tuple whose field value is "female" on the "gender" field is " 1, 5", then add the tuple corresponding to the tuple number "2, 3, 4, 6" and the tuple corresponding to the tuple number "1, 5" to two different tuple sets, and the two Each tuple set is divided into the "gender" partition set. The final result is shown in Figure 3. There are 5 division sets, corresponding to "name, gender, age, blood pressure, and medication" respectively. There are 2 tuple sets in the "gender" division set, and the first tuple set is "{ 2, 3, 4, 6}", there are 4 tuples, the field value of these 4 tuples on the gender field is "male", the second tuple set "{1, 5}", there are 2 Tuple, the field value of these 2 tuples on the gender field is "female".

In this embodiment, the tuples with the same field value are divided into the same tuple set, so that in the subsequent process of sampling the tuple set, only the tuples with the same field value in a certain field are sampled, reducing the number of The number of sampled tuple pairs is increased, the sampling efficiency is improved, and the computational overhead of subsequent generation function dependencies is further reduced.

Step 203, sampling all tuple sets to obtain tuple pairs, each tuple pair includes two tuples belonging to the same tuple set.

During the sampling process, the partition set or the tuple set in the partition set can be sorted, and the tuple set can be sampled in a certain order, or the tuple set can be sampled randomly without sorting, which is not limited in this specification. .

In an embodiment, each partition set is used as a queue unit, and the tuple sets in each queue unit are sorted according to the efficiency value, the size of the efficiency value and the number of tuples included in the corresponding tuple set. It is positively correlated; when sampling the tuple set in any queue unit, the tuple set in the queue unit is sampled in order according to the efficiency value in descending order to obtain tuple pairs.

Fig. 4 is a schematic diagram of sequential sampling provided by an exemplary embodiment. As shown in Fig. 4, in the "name" partition set, the number of tuples in the tuple set with the field value "male" is 4, and the field value is 4. If the number of tuples in the "female" tuple set is 2, then the efficiency value of the tuple set whose field value is "male" is higher than that of another tuple set, so the tuple set is sampled first. The result of sampling the set of tuples whose field value is "male" is "{2, 3}, {3, 4}, {4, 6}, {2, 4}, {3, 6}, {4 , 6}", the sampling result of the tuple set whose field value is "female" is "{1, 5}", and 7 tuple pairs are obtained. Similarly, sets of tuples in other partition sets are also sorted and sampled according to the efficiency value. The final sampling result should be "{2, 3}, {3, 4}, {4, 6}, {1, 5}, {5, 6}, {1, 3}, {3, 6}, { 4, 5}, {2, 4}, {1, 6}, {2, 6}", a total of 11 tuple pairs.

In this embodiment, the tuple sets are sorted according to the efficiency values of the tuple sets, and the tuple sets with high efficiency values are preferentially sampled, so that most tuple pairs can be extracted in the early stage of sampling, and the growth curve of the entire function dependence can be reduced. The earlier tends to be flat, so that sampling can be ended early, saving sampling time.

Step 204 , generate corresponding invalid functional dependencies for each tuple pair and add them to the invalid functional dependency set, so as to obtain all invalid functional dependencies corresponding to the sampled tuple pairs.

An invalid functional dependency can be a functional dependency that does not hold true. The left part of an invalid functional dependency is all fields whose two tuples have the same field value, and the right part is any corresponding tuple with different field values. a field. In a dataset, fields with the same field value cannot determine fields with different field values. Taking the tuple pair "{2, 3}" as an example, tuple 2 and tuple 3 only have the same field value on the "gender" field, and the field values on the other fields are different. Obviously, through the "gender" field It is impossible to determine the field values of tuple 2 and tuple 3 on other fields, so the functional dependencies generated in this way must be invalid functional dependencies, that is, invalid functional dependencies.

FIG. 5 is a schematic diagram of generating an invalid functional dependency provided by an exemplary embodiment. As shown in FIG. 5 , an invalid functional dependency is generated according to the field value difference of two tuples in a tuple pair on all fields, and the Take the group pair "{2, 3}" as an example, tuple 2 and tuple 3 only have the same field value in the "gender" field, and the field values in the other fields are different, and the "gender" field is regarded as an invalid function dependency The left set of , and the rest of the fields are used as the right set of invalid functional dependencies, so that 4 invalid functional dependencies can be generated, namely "{gender}→{name}, {gender}→{age}, {gender}→{ blood pressure}, {gender}→{medication}". Similarly, the tuples within all tuple pairs are compared to generate invalid functional dependencies. Among them, the functional dependencies obtained by the tuple pairs "{2, 3}, {3, 4}, {4, 6}, {1, 5}, {2, 4}, {2, 6}" are the same. is: {sex}→{name}, {sex}→{age}, {sex}→{blood pressure}, {sex}→{medication}; the functional dependency obtained by tuple pair "{5, 6}" is : {age}→{name}, {age}→{gender}, {age}→{blood pressure}, {age}→{medication}; pass the tuple pair "{1,3}{4,5}{1 , 6}”, the functional dependencies obtained are: {blood pressure, medication}→{name}, {blood pressure, medication}→{gender}, {blood pressure, medication}→{age}; }" The obtained functional dependencies are: {gender, blood pressure, medication}→{name}, {gender, blood pressure, medication}→{age}. In this embodiment, invalid functional dependencies are obtained by comparing the field values on all fields with tuples, which pave the way for subsequent generation of valid functional dependencies.

In the invalid functional dependencies generated by this example, the two functional dependencies "{sex}→{name}" and "{sex, blood pressure, medication}→{name}" have an inclusive relationship, and the right part sets the same case, the left set of the first given invalid functional dependency belongs to the left set of the second invalid functional dependency. In fact, in the case that the "Sex" field, the "Blood pressure" field and the "Medication" field cannot determine the "Name" field, the "Sex" field must not be able to determine the "Name" field, so the first Invalid functional dependencies are actually unnecessary, and such unnecessary invalid functional dependencies will affect the efficiency of subsequent generation of valid functional dependencies.

In one embodiment, determining unnecessary invalid functional dependencies whose left-hand set is contained in the left-hand set of other invalid functional dependencies, the unnecessary right-hand set of invalid functional dependencies The same as the right part set of the other functional dependencies; the adding a candidate field in the left part set of any invalid functional dependency includes: in the left part set of any invalid functional dependency different from the unnecessary invalid functional dependency Add candidate fields.

Further, by constructing a binary search tree with the left set of invalid functional dependencies as a path, unnecessary functional dependencies are pruned, and the efficiency of converting invalid functional dependencies into valid functional dependencies is improved.

Fig. 6 is a schematic diagram of constructing a binary search tree provided by an exemplary embodiment. As shown in Fig. 6, a binary search tree is established for each field. , blood pressure, medication" five fields are represented by "A, B, C, D, E". Take the "blood pressure" field as an example, take the "blood pressure" field as the root node, and find out the invalid functional dependencies whose left set contains the blood pressure field from all invalid functional dependencies. The result is "{blood pressure, medication}→{name}, { blood pressure, medication}→{sex}, {blood pressure, medication}→{age}, {sex, blood pressure, medication}→{name}, {sex, blood pressure, medication}→{age}”. The right-hand set of these 5 invalid functional dependencies contains 3 fields, namely "{name}, {gender}, {age}", so A, B, C are drawn next to the root node (in the actual construction process The right-hand set of invalid functional dependencies does not appear in a binary tree, and this specification displays the right-hand set next to the node for ease of understanding). The left sets of the five invalid functional dependencies all contain "{blood pressure, medication}", so they are connected to form a leaf node E, and A, B, and C are drawn on the edge of node E to represent the right set. Since the left-hand sets of the latter two invalid functional dependencies also contain gender fields, and the right-hand sets of the two invalid functional dependencies here are "{name}, {age}" respectively, the original E node is divided into two, Draw A and C next to the left E node, and draw B on the right E node. Construct the child node B of the left E node, and continue A and C as the right set on the edge of the child node B. As shown in the figure, there are two branches in the constructed binary search tree, which are represented as "DEB→A, DEB→C" and "DE→B", namely "{sex, blood pressure, medication}→{name} , {gender, blood pressure, medication}→{age}, {blood pressure, medication}→{gender}”. It is not difficult to see that the two invalid functional dependencies of "{blood pressure, medication}→{name}, {blood pressure, medication}→{age}" have been cut off, which is not difficult to understand, since the three fields of gender, blood pressure, and medication are The name field cannot be determined, so the two fields of blood pressure and medication cannot be determined. In this embodiment, by constructing a binary search tree with the left part set of invalid functional dependencies as a path, unnecessary invalid functional dependencies are pruned, and the subsequent calculation overhead is reduced.

In the data set, the data types of field values are not necessarily the same. When comparing the field values of two tuples in a tuple pair, it often takes up a lot of storage space due to the data type. Other methods can be used to replace specific field values. to compare.

In an embodiment, the field values corresponding to the multiple tuples in the original data set are respectively replaced by: the serial number of the partition set corresponding to the field to which the corresponding field value belongs and the tuple where the tuple to which the field value belongs is located. The sequence number of the field value formed by the combination of the sequence numbers of the set to generate the replacement data set;

The generating a corresponding invalid functional dependency for each tuple pair includes: generating the invalid functional dependency according to the replacement data set.

Take the tuple with the tuple sequence number "1" as an example. Assume that the five fields of "name, gender, age, blood pressure, and medication" correspond to the sequence numbers "1, 2, 3, 4, and 5" respectively. "The sequence number of the tuple set where the division set is located is 1 (for the convenience of understanding, the sequence number of the tuple set can be sorted according to the writing order of the tuple set in the division set in Figure 2, for example: "Gender" The field value of the division set is " The serial number of the tuple set of "male" is 1, and the serial number of the tuple set of the field value of "female" is 2), then the field value of the tuple in the "name" field can be replaced with "1-1". Similarly, other field values are also replaced accordingly, and the results after replacement are shown in Table 2. When comparing the field values of the two tuples in the tuple pair subsequently, the difference in serial numbers can be directly compared without comparing specific field values.

Tuple sequence number Name gender age blood pressure Medication 1 1-1 2-2 3-2 4-1 5-1 2 1-2 2-1 3-3 4-3 5-3 3 1-3 2-1 3-4 4-1 5-1 4 1-4 2-1 3-5 4-2 5-2 5 1-5 2-2 3-1 4-2 5-2 6 1-6 2-1 3-1 4-1 5-1

Table 2

This embodiment replaces the original field value with the same serial number. On the one hand, considering the characteristics of the underlying storage structure of the computer, the time and space overhead required for calculation is reduced; The original data is transformed to reduce the risk of privacy leakage.

Step 205 , inverting the invalid functional dependency set to obtain a valid functional dependency set, where the valid functional dependency set includes valid functional dependencies corresponding to the original data set.

Generalization refers to a covering relationship between functional dependencies. The left part of the generalization of an invalid functional dependency is included in the left part of the invalid functional dependency, and the right part is the right part of the invalid functional dependency. The same, for example, the invalid function dependency "{gender, blood pressure, medication}→{name}" can completely cover the invalid function dependency "{blood pressure, medication}→{name}", then it can be called "{blood pressure, medication}→{name} " is a generalization of "{gender, blood pressure, medication}→{name}".

In one embodiment, inverting the invalid functional dependency set to obtain a valid functional dependency set includes: extracting any invalid functional dependency in the invalid functional dependency set, a left part of the invalid functional dependency set. is all fields with the same field value in the two tuples contained in the corresponding tuple pair, and the right part is any field in which the two tuples contained in the corresponding tuple pair have different field values; in any of the invalid functional dependencies Add candidate fields to the left set of the to-be-verified functional dependencies, and the candidate fields are other fields in the at least two fields that are different from the fields contained in the left set and the right set of any invalid functional dependency. If the functional dependency to be verified does not belong to the generalization of the invalid functional dependency set and the invalid functional dependency in this set, then determine that the functional dependency to be verified is an effective functional dependency, and the valid functional dependency is Add to the set of valid functional dependencies.

The inversion method is described in detail below with reference to FIG. 7. FIG. 7 is a schematic diagram of generating a valid functional dependency provided by an exemplary embodiment. As shown in FIG. 7, a candidate field is added to the left part of any invalid functional dependency, and verify that it converts to a valid functional dependency. Taking the invalid functional dependency "{blood pressure, medication}→{age}" as an example, the fields in the left and right sets of the invalid functional dependency are the "blood pressure" field, the "age" field, and the "medication" field, so Candidate fields can be the "Gender" field and the "Name" field. Add these two fields to the left set of invalid functional dependencies respectively, and get two functional dependencies to be verified: "{gender, blood pressure, medication}→{age}, {name, blood pressure, medication}→{age}", It can be seen that among these two functional dependencies, the first functional dependency to be verified belongs to the invalid functional dependency set, and the second functional dependency to be verified does not belong to the invalid functional dependency set, so it can be determined that "{name, blood pressure, medication }→{age}" is a valid functional dependency. In this embodiment, candidate fields are added to the invalid functional dependencies to obtain the functional dependencies to be verified, and it is determined whether the functional dependencies to be verified are valid functional dependencies by verifying whether the invalid functional dependencies to be verified belong to the set of invalid functional dependencies. This process converts invalid functional dependencies into valid functional dependencies, without directly verifying whether the functional dependencies are valid, but uses the invalid functional dependency set for judgment, which reduces computational overhead and improves the efficiency of determining functional dependencies.

Further, if the functional dependency to be verified belongs to the set of invalid functional dependencies or the generalization of the invalid functional dependencies in the set, then continue to add in the left set of the functional dependencies to be verified that are different from the ones that have been added. Other candidate fields of candidate fields until the functional dependency to be verified is determined to be the valid functional dependency. Continue to add candidate fields to the second functional dependency to be verified. Since the added candidate field of "{gender, blood pressure, medication}→{age}" is the "gender" field, the candidate field added again should be "name" " field, get the functional dependency to be verified "{name, gender, blood pressure, medication}→{age}", this functional dependency does not belong to the set of invalid functional dependencies, so it can be determined that "{name, gender, blood pressure, medication}→{ age}" is a valid functional dependency. This embodiment enables invalid functional dependencies to be fully converted into valid functional dependencies, avoiding omission of certain functional dependencies.

As mentioned above, in the sampling process, the tuple set can be sorted according to the efficiency value. Since the tuple set with high efficiency value extracts more tuple pairs, the tuple pair extracted from the tuple set with low efficiency value Therefore, when faced with a dataset with a large number of tuples and a large number of fields, if the tuple set with a higher efficiency value is sampled first, it can ensure that most of the tuple pairs can be extracted in the early stage of sampling. In practical situations, in order to ensure the efficiency of sampling, the adequacy of sampling can be sacrificed to a certain extent to improve the efficiency of generating function dependencies.

In one embodiment, performing multiple rounds of sampling on all tuple sets to obtain tuple pairs includes: in the first round of sampling, performing sliding sampling on all tuple sets according to a sampling window of the initial size; or, in non-first rounds of sampling , increase the sampling window on the basis of the previous round, and restart a round of sliding sampling for all tuple sets; wherein, the tuples at both ends of the sliding window are used to generate a tuple pair; if the current round of sliding sampling ends After it is calculated that the growth rate of the invalid function dependency reaches the first preset lower limit threshold, the sampling is ended; otherwise, the next round of sliding sampling is entered.

FIG. 8 is a schematic diagram of a sliding window sampling provided by an exemplary embodiment. As shown in FIG. 8 , a sliding window is used to perform multiple rounds of sampling on a tuple set (because a tuple set with one tuple cannot extract two tuples as tuple pairs do not have the value of sampling, so they are not enqueued here). In the first round of sampling, the initial size of the sampling window is 2. From the set of tuples at the head of the queue, that is, the set of tuples whose field value is "male" in the "gender" partition set, sliding sampling is performed on all sets of tuples. Extract the two tuples at both ends of the sliding window as a tuple pair. The sampling results of the first round of sampling are: {2, 3}, {3, 4}, {4, 6}, {1, 5}, {5, 6}, {1, 3}, {3, 6}, {4, 5}. It can be seen from Figure 4 that the invalid functional dependencies generated by these tuples include all invalid functional dependencies, which will not be described one by one here. Since it is the first round of sampling, the growth rate of invalid functional dependencies is 100%.

In the second round of sampling, the sliding window size is increased to 3, and all tuple sets are traversed. The sampling results of the second round of sampling are: {2, 4}, {3, 6}, {1, 6}. Obviously, the invalid functional dependencies generated by these tuple pairs must be repeated with the invalid functional dependencies generated in the first round, so the growth rate of invalid functional dependencies in this round is 0. If the first preset lower limit threshold is 5%, then the growth rate of the invalid function dependence after the second round has reached the first preset lower limit threshold, and the sampling can be ended. In order to verify that the value of the third round of sampling is not large, the sliding window size is still increased to 4 for the third round of sampling, and the sampling result is: {2, 6}. The invalid functional dependencies generated by the tuple pair "{2, 6}" also duplicate the invalid functional dependencies generated by the first round of sampling. Therefore, it can be seen that with the progress of the sampling round, the sampling efficiency will become lower and lower, and setting the first preset lower limit threshold can reduce the time and calculation overhead while ensuring the accuracy.

Further, if it is calculated that the growth rate of the effective function dependence does not reach the second preset lower threshold after the current round of sliding sampling ends, enter the next round of sliding sampling until the effective function is calculated after any round of sliding sampling. The growth rate of the dependency reaches a preset threshold.

After each round of sampling, it is necessary to calculate not only the growth rate of invalid functional dependencies, but also the growth rate of valid functional dependencies. The growth rate of valid functional dependencies here refers to the newly added invalid functional dependencies after each round of sampling. The growth rate of the effective functional dependence. For example: if after one round of sampling, an invalid function dependency "{age}→{name}" is added, the valid function dependency generated by the invalid function dependency is "{gender, age}→{name}, {blood pressure, age }→{name}, {medication, age}→{name}", if these three valid functional dependencies have not been generated in previous rounds, the growth rate of effective functional dependencies is the number of new effective functional dependencies ( 3) Ratio to the total number of effective functional dependencies (the number of effective functional dependencies generated before plus 3).

In the above embodiment, since all invalid functional dependencies are obtained in the first round of sampling, the growth rate of valid functional dependencies is 0 after the second round of sampling ends, and there is no need to continue sampling. In addition to this situation, the growth rate of the invalid function dependence has reached the first preset lower threshold, but the growth rate of the effective function dependence has not reached the second preset lower threshold. At this time, the sampling will not end, and it is still necessary to Continue sampling. Sampling ends only when the growth rate of the invalid function dependency reaches the first preset lower threshold and the growth rate of the valid function dependency reaches the second preset lower threshold.

While improving the sampling efficiency, this embodiment ensures the adequacy of sampling to a certain extent, thereby reducing the possibility of missing functional dependencies.

In the above-mentioned embodiment, an effective functional dependency can be generated based on the original data set, and the effective functional dependence can be used to characterize the relationship between the fields in the original data set. Take the valid functional dependency "{gender, age}→{name}" as an example, the left part of the functional dependency includes the "gender" field and the "age" field, and the right part includes the "name" field, so the valid functional dependency can be Representation: The field value of the tuple in the "Name" field can be determined by the field value in the "Gender" field and the field value in the "Age" field of a tuple. Due to the special role of functional dependencies, functional dependencies are often used in scenarios where field relationships are determined.

In one embodiment, the user needs to determine the relationship between the "gender" field and the "age" field in the data set shown in Table 1. It is determined that "{sex}→{age}" or "{age}→{sex}" does not exist in the valid functional dependencies, and there are "{sex, blood pressure, medication}→{age}" and "{ age}→{gender}". It can be seen that in the case of determining the field value of a tuple in the "gender" field, the field value of the tuple in the "age" field cannot be determined, and vice versa. Therefore, there is no direct connection between the "Gender" field and the "Age" field, and the two fields cannot be derived from each other.

In another embodiment, the user needs to determine the relationship between the "name" field and the "age" field in the dataset shown in Table 1. It is determined that "{name}→{age}" exists in valid functional dependencies, "{age}→{name}" does not exist, and "{age}→{name}" exists in the set of invalid functional dependencies. It can be seen that in the case of determining the field value of a tuple in the "name" field, the field value of the tuple in the "age" field can be determined; in the case of determining the field value of a tuple in the "age" field , but could not determine the field value of the tuple in the "Name" field. Therefore, the field value of the "age" field can be deduced from the field value of the "name" field, but the field value of the "name" field, the "name" field and the "age" field cannot be deduced from the field value of the "age" field. The relationship is: The "Name" field unilaterally determines the "Age" field.

Specifically, the method of determining functional dependencies can be applied to sensitive field recognition scenarios. In one embodiment, the sensitive fields marked by the user in the at least two fields are determined; the objective functional dependencies are determined from the effective functional dependencies, and the fields contained in the left part set of the objective functional dependencies belong to the sensitive fields ; If the fields contained in the right part set of the objective functional dependency are different from the sensitive fields, then determine that the fields contained in the right part set of the objective functional dependence are potential sensitive fields.

Sensitive fields can refer to data fields that may cause serious harm to society or individuals after leakage, including personal privacy information, such as name, gender, etc. The user can mark sensitive fields by simply marking the fields as non-sensitive fields and non-sensitive fields, or by classifying the fields. For example, the field level is divided into 5 levels, the non-sensitive fields are set to level 0, and the sensitive fields are set to As the sensitivity increases, the higher the field level is set, and the field with the highest sensitivity is level 5. This specification does not limit the specific marking method.

If the sensitive fields marked by the user are: gender and age, the valid functional dependencies with these two sensitive fields as the left set are: {gender, age}→{name}, {gender, age}→{blood pressure}, {gender , age}→{medication}. The right-hand sets of these three valid functional dependencies are the "name" field, the "blood pressure" field, and the "medication" field. These three fields are not sensitive fields, so it can be determined that the "name" field, "blood pressure" field, "" "Medication" fields are all potentially sensitive fields.

This embodiment applies the method for determining functional dependencies to the field of sensitive field identification, determines the relationship between fields based on functional dependencies, and infers potential sensitive fields through functional dependencies and sensitive fields, avoiding manual marking of sensitive fields and reducing time and labor costs .

FIG. 9 is a flowchart of a method for reasoning based on a sensitive field of functional dependencies provided by an exemplary embodiment. As shown in FIG. 9 , the method may include the following steps:

Step 901: Obtain a functional dependency generated based on a data set, where the functional dependency is used to characterize the relationship between fields in the data set.

The way of determining the functional dependency may be implemented by the foregoing embodiments, or may be implemented by other algorithms in the related art, which are not limited in this specification.

Step 902: Determine the sensitive fields marked by the user in the fields of the data set.

Step 903: Determine an objective functional dependency from the functional dependencies, and the fields contained in the left part set of the objective functional dependencies belong to the sensitive fields.

Step 904 , if the fields contained in the right-hand set of the objective functional dependency are different from the sensitive fields, determine that the fields contained in the right-hand set of the objective functional dependence are potential sensitive fields.

Still taking the data set in Table 1 as an example, if the sensitive fields marked by the user are the "age" field and the "blood pressure" field, the functional dependencies with these two fields as the left set include: {blood pressure, age}→{name }, {blood pressure, age}→{gender}, {blood pressure, age}→{medication}. The right sets of these three functional dependencies are the "name" field, the "gender" field and the "medication" field. These three fields are not sensitive fields, so it can be determined that the "name" field, the "gender" field and the "medication" field ” field is a potentially sensitive field.

This embodiment characterizes the relationship between fields in the data set through functional dependencies, so that users can infer potential sensitive fields based on functional dependencies and marked sensitive fields, reducing time and labor costs, and improving the efficiency of sensitive field identification.

Fig. 10 is a schematic structural diagram of a device provided by an exemplary embodiment. Referring to FIG. 10 , at the hardware level, the device includes a processor 1002 , an internal bus 1004 , a network interface 1006 , a memory 1009 and a non-volatile memory 1010 , and may also include hardware required for other functions. One or more embodiments of the present specification may be implemented based on software, for example, the processor 1002 reads the corresponding computer program from the non-volatile memory 1010 into the memory 1008 and then executes it. Of course, in addition to software implementations, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subjects of the following processing procedures are not limited to each Logic unit, which can also be hardware or logic device.

Please refer to FIG. 11 , an apparatus for determining functional dependencies can be applied to the device shown in FIG. 11 to realize the technical solution of this specification, and the apparatus may include:

a first obtaining unit 1101, configured to obtain an original data set, where the original data set includes multiple tuples, and each tuple includes field values corresponding to at least two fields;

The dividing unit 1102 is configured to generate a corresponding dividing set for each field, the dividing set includes at least one tuple set, and the tuples in each tuple set have the same field value on the corresponding field of the dividing set;

Sampling unit 1103, for sampling all tuple sets to obtain tuple pairs, each tuple pair includes two tuples belonging to the same tuple set;

A generating unit 1104, configured to generate a corresponding invalid functional dependency for each tuple pair and add it to the invalid functional dependency set, so as to obtain all invalid functional dependencies corresponding to the sampled tuple pair;

The inversion unit 1105 inverts the invalid functional dependency set to obtain a valid functional dependency set, where the valid functional dependency set includes valid functional dependencies corresponding to the original data set.

Optionally, also include:

The replacement unit 1106 is configured to replace the field values corresponding to the multiple tuples in the original data set with: the serial number of the division set corresponding to the field to which the corresponding field value belongs and the tuple where the tuple to which the field value belongs is located. The sequence number of the field value formed by the combination of the sequence numbers of the set to generate the replacement data set;

The generating unit 1104 is specifically configured to generate the invalid functional dependency according to the replacement data set.

Optionally, the sampling unit 1103 is specifically configured to use each partition set as a queue unit, and sort the tuple sets in each queue unit according to the efficiency value, the size of the efficiency value being related to the corresponding tuple. The number of tuples contained in the set is positively correlated;

When sampling the tuple set in any queue unit, the tuple set in the queue unit is sampled in order according to the efficiency value in descending order to obtain a tuple pair.

Optionally, the sampling unit 1103 is specifically configured to perform sliding sampling on all tuple sets according to the sampling window of the initial size in the first round of sampling; The sampling window is enlarged, and a round of sliding sampling is restarted for all tuple sets; wherein, the tuples at both ends of the sliding window are used to generate a tuple pair;

If it is calculated that the growth rate of the invalid function dependency reaches the first preset lower limit threshold after the current round of sliding sampling ends, the sampling is ended; otherwise, the next round of sliding sampling is entered.

Optionally, the sampling unit 1103 is specifically configured to enter the next round of sliding sampling if it is calculated that the growth rate of the effective function dependence does not reach the second preset lower threshold after the current round of sliding sampling ends, until any wheel sliding. After dynamic sampling, it is calculated that the growth rate of the effective function dependence reaches a preset threshold.

Optionally, the inversion unit 1105 is specifically used for:

Extract any invalid functional dependency in the invalid functional dependency set, the left part of the invalid functional dependency set is all fields with the same field value for the two tuples contained in the corresponding tuple, and the right part is the corresponding tuple any field that has different field values for the two tuples it contains;

A candidate field is added to the left set of any invalid functional dependency to obtain the functional dependency to be verified, and the candidate field is the left set and right set of the at least two fields that are different from the any invalid functional dependency. other fields of the fields contained in the part set;

If the functional dependency to be verified does not belong to the set of invalid functional dependencies and the generalization of the invalid functional dependencies in the set, the functional dependency to be verified is determined to be a valid functional dependency, and the valid functional dependency is added into the set of valid functional dependencies.

Optionally, also include:

The first determination unit 1107 is configured to determine unnecessary invalid functional dependencies, the left part set of the unnecessary invalid functional dependencies is included in the left part sets of other invalid functional dependencies, and the right part of the unnecessary invalid functional dependencies is included. The part set is the same as the right part set of the other functional dependencies;

The inversion unit 1105 is specifically configured to add a candidate field in the left part set of any invalid functional dependency that is different from the unnecessary invalid functional dependency.

Optionally, also include:

The adding unit 1108 is configured to, if the functional dependency to be verified belongs to the set of invalid functional dependencies or a generalization of the invalid functional dependencies in the set, continue to add in the left set of the functional dependencies to be verified the difference from Other candidate fields of candidate fields that have been added until the functional dependency to be verified is determined to be the valid functional dependency.

Optionally, also include:

a second determining unit 1109, configured to determine the sensitive fields marked by the user in the at least two fields;

a third determining unit 1110, configured to determine an objective functional dependency from the effective functional dependency, and the fields contained in the left part set of the objective functional dependency belong to the sensitive field;

The first determining unit 1111 is configured to determine that a field included in the right-hand set of the objective functional dependency is a potential sensitive field if the field included in the right-hand set of the objective functional dependency is different from the sensitive field.

Please refer to FIG. 12 , an inference apparatus based on functional dependency sensitive fields can be applied to the device shown in FIG. 12 to realize the technical solution of this specification, and the apparatus may include:

a second obtaining unit, configured to obtain a functional dependency generated based on a data set, where the functional dependency is used to characterize the relationship of fields included in the data set;

a fourth determining unit, configured to determine the sensitive fields marked by the user in the fields of the data set;

a fifth determining unit, for determining an objective functional dependency from the functional dependency, and the field contained in the left part set of the objective functional dependency belongs to the sensitive field;

The second determination unit is configured to determine that the fields included in the right-hand set of objective functional dependencies are potential sensitive fields if the fields included in the right-hand set of objective functional dependencies are different from the sensitive fields.

The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable, or a combination of any of these devices.

In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory in the form of, for example, read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridges, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The terminology used in one or more embodiments of this specification is for the purpose of describing a particular embodiment only and is not intended to limit the one or more embodiments of this specification. As used in the specification or embodiments and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information without departing from the scope of one or more embodiments of the present specification. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

The above descriptions are only preferred embodiments of one or more embodiments of this specification, and are not intended to limit one or more embodiments of this specification. All within the spirit and principles of one or more embodiments of this specification, Any modifications, equivalent replacements, improvements, etc. made should be included within the protection scope of one or more embodiments of this specification.

Claims (12)

1. a determination method of functional dependence, is characterized in that, described method comprises: Obtaining an original data set, the original data set includes a plurality of tuples, and each tuple includes field values corresponding to at least two fields; A corresponding partition set is generated for each field, the partition set includes at least one tuple set, and the tuples in each tuple set have the same field value on the corresponding field of the partition set; Sampling all tuple sets to obtain tuple pairs, each tuple pair contains two tuples belonging to the same tuple set; A corresponding invalid functional dependency is generated for each tuple pair and added to the invalid functional dependency set to obtain all invalid functional dependencies corresponding to the sampled tuple pair; Inverting the invalid functional dependency set to obtain a valid functional dependency set, where the valid functional dependency set includes valid functional dependencies corresponding to the original data set. 2. The method according to claim 1, wherein The method further includes: respectively replacing the field values corresponding to the plurality of tuples in the original data set with: the serial number of the division set corresponding to the field to which the corresponding field value belongs and the tuple where the tuple to which the field value belongs is located. The sequence number of the field value formed by the combination of the sequence numbers of the set to generate the replacement data set; The generating a corresponding invalid functional dependency for each tuple pair includes: generating the invalid functional dependency according to the replacement data set. 3. The method according to claim 1, wherein the sampling of all tuple sets to obtain tuple pairs, comprising: Taking each partition set as a queue unit, and sorting the tuple sets in each queue unit according to the efficiency value, the size of the efficiency value is positively correlated with the number of tuples included in the corresponding tuple set; When sampling the tuple set in any queue unit, the tuple set in the queue unit is sampled in order according to the efficiency value in descending order to obtain a tuple pair. 4. The method according to claim 1, wherein the sampling of all tuple sets to obtain tuple pairs, comprising: In the first round of sampling, sliding sampling is performed on all tuple sets according to the sampling window of the initial size; or, in the non-first round of sampling, the sampling window is increased on the basis of the previous round, and all tuple sets are Restart a round of sliding sampling; among them, the tuples at both ends of the sliding window are used to generate a tuple pair; If it is calculated that the growth rate of the invalid function dependency reaches the first preset lower limit threshold after the current round of sliding sampling ends, the sampling is ended; otherwise, the next round of sliding sampling is entered. 5. The method according to claim 4, wherein the sampling of all tuple sets to obtain tuple pairs, comprising: If it is calculated after the current round of sliding sampling that the growth rate of the effective functional dependency does not reach the second preset lower threshold, enter the next round of sliding sampling until the increase of the effective functional dependence is calculated after any round of sliding sampling The rate reaches a preset threshold. 6. The method according to claim 1, wherein the inverting the invalid functional dependency set to obtain a valid functional dependency set, comprising: Extract any invalid functional dependency in the invalid functional dependency set, the left part of the invalid functional dependency set is all fields with the same field value for the two tuples contained in the corresponding tuple, and the right part is the corresponding tuple any field that has different field values for the two tuples it contains; A candidate field is added to the left set of any invalid functional dependency to obtain the functional dependency to be verified, and the candidate field is the left set and right set of the at least two fields that are different from the any invalid functional dependency. other fields of the fields contained in the part set; If the functional dependency to be verified does not belong to the set of invalid functional dependencies and the generalization of the invalid functional dependencies in the set, the functional dependency to be verified is determined to be a valid functional dependency, and the valid functional dependency is added into the set of valid functional dependencies. 7. The method of claim 6, wherein The method further includes determining unnecessary invalid functional dependencies whose left-hand set is contained in the left-hand set of other invalid functional dependencies, the unnecessary right-hand set of invalid functional dependencies the same as the right-hand set of said other functional dependencies; The adding a candidate field to the left part set of any invalid functional dependency includes: adding a candidate field to the left part set of any invalid functional dependency that is different from the unnecessary invalid functional dependency. 8. The method of claim 6, further comprising: If the functional dependency to be verified belongs to the set of invalid functional dependencies or a generalization of the invalid functional dependencies in the set, continue to add fields that are different from the candidate fields already added in the left set of the functional dependencies to be verified. other candidate fields until the functional dependency to be verified is determined to be the valid functional dependency. 9. The method of claim 1, further comprising: determining the sensitive fields marked by the user in the at least two fields; Determine objective functional dependencies from the effective functional dependencies, and the fields contained in the left part set of the objective functional dependencies belong to the sensitive fields; If the field contained in the right part set of the objective functional dependency is different from the sensitive field, it is determined that the field contained in the right part set of the objective functional dependence is a potential sensitive field. 10. An inference method based on functional dependency sensitive fields, characterized in that the method comprises: Obtaining a functional dependency generated based on the dataset, the functional dependency is used to characterize the relationship between the fields in the dataset; determine the sensitive fields that the user has flagged in the fields of the dataset; Determine objective functional dependencies from the functional dependencies, and the fields contained in the left part set of the objective functional dependencies belong to the sensitive fields; If the field contained in the right part set of the objective functional dependency is different from the sensitive field, it is determined that the field contained in the right part set of the objective functional dependence is a potential sensitive field. 11. An electronic device, characterized in that, comprising: processor; memory for storing processor-executable instructions; Wherein, the processor implements the method of any one of claims 1-10 by executing the executable instructions. 12. A computer-readable storage medium on which computer instructions are stored, wherein the instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-10.

Images