CN116910650A - Data identification method, device, storage medium and computer equipment - Google Patents

Abstract

The invention relates to the field of data processing technology, and discloses a data identification method, device, storage medium and computer equipment. The method includes: responding to a data identification request, obtaining data filtering rules in the data identification request, and based on the data filtering rules Extract the data to be identified from the source data table, then determine the multiple index items to be identified and the corresponding identification algorithms, and then identify the field contents of the index items to be identified based on the identification algorithm to obtain the identification results. Finally, multiple The recognition results are substituted into the data matching calculation formula for calculation, the calculation results are obtained, and the data category of the data to be identified is determined based on the calculation results. The above method improves the efficiency of data identification and combines multiple dimensions to identify data, improves the accuracy of data identification, accurately obtains the sensitivity identification results of data, and achieves accurate classification and control of sensitive data content.

Description

Data identification methods, devices, storage media and computer equipment

Technical field

The present invention relates to the field of data processing technology, and in particular, to a data identification method, device, storage medium and computer equipment.

Background technique

As various industries gradually standardize and standardize data classification and management, and with the implementation of the data security protection law, enterprises are paying more and more attention to the data stored in the database, especially some data involving sensitive content. Pay attention to intensity. In order to better classify and classify the stored data, the sensitivity of the data content needs to be accurately identified in advance to achieve reasonable management and control of the data.

In the existing technology, the identification of the sensitivity of stored data is still limited to preliminary identification and judgment based on the data content, or a simple judgment of the data content based on two-dimensional composite rules. However, the above two methods are not suitable for large batches of data. When performing sensitivity identification, there is a problem of low data identification efficiency, and the obtained data sensitivity identification results are prone to errors. Sensitive data cannot be accurately identified, and the stored data cannot be accurately identified and classified.

Contents of the invention

In view of this, the main purpose of the data identification method, device, storage medium and computer equipment provided by this application is to solve the technical problems in the prior art of low identification efficiency and low accuracy of identification results of sensitive data identification methods.

According to a first aspect of the present invention, a data identification method is provided, which method includes:

In response to the data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified in the source data table based on the data filtering rules;

Determine multiple index items to be identified of the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified;

Identify the field content of each indicator item to be identified in the data to be identified based on the identification algorithm, and obtain the identification result corresponding to each indicator item to be identified in the data to be identified;

The plurality of identification results of the data to be identified are input into a preset rule matching calculation expression for calculation, the calculation result is obtained, and the sensitivity identification result of the data to be identified is determined based on the calculation result.

According to a second aspect of the present invention, a data identification device is provided, which device includes:

A data extraction module, configured to respond to a data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified in the source data table based on the data filtering rules;

An algorithm confirmation module, used to determine multiple index items to be identified of the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified;

A data identification module, configured to identify the field content of each indicator item to be identified in the data to be identified based on the identification algorithm, and obtain the identification corresponding to each indicator item to be identified in the data to be identified. result;

A result output module is configured to input multiple recognition results of the data to be recognized into a preset rule matching calculation expression for calculation, obtain the calculation results, and determine the number of the data to be recognized based on the calculation results. Sensitivity identification results.

According to a third aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned data identification method is implemented.

According to a fourth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above data identification method is implemented.

The invention provides a data identification method, device, storage medium and computer equipment. This application first responds to a data identification request, obtains the data filtering rules carried in the data identification request, and extracts the data from the source data table based on the data filtering rules. The data to be identified is then determined to identify multiple indicator items of the data to be identified, and the identification algorithm corresponding to each indicator item to be identified is obtained, and then the field content of each indicator item to be identified in the data to be identified is identified based on the identification algorithm. Obtain the identification results corresponding to each index item to be identified in the data to be identified, and finally input the multiple identification results of the data to be identified into the preset rule matching calculation expression for calculation, obtain the calculation results, and determine the identification results based on the calculation results. Sensitivity identification results of identification data.

Before identifying and processing the data, the above method pre-screens the data in the source data table based on the data filtering rules in the data identification request to obtain the data that really needs to be identified. When the amount of data to be identified is large, This method can quickly conduct preliminary screening of large batches of data, effectively improving the efficiency of data processing; then determine multiple indicators to be identified in the data to be identified, and identify each indicator one by one based on the identification algorithm of each indicator to be identified. Targeted identification of the field content of the item, identification of data from multiple indicator dimensions, a more comprehensive understanding of the data, and accurate identification of the data under each indicator dimension; finally, the identification of each indicator item to be identified The results are calculated through rule matching calculation expressions, and the sensitivity identification results of the data to be identified are determined based on the calculation results between multiple indicator dimensions, which improves the data recognition rate. Finally, the data classification results of the data to be identified are obtained, and the sensitive data is accurately identified. data. The above method can improve the efficiency of data identification, especially it can accurately identify large batches of complex data, and combine multiple dimensions to identify data, effectively improve the identification accuracy of data, accurately obtain the sensitivity identification results of data, and realize the identification of sensitive data content. Accurate classification and control.

The above description is only an overview of the technical solutions of the present application. In order to have a clearer understanding of the technical means of the present application, they can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and understandable. , the specific implementation methods of the present application are specifically listed below.

Description of the drawings

The drawings described here are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached picture:

Figure 1 shows a schematic flow chart of a data identification method in an embodiment provided by the present invention;

Figure 2 shows a schematic flow chart of a data identification method in an embodiment provided by the present invention;

Figure 3 shows a principle flow chart of a data identification method in an embodiment provided by the present invention;

Figure 4 shows a schematic structural diagram of a data identification device in an embodiment provided by the present invention;

Figure 5 shows a schematic structural diagram of a data identification device in an embodiment provided by the present invention;

Figure 6 shows a schematic structural diagram of a computer device in an embodiment provided by the present invention.

Detailed ways

Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a thorough understanding of the present application, and to fully convey the scope of the present application to those skilled in the art.

The embodiment of the present application provides a data identification method, as shown in Figure 1. The method includes the following steps:

101. In response to the data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified from the source data table based on the data filtering rules.

Specifically, data filtering rules specifically refer to filtering out data that needs to be identified or analyzed based on specific conditions, and the specific conditions can be numerical ranges, date ranges, or specific values in a certain column of data, etc., using data filtering rules The target data can be determined in advance before data identification, effectively improving the efficiency of data identification.

Furthermore, this application proposes a data identification method that first responds to a data identification request sent by a user. The data identification request carries data filtering rules, and the data filtering rules can be edited and set by the user themselves, or pre-stored. Good rules, but no matter what form of implementation, data filtering rules can help users quickly filter out the data that needs to be identified when processing large batches of data, thereby improving the efficiency of data identification. The scope of the data filtering rules can also be set by the user. The more stringent the data filtering rules are, the smaller the scope of the filtered data will be, which facilitates accurate analysis. Specific to this application, the source data table contains a large amount of data, but not all data needs to be identified. For example, for a source data table configured with a large amount of personnel information, a list of people over 30 years old needs to be identified. To identify, you can then set up data filtering rules. The data content in the age data column is greater than 30, effectively excluding the list of people whose age indicator is less than 30 years old, and avoiding subsequent meaningless data identification. Therefore, you can use data filtering rules to quickly filter Extract multiple pieces of data to be identified in the source data table for subsequent processing.

102. Determine multiple index items to be identified in the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified.

Specifically, the data to be identified contains multiple data indicator items. Each data indicator item can be used as an indicator item to be identified. The indicator item to be identified can be used as a data feature indicator for data recognition analysis, which is used to identify or evaluate the data. One dimension. In the actual data identification process, we need to conduct in-depth mining and analysis of the data content, and filter out representative indicator items to better understand and analyze the data. For each indicator item to be identified, we need a specific identification method to ensure the accuracy and credibility of the data, so as to better support data identification analysis. Among them, the identification algorithm is used to analyze and analyze the data indicator items. Mathematical methods and algorithms for identification, and each data indicator item corresponds to an identification algorithm corresponding to its own field content. Choosing an appropriate identification algorithm is very important for correct data identification and analysis.

In the embodiment of the present application, after extracting the data to be identified, the indicator items to be identified of the data to be identified are determined based on the data indicator items of the data to be identified. The indicator items to be identified can specifically be all data indicator items, or can be selected by the user. Certain data indicator items, that is, users need to understand the data to be identified from certain specific dimensions. After determining the indicator items to be identified, the identification algorithm corresponding to each indicator item to be identified is further obtained to prepare for subsequent identification work. Be prepared. Specifically in this application, each indicator item to be identified represents a dimension for identifying the data to be identified. Obtaining multiple indicator items to be identified for the data to be identified is to determine multiple dimensions for identifying the data to be identified. For example, the database name, resource name, table name, table comment, field name, field comment, data type, data length, data proportion, and deduplication proportion in the data to be identified. Any field and any comment can be used as an identification dimension. , and obtaining the identification algorithm corresponding to each index item to be identified is to further identify the data to be identified from various dimensions, so as to finally obtain accurate identification results of the data to be identified in the current dimension.

103. Identify the field content of each index item to be identified in the data to be identified based on the identification algorithm, and obtain the identification result corresponding to each index item to be identified in the data to be identified.

In the embodiment of this application, after determining the indicator item to be identified and the corresponding identification algorithm, the field content in the indicator item to be identified is extracted, and the corresponding identification algorithm is used to identify the field content to obtain the identification result. For example, when the data to be identified is When it is a series of sales data, we can use a specific identification algorithm, such as a decision tree algorithm, to identify whether the terminal product of a certain indicator item to be identified belongs to a specific product category, thereby determining the sales of the product. By using the identification algorithm corresponding to the indicator item to be identified to perform targeted identification of the field content in the indicator item to be identified, accurate identification results can be obtained, which facilitates a more intuitive and accurate understanding and analysis of the data to be identified.

104. Input multiple identification results of the data to be identified into the preset rule matching calculation expression for calculation, obtain the calculation results, and determine the sensitivity identification result of the data to be identified based on the calculation results.

Specifically, the rule matching calculation expression is used to calculate the rule matching results based on the recognition results and operators of different indicator dimensions. Specifically, the recognition results of each indicator dimension are input into an expression composed of operators, and the results are calculated according to the predefined operations. The rules are calculated and the final rule matching result is obtained. During this process, the identification result of each indicator dimension will be mapped into a numerical or logical value, and the corresponding position in the rule matching calculation expression will be filled in accordingly. It is specifically used to calculate the relationship between the two indicator dimensions. It can also be used To determine whether a certain condition is true.

In the embodiment of the present application, the process of substituting the identification results corresponding to multiple data identification index items into the rule matching calculation expression for calculation refers to performing the identification results of the multiple index items to be identified according to the predefined calculation formula. Combine and calculate to get the calculation result. The calculation result can be a specific value used to evaluate the data to be identified, such as the score of the data in a certain aspect, or it can be a classification label, such as excellent, good, poor, etc., or it can be the division of indicators, such as normal. , risks, etc., and the data categories of the data to be identified can be accurately determined based on different types of calculation results, so as to accurately obtain the classification results of the data to be identified, so as to clearly understand the value and characteristics of the data.

The invention provides a data identification method, device, storage medium and computer equipment. The specific principle flow chart is shown in Figure 3. This application first responds to the data identification request, obtains the data filtering rules carried in the data identification request, and based on The data filtering rule extracts the data to be identified from the source data table, then determines multiple index items to be identified in the data to be identified, and obtains the identification algorithm corresponding to each index item to be identified, and then based on the identification algorithm for each index item to be identified. The field contents of the indicator items to be identified are identified to obtain the identification results corresponding to each indicator item to be identified in the data to be identified. Finally, multiple identification results of the data to be identified are input into the preset rule matching calculation expression for calculation. The calculation results are obtained, and the sensitivity identification results of the data to be identified are determined based on the calculation results.

Before identifying and processing the data, the above method pre-screens the data in the source data table based on the data filtering rules in the data identification request to obtain the data that really needs to be identified. When the amount of data to be identified is large, This method can quickly conduct preliminary screening of large batches of data, effectively improving the efficiency of data processing; then determine multiple indicators to be identified in the data to be identified, and identify each indicator one by one based on the identification algorithm of each indicator to be identified. Targeted identification of the field content of the item, identification of data from multiple indicator dimensions, a more comprehensive understanding of the data, and accurate identification of the data under each indicator dimension; finally, the identification of each indicator item to be identified The results are calculated through rule matching calculation expressions, and the sensitivity identification results of the data to be identified are determined based on the calculation results between multiple indicator dimensions, which improves the data recognition rate. Finally, the data classification results of the data to be identified are obtained, and the sensitive data is accurately identified. data. The above method can improve the efficiency of data identification, especially it can accurately identify large batches of complex data, and combine multiple dimensions to identify data, effectively improve the identification accuracy of data, accurately obtain the sensitivity identification results of data, and realize the identification of sensitive data content. Accurate classification and control.

The embodiment of the present application also provides a data identification method, as shown in Figure 2. The method includes the following steps:

201. In response to the data identification request, obtain the data filtering rules carried in the data identification request.

In the embodiment of this application, a data identification request refers to a request sent by the user to the system through a certain method, such as a network interface, an application program or a human-computer interaction interface, requiring identification operations on certain specific data and returning relevant identifications. As a result, in actual application scenarios, data identification requests are often triggered by a certain business requirement, problem or scenario. At the same time, in different application scenarios, data identification requests may have different forms and carry different contents. In this application, the data identification requests carry data filtering rules, and the data filtering rules are used to filter the data in the source data table. Preliminary screening is performed so that the method provided in this application can be applied to scenarios of processing large batches of data.

202. Extract the data to be identified from the source data table based on the data filtering rules.

Specifically, the source data table includes multiple data records, and each data record includes multiple data indicator items; first, the data filtering rules are obtained, and the data filtering conditions in the data filtering rules are determined, where the data filtering conditions include determination indicator items and Determine the conditions, then match the determination indicator item with multiple data indicator items one by one, extract the data indicator items that are the same as the determination indicator item, and finally determine the field content of the data indicator item according to the determination conditions corresponding to the determination indicator item. If the data If the field content of the indicator item meets the judgment conditions, it is determined that the data record corresponding to the data indicator item meets the data filtering conditions, and the data record is marked as data to be identified.

In the embodiment of this application, the data filtering rules are obtained, and the data filtering conditions in the data filtering rules are determined. The data filtering conditions specifically include determination indicator items and determination conditions. For example, in a source data table, the determination indicator item can be a database Name, resource name, table name, table comment, field name, field comment, data type, data length, data proportion, deduplication proportion, any field, any comment and other indicators, among which the determination indicator items and the determination conditions There can be a variety of determination methods for time. The determination methods can be including, not containing, equal to, not equal to, regular, starting with a specified character, ending with a specified character, between, dictionary and other character contents, or it can also be equal to, less than, Greater than, less than or equal to, greater than or equal to, not equal to, interval, special industry data algorithm, identification algorithm and other quantitative contents, the final form is similar to "library name equals bus" "column name contains add" "data content address identification algorithm" "table comments" "Contains users" and other data filtering conditions, and then match the data indicator items in the existing data records according to the determination indicator items, extract the data indicator items that are completely consistent with the determination indicator items, and then match the field contents of the data indicator items according to the determination conditions Make a judgment. If the field content meets the judgment conditions, it means that this data record is the data to be identified, and the preliminary screening of the data is completed.

Further, the number of data filtering conditions is multiple. First, multiple data records in the source data table are filtered one by one based on multiple data filtering conditions. Then, when there are data records that meet all data filtering conditions at the same time, the data records are extracted. And perform redundant processing on the data records, and finally integrate the redundant processed data records, and mark the integrated data records as data to be identified.

In the embodiment of this application, usually the number of data filtering conditions in the data filtering rules is multiple, and setting a smaller data filtering range is convenient for accurately and quickly finding the data to be identified, so based on each data filtering condition Filter the data records one by one, and finally determine the data records that meet all data filtering conditions. These data records are the data to be identified that need to be identified next. All the obtained data records will be integrated and assembled, and redundant processing will be performed. After eliminating some duplicate and invalid data, the entire data record formed becomes the data to be identified that needs to be identified later.

203. Determine multiple index items to be identified of the data to be identified. The index items to be identified include qualitative index items and quantitative index items.

Specifically, first obtain the source data table information of the data to be identified, generate metadata indicator items based on the source data table information, then obtain the data indicator items of the data to be identified, integrate the metadata indicator items and data indicator items, and obtain the data indicator items to be identified. Qualitative indicator items of the data, and then obtain the field content of each data indicator item in the data to be identified, and perform statistical calculations on the field content based on the preset statistical algorithm to obtain statistical results. Finally, based on the statistical results, generate a quantitative analysis of the data to be identified. indicator items.

In the embodiment of this application, the categories of indicators to be identified are divided into qualitative indicator items and quantitative indicator items, and the qualitative indicator items are further divided into metadata indicator items and data indicator items. Specifically, when the data to be identified is obtained, the source data table information is obtained based on the source of the data to be identified, that is, the source data table to which the data to be identified belongs. The source data table information specifically includes the database name of the database where the source data table is located. The table name of the source data table, and the table comments of the source data table and other information. The above information is the metadata indicator item of the data to be identified, and the data indicator items of the data to be identified correspond to the column comments and column names in the data to be identified. and other related information, the qualitative indicator items are composed of source data indicator items and data indicator items. By performing data processing on the field content in the qualitative index items of the data to be identified, the specific quantitative index items of the data to be identified can be obtained. For example, statistical calculations are performed on the field content through statistical algorithms to obtain data length, data proportion, and deduplication. Quantitative indicators such as proportions are used as quantitative indicator items in the data table to be identified. Finally, the qualitative index items and the quantitative index items are integrated to obtain the index items to be identified in the data table to be identified. The subsequent identification process of the data to be identified is based on the index items to be identified.

204. Obtain the identification algorithm corresponding to each index item to be identified in the data to be identified, identify the field content of each index item to be identified in the data to be identified based on the identification algorithm, and obtain the identification algorithm corresponding to each index item to be identified in the data to be identified. Recognition results.

Specifically, first obtain the identification algorithm corresponding to the indicator item to be identified, where the identification algorithm includes at least one of a string matching algorithm, a feature extraction algorithm, a statistical learning algorithm, and a machine learning algorithm, and then extract the field content of the indicator item to be identified. , determine the fields to be identified in the field content according to the identification algorithm corresponding to the indicator items to be identified, and finally identify the fields to be identified based on the identification algorithm to obtain the identification results of the indicator items to be identified.

In the embodiment of this application, each index item to be identified in the data to be identified corresponds to an identification algorithm, and commonly used identification algorithms specifically include performing sensitive content on the data by matching specified strings, keywords, regular expressions, etc. String matching algorithm for identification and filtering; feature extraction algorithm for identifying sensitive content by adopting different methods to extract, analyze, model and mine feature information in the data; through statistical analysis and modeling of data, establish Statistical models, statistical learning algorithms that can identify and classify sensitive content in data, and machine learning algorithms that use machine learning sum models to train and learn data to predict and classify sensitive content in data. . After obtaining the respective identification algorithms of the indicator items to be identified, based on the data categories to be identified by different identification algorithms, the fields to be identified in the field content of the indicator items to be identified are extracted and identified, and finally the corresponding identification of each indicator item to be identified is obtained. result.

Specifically, as shown in Figure 3, each indicator item to be identified represents a dimension of data identification. Based on each indicator item to be identified, the data to be identified is processed and identified, and the data to be processed can be accurately understood and identified from one dimension. This application combines the identification of field contents in multiple index items to be identified to obtain multi-dimensional data identification results to comprehensively and accurately identify data.

205. Edit the rule matching calculation expression, and substitute multiple recognition results into the rule matching calculation expression for calculation.

Specifically, in response to the expression editing instruction, the editing rules of the rule matching calculation expression are obtained. First, based on the editing rules, operators associated with multiple indicator items to be identified are obtained, wherein the number of operators is at least one, and The operators include at least one of arithmetic operators, relational operators and logical operators, and then multiple index items to be identified and at least one operator are synthesized to obtain a rule matching calculation expression.

In the embodiment of this application, this application identifies data from multiple dimensions, and the specific identification results need to be determined by integrating the identification results of each dimension. Therefore, after obtaining the identification results of each indicator item to be identified, it is necessary to synthesize rules Match calculation expressions to perform comprehensive calculations on each result to finally obtain data recognition results that combine various dimensions. The specific editing process of the rule matching calculation expression requires obtaining the operators associated with each indicator item to be identified based on the previously determined indicator items to be identified. The operators here support commonly used arithmetic operators, relational operators and logical operations. Operators can support various calculation methods. After determining the operators associated with all indicator items to be identified, all indicator items to be identified and the operators are synthesized to obtain the data matching calculation formula. For example, when the operator When it is a logical operator, the rule matching calculation expression generated based on the editing of the data indicator item to be identified can be (1||2)&&3, that is, condition 1 or condition 2 is met, and condition 3 is met, or the rule matching calculation expression It can be 1||2||3||4, that is, it satisfies condition 1, condition 2, condition 3 and condition 4 at the same time. Among them, 1234 in the formula are the identification results of each data index item to be identified.

206. Obtain the calculation result, and compare the calculation result with the preset sensitivity threshold to determine the sensitivity identification result of the data to be identified.

Specifically, a preset sensitivity threshold is obtained, and the calculation results are compared based on the sensitivity threshold; when the calculation result is greater than the sensitivity threshold, the calculation result is marked as the first data category, and sensitivity is added to the first data category. Data label of the content; when the calculation result is less than or equal to the sensitivity threshold, mark the calculation result as the second data category, and add a data label of non-sensitive content to the second data category.

In the embodiment of this application, based on the calculation results obtained by different data matching calculation formulas, the sensitivity thresholds used to distinguish different data categories are used to judge the calculation results, and finally complete the specific identification and classification of the data to be identified, and determine Whether the data to be identified is sensitive content. For example, when the calculation result is a specific value, the preset sensitivity threshold is also a specific value. The two specific values are compared. When the calculation result is greater than the preset threshold, that is The data to be identified can be divided into the first data category, and the data to be identified is determined to be sensitive content. When the value of the calculation result is less than or equal to the preset sensitivity threshold, the data to be identified can be divided into the second data category. Determine that the data to be identified is sensitive content. The setting of the preset sensitivity threshold allows users to quickly analyze the calculation results, accurately obtain the accurate data category of the data to be identified, complete the entire data analysis process, and quickly determine whether the data to be identified is sensitive content, which facilitates follow-up. for processing.

The invention provides a data identification method, device, storage medium and computer equipment. This application first responds to a data identification request, obtains the data filtering rules carried in the data identification request, and extracts the data from the source data table based on the data filtering rules. The data to be identified is then determined to identify multiple indicator items of the data to be identified. The indicator items to be identified include qualitative indicator items and quantitative indicator items. Then the identification algorithm corresponding to each indicator item to be identified of the data to be identified is obtained, based on the identification algorithm. Identify the field content of each index item to be identified in the data to be identified, obtain the identification results corresponding to each index item to be identified in the data to be identified, then synthesize the rule matching calculation expression, and substitute the multiple identification results into the rule matching calculation expression The calculation is performed in the formula, and finally the calculation result is obtained, and the sensitivity recognition result of the data to be identified is determined by comparing the preset sensitivity threshold with the calculation result.

The above method determines the data to be identified based on multiple data filtering conditions in the data filtering rules, and completes the accurate screening of large batches of data; determines multiple index items to be identified for the data to be identified, specifically including qualitative index items and quantitative index items. Then use the recognition algorithm corresponding to the indicator item to be identified to carry out targeted identification of the field content, and obtain accurate identification results; finally, a rule matching calculation expression is synthesized based on the indicator item to be identified, and each identification result is processed through the rule matching calculation expression Calculate, and compare the calculated results with the preset sensitivity threshold to determine whether the data to be identified is sensitive content. The above method effectively improves the efficiency of processing large batches of data, and can combine multiple dimensions to identify data, improve the identification accuracy of data, and finally use the preset sensitivity threshold to accurately determine whether the data to be identified is sensitive content.

Further, as a specific implementation of the method in Figure 1, the embodiment of the present application provides a data identification device, as shown in Figure 4. The device includes: a data extraction module 301, an algorithm confirmation module 302, a data identification module 303, and a result output module. 304.

The data extraction module 301 can be used to respond to the data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified in the source data table based on the data filtering rules;

The algorithm confirmation module 302 can be used to determine multiple index items to be identified in the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified;

The data identification module 303 can be used to identify the field content of each index item to be identified in the data to be identified based on the identification algorithm, and obtain the identification result corresponding to each index item to be identified in the data to be identified;

The result output module 304 can be used to input multiple recognition results of the data to be recognized into a preset rule matching calculation expression for calculation, obtain the calculation results, and determine the sensitivity recognition result of the data to be recognized based on the calculation results.

In specific application scenarios, the data extraction module 301 can be used to obtain data filtering rules and determine data filtering conditions in the data filtering rules, where each data filtering condition includes a determination indicator item and a determination condition; combine the determination indicator item with multiple The data index items are matched one by one, and the data index items that are the same as the judgment index items are extracted; the field content of the data index items is judged according to the judgment conditions corresponding to the judgment index items. If the field content of the data index items meets the judgment conditions, then it is determined The data record corresponding to the data indicator item meets the data filtering conditions, and the data record is marked as data to be identified.

In specific application scenarios, the data extraction module 301 can also be used to filter multiple data records in the source data table one by one based on multiple data filtering conditions; when a data record exists and meets all data filtering conditions, the data record is extracted. And perform redundant processing on the data records; integrate the redundant processed data records, and mark the integrated data records as data to be identified.

In a specific application scenario, the algorithm confirmation module 302 can be used to obtain the source data table information of the data to be identified, and generate metadata indicator items based on the source data table information; obtain the data indicator items of the data to be identified, and convert the metadata indicator items into Integrate with the data indicator items to obtain the qualitative indicator items of the data to be identified; obtain the field content of each data indicator item in the data to be identified, and perform statistical calculations on the field content based on the preset statistical algorithm to obtain the calculation results; based on calculation As a result, quantitative indicator items of the data to be identified are generated.

In specific application scenarios, the data identification module 303 can also be used to obtain the identification algorithm corresponding to the indicator item to be identified, where the identification algorithm includes at least one of a string matching algorithm, a feature extraction algorithm, a statistical learning algorithm, and a machine learning algorithm. ; Extract the field content of the indicator item to be identified, and determine the field to be identified in the field content according to the identification algorithm corresponding to the indicator item to be identified; identify the field to be identified based on the identification algorithm, and obtain the identification result of the indicator item to be identified.

In a specific application scenario, as shown in Figure 5, this application also includes a formula editing module 305. The formula editing module 305 can also be used to respond to expression editing instructions and obtain editing rules for rule matching calculation expressions; based on the editing rules , obtain the operators associated with multiple indicator items to be identified, where the number of operators is at least one, and the operators include at least one of arithmetic operators, relational operators and logical operators; for multiple to-be-identified indicator items, The identified indicator item is combined with at least one operator to obtain a rule matching calculation expression.

In a specific application scenario, the result output module 304 can be used to obtain a preset sensitivity threshold, and compare the calculation results based on the sensitivity threshold; when the calculation result is greater than the sensitivity threshold, mark the calculation result as the first One data category, and add a data label of sensitive content to the first data category; when the calculation result is less than or equal to the sensitivity threshold, mark the calculation result as a second data category, and add data of non-sensitive content to the second data category Label.

It should be noted that, for other corresponding descriptions of the functional units involved in the data identification device provided in this embodiment, reference can be made to the corresponding descriptions in Figures 1 and 2, which will not be described again here.

Based on the above method shown in Figure 1, correspondingly, this embodiment also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the above data identification method is implemented.

Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.). It includes several instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute each implementation scenario data identification method of this application.

Based on the above methods shown in Figures 1 and 2, and the data identification device embodiments shown in Figures 4 and 5, in order to achieve the above purpose, as shown in Figure 6, this embodiment also provides an entity for data identification The device includes a communication bus, a processor, a memory and a communication interface, and may also include an input-output interface and a display device, wherein each functional unit can communicate with each other through the bus. The memory stores a computer program and a processor, which is used to execute the program stored in the memory and execute the data identification method in the above embodiment.

Optionally, the physical device may also include a user interface, a network interface, a camera, a radio frequency (Radio Frequency, RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. The optional user interface may also include a USB interface, a card reader interface, etc. Optional network interfaces may include standard wired interfaces, wireless interfaces (such as WI-FI interfaces), etc.

Those skilled in the art can understand that the structure of a data recognition physical device provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or arrange different components. .

The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the above-mentioned physical device hardware and software resources to be identified, and supports the operation of information processing programs and other software and/or programs to be identified. The network communication module is used to realize communication between components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or can also be implemented by hardware. By applying the technical solution of this application, first in response to the data identification request, the data filtering rules carried in the data identification request are obtained, and based on the data filtering rules, the data to be identified is extracted from the source data table, and then multiple elements of the data to be identified are determined. The indicator items to be identified, and the identification algorithm corresponding to each indicator item to be identified is obtained, and then the field content of each indicator item to be identified in the data to be identified is identified based on the identification algorithm, and the corresponding indicator item to be identified in the data to be identified is obtained The identification results are finally input into the preset rule matching calculation expressions for calculation, and the calculation results are obtained, and the sensitivity identification results of the data to be identified are determined based on the calculation results.

Before identifying and processing the data, the above method pre-screens the data in the source data table based on the data filtering rules in the data identification request to obtain the data that really needs to be identified. When the amount of data to be identified is large, This method can quickly conduct preliminary screening of large batches of data, effectively improving the efficiency of data processing; then determine multiple indicators to be identified in the data to be identified, and identify each indicator one by one based on the identification algorithm of each indicator to be identified. Targeted identification of the field content of the item, identification of data from multiple indicator dimensions, a more comprehensive understanding of the data, and accurate identification of the data under each indicator dimension; finally, the identification of each indicator item to be identified The results are calculated through rule matching calculation expressions, and the sensitivity identification results of the data to be identified are determined based on the calculation results between multiple indicator dimensions, which improves the data recognition rate. Finally, the data classification results of the data to be identified are obtained, and the sensitive data is accurately identified. data. The above method can improve the efficiency of data identification, especially it can accurately identify large batches of complex data, and combine multiple dimensions to identify data, effectively improve the identification accuracy of data, accurately obtain the sensitivity identification results of data, and realize the identification of sensitive data content. Accurate classification and control.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred implementation scenario, and the modules or processes in the accompanying drawing are not necessarily necessary for implementing the present application. Those skilled in the art can understand that the modules in the devices in the implementation scenario can be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more devices different from the implementation scenario. The modules of the above implementation scenarios can be combined into one module or further split into multiple sub-modules.

The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. What is disclosed above are only a few specific implementation scenarios of the present application. However, the present application is not limited thereto. Any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims (10)

1. A data identification method, characterized in that the method includes: In response to the data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified in the source data table based on the data filtering rules; Determine multiple index items to be identified of the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified; Identify the field content of each indicator item to be identified in the data to be identified based on the identification algorithm, and obtain the identification result corresponding to each indicator item to be identified in the data to be identified; The plurality of identification results of the data to be identified are input into a preset rule matching calculation expression for calculation, the calculation result is obtained, and the sensitivity identification result of the data to be identified is determined based on the calculation result. 2. The method according to claim 1, characterized in that the source data table includes multiple data records, and each of the data records includes multiple data indicator items; the source data is filtered based on the data filtering rules. The data to be identified is extracted from the table, including: Obtain the data filtering rules, and determine the data filtering conditions in the data filtering rules, where the data filtering conditions include determination indicator items and determination conditions; Match the determination indicator item with multiple data indicator items one by one, and extract the data indicator items that are the same as the determination indicator item; The field content of the data indicator item is determined according to the determination condition corresponding to the determination indicator item. If the field content of the data indicator item meets the determination condition, the field content corresponding to the data indicator item is determined. The data records satisfy the data filtering conditions, and the data records are marked as data to be identified. 3. The method according to claim 2, characterized in that the number of the data filtering conditions is multiple; if the field content of the data index item satisfies the determination condition, then the data index item is determined The corresponding data record satisfies the data filtering conditions, and the data record is marked as data to be identified, including: Filter multiple data records in the source data table one by one based on multiple data filtering conditions; When the data record exists and satisfies all the data filtering conditions, extract the data record and perform redundant processing on the data record; The data records after redundancy processing are integrated, and the integrated data records are marked as data to be identified. 4. The method according to claim 1, wherein the indicator items to be identified include qualitative indicator items and quantitative indicator items; and the determination of multiple indicator items to be identified of the data to be identified includes: Obtain the source data table information of the data to be identified, and generate metadata indicator items based on the source data table information; Obtain the data indicator items of the data to be identified, integrate the metadata indicator items and the data indicator items, and obtain the qualitative indicator items of the data to be identified; Obtain the field content of each data indicator item in the data to be identified, and perform statistical calculations on the field content based on a preset statistical algorithm to obtain statistical results; Based on the statistical results, quantitative index items of the data to be identified are generated. 5. The method according to claim 1, characterized in that the field content of each index item to be identified in the data to be identified is identified based on the identification algorithm to obtain the field content of the index item to be identified. The identification results corresponding to each indicator item to be identified include: Obtain the identification algorithm corresponding to the indicator item to be identified, wherein the identification algorithm includes at least one of a string matching algorithm, a feature extraction algorithm, a statistical learning algorithm, and a machine learning algorithm; Extract the field content of the indicator item to be identified, and determine the field to be identified in the field content according to the identification algorithm corresponding to the indicator item to be identified; The field to be identified is identified based on the identification algorithm, and the identification result of the indicator item to be identified is obtained. 6. The method according to claim 1, characterized in that, before inputting the plurality of recognition results of the data to be recognized into a preset rule matching calculation expression for calculation, the method further include: In response to the expression editing instruction, obtain the editing rule for the rule matching calculation expression; Based on the editing rules, operators associated with a plurality of the indicator items to be identified are obtained, wherein the number of the operators is at least one, and the operators include arithmetic operators, relational operators and logical operations at least one of the symbols; A plurality of the indicator items to be identified and at least one of the operators are synthesized to obtain the rule matching calculation expression. 7. The method of claim 1, wherein determining the sensitivity identification result of the data to be identified based on the calculation result includes: Obtain a preset sensitivity threshold, and compare the calculation results based on the sensitivity threshold; When the calculation result is greater than the sensitivity threshold, mark the calculation result as a first data category, and add a data label of sensitive content to the first data category; When the calculation result is less than or equal to the sensitivity threshold, the calculation result is marked as a second data category, and a data label of non-sensitive content is added to the second data category. 8. A data identification device, characterized in that the device includes: A data extraction module, configured to respond to a data identification request, obtain the data filtering rules carried in the data identification request, and extract the data to be identified in the source data table based on the data filtering rules; An algorithm confirmation module, used to determine multiple index items to be identified of the data to be identified, and obtain the identification algorithm corresponding to each index item to be identified; A data identification module, configured to identify the field content of each indicator item to be identified in the data to be identified based on the identification algorithm, and obtain the identification corresponding to each indicator item to be identified in the data to be identified. result; A result output module is configured to input multiple recognition results of the data to be recognized into a preset rule matching calculation expression for calculation, obtain the calculation results, and determine the number of the data to be recognized based on the calculation results. Sensitivity identification results. 9. A storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented. 10. A computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that when the computer program is executed by the processor, any one of claims 1 to 7 is realized. The steps of the method described in the item.