CN112000699B - Data processing method creation method and device, data information processing method and device - Google Patents
Data processing method creation method and device, data information processing method and device Download PDFInfo
- Publication number
- CN112000699B CN112000699B CN201910446967.5A CN201910446967A CN112000699B CN 112000699 B CN112000699 B CN 112000699B CN 201910446967 A CN201910446967 A CN 201910446967A CN 112000699 B CN112000699 B CN 112000699B
- Authority
- CN
- China
- Prior art keywords
- information
- data
- data processing
- target
- tag
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24573—Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2445—Data retrieval commands; View definitions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0486—Drag-and-drop
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请公开一种数据处理方式的创建方法及装置，数据信息的处理方法及装置，计算机存储介质和电子设备。其中，所述创建方法包括：获取标签信息，所述标签信息用于描述实体的属性信息或者是实体关系的属性信息；将所述标签信息输出至配置界面；基于已有所述标签信息的配置界面，接收数据处理条件；将所述数据处理条件和所述标签信息保存为数据处理时的调用信息；从而基于已有标签数据的能够便捷快速的进行数据挖掘和数据分析，并生成目标数据。
The present application discloses a method and device for creating a data processing method, a method and device for processing data information, a computer storage medium and an electronic device. The creation method includes: obtaining label information, the label information is used to describe the attribute information of an entity or the attribute information of an entity relationship; outputting the label information to a configuration interface; receiving data processing conditions based on the configuration interface with the label information; saving the data processing conditions and the label information as call information for data processing; thereby enabling data mining and data analysis to be performed quickly and conveniently based on the existing label data, and generating target data.
Description
Technical Field
The application relates to the technical field of computer application, in particular to a method and a device for creating a data processing mode. The application also relates to a data information processing method and device.
Background
With the continuous development of the internet, a big data age has been formed, namely, data is in a form of sea quantization and diversification. In order to better analyze and understand data information, in the prior art, the data can be labeled, so that the analysis of the data is realized, and a user can read the needed data information conveniently.
The tag (tag) may be an attribute of an entity or an attribute of a relationship between entities. By marking and modeling the data of the user, the data of the user is displayed in the form of labels, so that a view which is easier to understand and use by the user can be provided, and the functions of fast label data viewing, processing, statistics metering, service release and the like are provided.
However, after user data is modeled using labels, there are problems in terms of:
1. The data of the original label does not meet the application requirement of the new service scene, so that the effect of the original label is reduced.
2. The data of the original label and the new label generated by processing are not in one data source, so that the data information is obtained inaccurately.
Disclosure of Invention
The application provides a method for creating a data processing mode, which aims to solve the problem that the prior art cannot meet the data analysis and use requirements of the original label.
The application provides a method for creating a data processing mode, which comprises the following steps:
acquiring tag information, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relationship;
Outputting the label information to a configuration interface;
receiving data processing conditions based on a configuration interface of the existing tag information;
and storing the data processing conditions and the tag information as call information in data processing.
In some embodiments, the acquiring tag information includes:
selecting a data source to be processed from the provided selectable data sources;
And acquiring the tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source.
In some embodiments, further comprising:
Determining whether a data partition exists in the data source;
if yes, setting partition values of the data partitions;
the obtaining the tag information according to at least one of the entity information and the entity relationship information determined in the data source includes:
and acquiring the label information according to at least one of entity information, entity relation information and label custom function information which are included in the data partition corresponding to the partition value.
In some embodiments, further comprising:
and selecting a target data source according to the label information and the data processing conditions.
In some embodiments, the selecting a target data source based on the tag information and the data processing conditions comprises:
pre-selecting the label information which is expected to be acquired;
Detecting the pre-selected label information according to the data processing conditions;
And if the detection result is normal, selecting the pre-selected label information as a target data source.
In some embodiments, the saving the data processing condition and the tag information as call information at the time of data processing includes:
Determining whether to configure the target entity and the target tag based on the selected target data source;
if yes, configuring a target entity and a target label for the target data source;
And storing the configured target data source and the data processing condition as call information in data processing.
In some embodiments, the configuration interface based on the label information receives a data processing condition, including:
According to the requirement of the target label to be generated, determining screening conditions based on the label information on the configuration interface;
and determining the screening condition as the received data processing condition.
In some embodiments, the determining, on the configuration interface, a filtering condition based on the label information according to the requirement of the target label to be generated includes:
and selecting or setting screening conditions based on the existing label information according to the requirements of the target label to be generated.
In some embodiments, the outputting the tag information to a configuration interface includes:
and outputting the tag information to a configuration interface through the moving operation of the tag information.
In some embodiments, further comprising:
Converting the data processing conditions and the tag information into tag query sentences;
The step of storing the data processing condition and the tag information as call information in data processing includes:
And saving the data processing conditions and the label inquiry statement corresponding to the label information as call information in data processing.
The application also provides a device for creating the data processing mode, which comprises the following steps:
The information processing device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring tag information, and the tag information is used for describing attribute information of an entity or attribute information of an entity relationship;
The output unit is used for outputting the label information to a configuration interface;
the receiving unit is used for receiving data processing conditions based on the configuration interface of the existing tag information;
and the storage unit is used for storing the data processing conditions and the tag information as call information in data processing.
The application also provides a data information processing method, which comprises the following steps:
Creating calling information which is called when data processing is performed based on the tag information and the data processing conditions;
generating a target table for storing the processed data according to the submitted parameters recorded in the calling information;
And calling the calling information, carrying out data processing on the tag data to be processed, and storing the processed data in the target table.
In some embodiments, the creating call information that is called when the data processing is performed based on the tag information and the data processing condition includes:
acquiring tag information, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relationship;
Outputting the label information to a configuration interface;
receiving data processing conditions based on a configuration interface of the existing tag information;
and storing the data processing conditions and the tag information as call information in data processing.
In some embodiments, the acquiring tag information includes:
selecting a data source to be processed from the provided selectable data sources;
And acquiring the tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source.
In some embodiments, further comprising:
Determining whether a data partition exists in the data source;
if yes, setting partition values of the data partitions;
the obtaining the tag information according to at least one of the entity information and the entity relationship information determined in the data source includes:
and acquiring the label information according to at least one of entity information, entity relation information and label custom function information which are included in the data partition corresponding to the partition value.
In some embodiments, further comprising:
and selecting a target data source according to the label information and the data processing conditions.
In some embodiments, the selecting a target data source based on the tag information and the data processing conditions comprises:
pre-selecting the label information which is expected to be acquired;
Detecting the pre-selected label information according to the data processing conditions;
And if the detection result is normal, selecting the pre-selected label information as a target data source.
In some embodiments, the saving the data processing condition and the tag information as call information at the time of data processing includes:
Determining whether to configure the target entity and the target tag based on the selected target data source;
if yes, configuring a target entity and a target label for the target data source;
And storing the configured target data source and the data processing condition as call information in data processing.
In some embodiments, the configuration interface based on the label information receives a data processing condition, including:
According to the requirement of the target label to be generated, determining screening conditions based on the label information on the configuration interface;
and determining the screening condition as the data processing condition.
In some embodiments, the determining, on the configuration interface, a filtering condition based on the label information according to the requirement of the target label to be generated includes:
and selecting or setting screening conditions based on the existing label information according to the requirements of the target label to be generated.
In some embodiments, the outputting the tag information to a configuration interface includes:
and outputting the tag information to a configuration interface through the moving operation of the tag information.
In some embodiments, further comprising:
Converting the data processing conditions and the tag information into tag query sentences;
The step of storing the data processing condition and the tag information as call information in data processing includes:
And saving the data processing conditions and the label inquiry statement corresponding to the label information as call information in data processing.
In some embodiments, the generating a target table for storing the processed data according to the parameters recorded in the submitted call information further includes:
Determining whether the communication path of the data source and the communication path of the target data source in the calling information are normal or not;
And if the data is normal, generating a target table for storing the processed data according to the parameters recorded in the calling information.
In some embodiments, the generating a target table for storing the processed data according to the parameters recorded in the submitted call information further includes:
Determining whether the target table is bound with the processed data;
If yes, binding the target table with at least one of the processed target entity information and the processed target entity relation information, and binding the list of the target table with the processed target label.
In some embodiments, the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table, further includes:
When the label data to be processed is processed, determining whether a data processing period is configured;
If yes, carrying out data processing on the label data to be processed according to the configured data processing period, and storing the processed data into the target table.
In some embodiments, the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table, further includes:
acquiring at least one of entity data and entity relation data to be processed, and tag data;
importing at least one of the entity data and the entity relation data, and the tag data to a target data source;
And according to the calling information, carrying out data processing on at least one of the entity data and the entity relation data imported in the target data source and the tag data, and storing the processed data in the target table.
In some embodiments, further comprising:
Detecting the communication path of the common data source of the acquired at least one of the entity data and the entity relation data to be processed and the tag data;
And if the label data is detected to be normal, calling the calling information, carrying out data processing on the label data to be processed, and storing the processed data in the target table.
In some embodiments, the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table, further includes:
Outputting the target table or outputting the target label in the target table at the configuration interface according to the binding relation determined between the target table and the target label.
The application also provides a device for processing data information, which comprises:
the creating unit is used for creating calling information which is called when the data processing is performed based on the tag information and the data processing conditions;
The generating unit is used for generating a target table for storing processed data according to the submitted parameters recorded in the calling information;
And the processing unit is used for calling the calling information, carrying out data processing treatment on the tag data to be processed, and storing the processed data into the target table.
The application also provides a computer storage medium for storing the network platform generated data and a program for processing the network platform generated data;
the program, when read and executed, performs the steps of the creation method of the data processing method as described above, or performs the steps of the processing method of the data information as described above.
The present application also provides an electronic device including:
a processor;
And a memory for storing a program for processing the network platform generated data, which when read and executed by the processor, performs the steps of the method for creating the data processing mode as described above, or performs the steps of the method for processing the data information as described above.
Compared with the prior art, the application has the following advantages:
according to the creation method of the data processing mode, the acquired tag information is output to the configuration interface, the data processing conditions are received based on the configuration interface of the existing tag information, the data processing conditions and the tag information are stored as call information in data processing, and data mining and data analysis can be conveniently and rapidly carried out based on the existing tag data by calling the call information, so that target data are generated. The target data may be at least one of target entity information and target entity relation information, and the target tag information may also be a physical table that generates only the target data.
In addition, the application also provides a data information processing method, which comprises the steps of creating calling information which is called when data processing is carried out based on tag information and data processing conditions, generating a target table for storing processed data according to parameters recorded in the submitted calling information, calling the calling information, carrying out data processing on the tag data to be processed and storing the processed data in the target table, thereby completing the mining and the generation of the target data based on the created data processing mode, improving the full utilization of the source data and the existing tag information, ensuring that the generated target data is from the same target data source, and adopting a mobile operation mode to output the existing tag information to a configuration interface, so that the generation process of the target data is simple and quick to operate.
Drawings
FIG. 1 is a flowchart of an embodiment of a method for creating a data processing manner provided by the present application;
FIG. 2 is a schematic diagram of a data processing method creation device according to an embodiment of the present application;
FIG. 3 is a flowchart of an embodiment of a method for processing data information according to the present application;
Fig. 4 is a schematic structural diagram of an embodiment of a data information processing apparatus according to the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application may be embodied in many other forms than those herein described, and those skilled in the art will readily appreciate that the present application may be similarly embodied without departing from the spirit or essential characteristics thereof, and therefore the present application is not limited to the specific embodiments disclosed below.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The terms "a," "an," "the" and "second" used herein and in the appended claims, for example, are not intended to be limited in number or order, but rather to distinguish one type of information from another.
Based on the description of the background art section, in order to better understand the creation method of the data processing mode provided by the application, the background of the technical scheme of the application is described through application scenes. For example, based on the scene of commodity transaction generation data, the attribute of the relationship between the shopping platform and the customers can be extracted as a label by analyzing the transaction data generated by the shopping platform, for example, the data of 10 customers before the purchasing power of the shopping platform or the data of the customers as females, the commodity attribute can be extracted as the label, for example, the commodity with the largest consumption number is displayed to the user in the form of the label, the user only needs to know the content displayed by the label, and the physical data information of the bottom layer does not need to be displayed. Therefore, the labels are attribute information of the entities or attribute information of the entity relationship, and the analysis requirements of different business scenes on the data can be met by establishing a label system for the data information. For example, analysis of commodity data information, analysis of medical data information, analysis of traffic data information, analysis of financial data information, etc., data of interest to the user can be directly obtained through the tag.
However, when the existing tag content cannot meet the user demand, the target tag can be generated by reprocessing the existing tag (original tag) content. The creation method of the data processing mode provided by the application aims to create a data information processing process capable of forming a target label by reprocessing data information on the basis of the existing label, and can generate the target label based on the existing label, entity and other data information by calling the data information processing process so as to meet the demands of users. Therefore, the method for creating the data processing mode is based on the creation of a processing process of the target label derived on the basis of the existing label.
The method for creating the data processing manner provided by the application will be described in detail below, and it should be noted that when the method for creating is described, the method for creating is described in synchronization with an example of an application scenario, so as to better understand the scheme, but the method for creating provided by the application is not limited to the application scenario in the example. Referring to fig. 1, fig. 1 is a flowchart of an embodiment of a method for creating a data processing manner, where the method includes:
step S101, acquiring tag information, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relationship.
The purpose of step S101 is to learn about existing tag information. In this embodiment, the tag information may be acquired by a data source, and thus includes:
selecting a data source to be processed from the provided selectable data sources;
And acquiring the tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source.
By data source is meant a database in which data is stored, possibly in different databases for different types of data or different data contents. Thus, it is desirable to select a data source to be processed from among the selectable data sources, where the data source to be processed is understood as a data source that requires reprocessing of the data.
The data source may be a database for storing data, so that the data source to be processed is selected first, so that the processing of the subsequent data provides a basis. In this embodiment, the data source may store data information of storage and calculation of batch structured data, and may implement importing and outputting of data, and so on.
Based on the foregoing, it will be appreciated that there may be data partitions in various dimensions for a selected data source, such as data partitions divided by a geographic dimension or data partitions divided by a temporal dimension, and so forth. Data partitioning is a physical database design technique that aims to reduce the total amount of data read and write in a particular SQL operation to reduce response time. The data partition does not generate a new data table, but uniformly distributes the data in the data table to different hard disks according to dimensions or needs, and the system or different server storage media improves the data retrieval efficiency and reduces the frequent IO pressure value of the database.
Thus, it further comprises:
Determining whether a data partition exists in the data source;
if yes, setting partition values of the data partitions;
the obtaining the tag information according to at least one of the entity information and the entity relationship information determined in the data source includes:
And acquiring the label information according to at least one of entity information, entity relation information and label custom function information which are included in the data partition corresponding to the partition value. After the tag information is acquired, secondary processing is required to be performed on the data related to the tag information so as to generate target data, wherein the target data can comprise at least one target data in a target tag, a target entity and a target entity relationship, and therefore, a target data source can be selected according to the tag information and the data processing condition.
And determining entity information, entity relation information and original tag information based on the selected data sources, and then selecting a target data source according to at least one of the entity information, the entity relation information and the tag information and the data processing conditions.
In this embodiment, the selection of the target data source may be:
the step S101-1 is to determine original entity information and original entity relationship information according to the data source, for example, based on commodity information, merchant information and customer information to be provided in a commodity transaction platform, namely, the entity can be merchant, customer and commodity, the relationship information between merchant and customer can be entity relationship information, the relationship information between commodity and merchant can be entity relationship information, and the relationship information between commodity and customer can be entity relationship information.
Step S101-2, acquiring the label information from a label list describing the entity information attribute or the entity relation information attribute according to the determined entity information and entity relation information;
The entity information and the relationship information between entity information in step S101-2 may be stored in the database in the form of a data table. Based on the example in step S102-1, the tag information may be based on at least one of the properties of the merchandise, the merchant properties, the customer properties, the relationship between the merchant and the customer, and the properties between the merchandise and the customer in the data table. Tag information generated based on the entity information and the entity relationship information is acquired by a tag column (which may also be referred to as an attribute column) in the data table.
Therefore, it can be understood that the data processing manner in the method for creating a data processing manner provided by the present application may be created based on a visual operation interface, and the target data source is selected through the operation behavior of the user, which is described in the form of a drag operation in this embodiment, and other operation behaviors such as a voice operation, a click operation, and the like are not excluded. Of course, the original entity relation information and the original label information can be edited by adopting a label sentence editing mode, an editing result is obtained, and a target data source is selected according to the editing result. The tag statement can be TQL: tag query language, a tag query statement, a statement composed of entity relations and tags, the writing form is similar to SQL, the entity relations are similar to tables in SQL, and the tags are similar to fields in the SQL tables. That is, the selected target data source may be implemented based on a visual interface or may be implemented by a tag query statement.
And step S102-3, selecting a target data source according to at least one of entity information and entity relation information included in the tag information and the data processing condition.
The steps S101-1 to S101-3 are mainly used for describing the process of selecting the target data source, because the target data source needs to be reprocessed to form the target tag, and the target data source needs to be selected according to at least one of the entity information, the entity relationship information and the data processing condition included in the formed tag information on the selected data source, and the processing of the target tag (new tag) is performed on the target data source.
In this embodiment, the obtaining of the tag information may further obtain a tag custom function from a custom tag processing function list describing the entity information attribute or the entity relationship information attribute according to the determined entity information and entity relationship information, that is, obtain a UDF function from a UDF (User Defined Function) tag custom function list, thereby obtaining the tag information.
Accordingly, the target data source may also select the target data source according to at least one of entity information, entity relationship information, UDF function information, and data processing conditions included in the tag information.
Since the selected target data source may not be the desired target data source, the selected target data source is required to be tested, and therefore, the selecting the target data source according to the tag information and the data processing condition includes:
pre-selecting the label information which is expected to be acquired;
Detecting the pre-selected label information according to the data processing conditions;
And if the detection result is normal, selecting the pre-selected label information as a target data source.
If the detection result is wrong, the label information is acquired again.
It will be understood, of course, that editing the original entity relationship information and the original label information by using a label sentence editing mode may further include detecting a written TQL sentence, and if the detection result is correct, it is indicated that there is no problem in writing, and the target data source may be selected according to the TQL sentence. If the detection result is wrong, the TQL statement needs to be written again to ensure the accuracy of the subsequent selected target data source.
And step S102, outputting the label information to a configuration interface.
The purpose of S102 is to display the obtained tag information on the configuration interface for the user to view. The configuration interface can be understood as outputting part of parameter information of the processed data on a visual operation interface before obtaining a processing result in the data processing process, and outputting the obtained label information under the interface. The tag information output may be performed in a configuration interface by a mobile operation on the tag information, such as a voice operation, a click operation, etc. In this embodiment, the tag information may be dragged to the configuration interface in the form of a drag operation.
It will be appreciated that the target data source can also be determined by the above-described operation of dragging the tag information.
Step S103, receiving data processing conditions based on the configuration interface of the label information.
The data processing condition in step S103 may be understood as configuration information for generating the target tag based on the target data source configuration. Thus, the step S103 may include:
Step S103-1, according to the requirement of a target label to be generated, determining screening conditions based on the label information on the configuration interface;
The screening conditions in step S103-1 may be selected by the screening conditions already present, or may be set when it is necessary to generate a derivative target tag. The setting is to configure a screening condition, for example, the target data source is transaction data of the first quarter, the derived target label requirement to be generated may be based on the label of the customer as the female in the transaction data of the first quarter in the target data source, and the configured screening condition may be that the customer is the female customer.
And step S103-2, determining the screening condition as the data processing condition.
Taking the example in step S103-1, the screening condition may be a condition that the consumer meets the female requirement, and the condition may be determined as a data processing condition, and in the subsequent processing of the data, the data that needs to meet the screening condition and describe attribute information in the data that meets the screening condition is the target label.
After the target data source is selected in step S101, step S103 may include:
Determining whether to configure the target entity and the target tag based on the selected target data source;
if yes, configuring a target entity and a target label for the target data source;
And storing the configured target data source and the data processing condition as call information in data processing.
And step S104, saving the data processing conditions and the tag information as call information in data processing.
The purpose of step S104 is to determine, based on the tag information acquired in steps S101 to S103 and the received data processing conditions, a data processing manner at the time of generating the target tag, that is, when the target tag needs to be generated for the original tag, the generation of the target tag may be achieved by calling the procedures described in steps S101 to S103.
Therefore, the above steps S101 to S104 are only for creating a data processing manner of the target label to be generated, which may also be referred to as a processing or generating process, and when the target label is generated, the user may be guided to select a required original label based on the target data source through operations such as dragging, and after the dragging operation, the generation of the target label may be completed based on the original label and the data processing conditions, so that the generating process of the target label is easy to operate, and the target label which is more significant and meets the user requirement may be generated based on the original label.
In order to facilitate the operation of the user, the data processing mode can be based on the visual interface to complete corresponding operation, and in the data processing process, the calling information is called based on the visual interface, so that the data can be quickly mined and analyzed.
It should be noted that, in the steps S101 to S104, the creation of the data processing manner may be completed based on the configuration interface (visual interface) through the requirements of the original tag and the generation of the target tag, so that the user can quickly perform the operation on the original tag to generate the target data. To provide different operational requirements for different user groups, further comprising:
Converting said data processing conditions and said tag information into tag query statements (TQL: tag query language);
In the implementation process of step S104, the data processing condition and the tag query statement corresponding to the tag information may be stored as call information during data processing.
The TQL statement may be understood as a statement composed of an entity relationship and a label, the written form is similar to SQL (database query statement), the entity relationship may be regarded as a table in SQL, and the label may be regarded as a field in the table in SQL.
The foregoing is a detailed description of an embodiment of a method for creating a data processing manner, which corresponds to the foregoing embodiment of the method for creating a data processing manner, and the application also discloses an embodiment of a device for creating a data processing manner, please refer to fig. 2, and since the device embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points refer to a part of the description of the method embodiment. The device embodiments described below are merely illustrative.
As shown in fig. 2, fig. 2 is a schematic structural diagram of an embodiment of a creating apparatus for a data processing manner according to the present application, where the apparatus includes:
An acquiring unit 201, configured to acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of a relationship of entities;
an output unit 202, configured to output the tag information to a configuration interface;
A receiving unit 203, configured to receive a data processing condition based on a configuration interface of the tag information;
A saving unit 204, configured to save the data processing condition and the tag information as call information during data processing.
The acquisition unit 201 includes:
a selecting subunit, configured to select a data source to be processed from the provided selectable data sources;
and the obtaining subunit is used for obtaining the label information according to at least one of the entity information, the entity relation information and the label custom function information included in the data source.
In an embodiment of the present application, the creating device for a data processing manner further includes:
A partition determining unit, configured to determine whether a data partition exists in the data source;
and the partition value setting unit is used for setting the partition value of the data partition if the data partition is determined in the partition determination unit.
The obtaining subunit is specifically configured to obtain the tag information according to at least one of entity information, entity relationship information, and tag custom function information included in the data partition corresponding to the partition value.
The output unit 202 includes:
A mobile subunit, configured to perform a mobile operation on the tag information;
And the output subunit is used for outputting the label information to a configuration interface according to the moving operation of the label information in the moving subunit.
The specific implementation procedure of the receiving unit 203 may include:
A condition determining subunit, configured to determine, on the configuration interface, a screening condition based on the label information already existing according to a requirement of a target label to be generated;
and the receiving subunit is used for determining the screening condition as the received data processing condition.
The condition determining subunit is specifically configured to select or set a screening condition based on the existing tag information according to a requirement of the target tag to be generated.
The creation apparatus further includes:
and the target source selecting unit is used for selecting a target data source according to the tag information and the data processing conditions.
The target source determining unit includes:
a preselection subunit, configured to preselect the tag information that is desired to be acquired;
a detection subunit, configured to detect the pre-selected tag information according to the data processing condition;
and the selecting subunit is used for selecting the pre-selected label information as a target data source if the detection result is normal.
If the detection result is an error, it may return to the acquisition unit 201 to re-acquire the tag data.
The saving unit 204 may include:
A configuration determination subunit configured to determine, based on the selected target data source, whether to configure the target entity and the target tag;
A configuration subunit, configured to configure a target entity and a target tag for the target data source when the determination result of the configuration determination subunit is yes;
and the storage subunit is used for storing the configured target data source and the data processing condition as call information during data processing.
Based on the above, the method further comprises:
The conversion unit is used for converting the data processing conditions and the tag information into tag inquiry sentences;
The storage unit is specifically configured to store the data processing condition and the tag query statement corresponding to the tag information as call information during data processing.
The above description of an embodiment of a data processing method creation device provided in the present application may refer to the description of step S101 to step S104 in the embodiment of the data processing method creation device.
The description of the embodiment of the creation method based on the data processing manner and the embodiment of the creation device based on the data processing manner is illustrative of a manner of providing a processing generation for generating the target tag, and can also be understood as providing a frame or mode of processing generation, in which the landing of the data information of the target tag can be performed, that is, the target tag to be generated is output by calling the data processing manner for reference of a user. Therefore, the present application also provides a method for processing data information, as shown in fig. 3, fig. 3 is a flowchart of an embodiment of a method for processing data information, where the method includes:
step S301, creating calling information which is called when data processing is performed based on the tag information and the data processing conditions;
The purpose of step S301 is to create a data processing manner of the target label or target relationship derived (or generated) based on the original entity information, the original entity relationship information, and the original label information. The specific data processing manner has been described in the above steps S101 to S104, and the detailed description is not repeated here, and the specific implementation process of step S301 may refer to the above steps S101 to S104.
The step S301 may also be understood as providing a preparation condition for generating the target tag or the target relationship data information.
And step S302, generating a target table for storing processed data according to the submitted parameters recorded in the call information.
Step S302 is to generate a target table storing processed data based on the created data processing manner, and detect a data source and a target data source in the created data processing manner before generating the target table storing processed data, so as to detect whether the data communication path is normal. Of course detection of the UDF function may also be included.
If the detection is normal, a target table storing processed data is generated according to parameters related to the created data processing mode, such as a selected target data source, data processing conditions and the like.
In order to meet different requirements of different users, for example, a technical development user can obtain target data information of a technical layer through a data processing mode, namely a physical table of the target data information, and a business analysis user can obtain at least one of target realization relation information and target label information which are visually displayed through a data processing process.
Thus, after generating the target table, it may further include:
Determining whether the target table is bound with the processed data;
If yes, binding the target table with at least one of target entity information and target entity relation information related to the target data source, and binding a table list of the target table with a target label related to the target data source.
And providing at least one of corresponding target entity information, target entity relation information and target label information for the user based on the binding result.
And when the binding result of the target table and the processed data is determined to be no, providing a corresponding target data physical table for the user.
Step S303, calling the calling information, carrying out data processing on the tag data to be processed, and storing the processed data in the target table.
The specific implementation process of step S303 may include:
converting the data processing conditions in the data processing mode into a Structured Query Language (SQL);
And processing data in the target data source based on the structured query language, and storing the processed data in the target table.
The purpose of step S303 is to extract and store the data meeting the data processing conditions in the target data source into the target table based on the execution of step S301 and step S302, and output the data in a visual form for the user to refer to and analyze.
It may be understood that, in the step S303, the communication detection may be performed on the target data source, and when a UDF function exists, the detection may further include detecting the UDF function, or detecting the data processing manner created in the step S301, where the specific step of detecting occurs is not specifically limited, so that it is only required to ensure that the data processing manner, the related data information and the path are normal.
In the data processing mode in step S303 the calling process can also comprise the following steps:
When data processing is carried out on the data in the target data source according to the data processing conditions in the data processing mode, whether the data processing is carried out according to the configured processing period or not;
if yes, data processing is carried out on the data in the target data source according to the configured processing period.
If not, data processing is carried out according to the configured one-time processing mode, and the processing process can be carried out by generating a processing SQL sentence.
And finally, the processed data can be stored in a target table and output and displayed in a visual display form, for example, at least one of the entity relation information, the entity information and the label information derived from the target table is displayed.
The foregoing is a detailed description of an embodiment of a data information processing method according to the present application, corresponding to the foregoing embodiment of a data information processing method, and the present application further discloses an embodiment of a data information processing apparatus, please refer to fig. 4, and since the apparatus embodiment is substantially similar to the method embodiment, the description is relatively simple, and the relevant points refer to part of the description of the method embodiment. The device embodiments described below are merely illustrative.
As shown in fig. 4, fig. 4 is a schematic structural diagram of an embodiment of a data information processing apparatus according to the present application. The processing device comprises:
A creating unit 401 for creating call information that is called when data processing is performed based on the tag information and the data processing conditions;
The creation unit 401 may refer to the description of the creation device for the data processing procedure, which is not repeated herein, and in the embodiment of the data information processing device provided by the present application, the creation of the data processing manner is the same as the creation method and the creation device of the data processing manner.
A generating unit 402, configured to generate a target table storing processed data according to parameters recorded in the submitted call information;
In this embodiment, it may further include:
a communication path determining unit configured to determine whether a communication path of the data source and a communication path of the target data source are normal;
The generating unit may be specifically configured to generate, when the determination result in the communication path determining unit is normal, a target table storing the processed data according to the parameters recorded in the data processing manner.
In this embodiment, the method may further include:
a binding determining unit, configured to determine whether the target table is bound with the processed data;
And the binding unit is used for binding the target table with at least one of target entity information and target entity relation information related to the target data source according to the determined result in the binding determination unit, and binding the list of the target table with target labels related to the target data source.
And the processing unit 403 is configured to call the call information, perform data processing on the tag data to be processed, and store the processed data in the target table.
The processing unit 403 includes:
The conversion subunit is used for converting the data processing conditions in the data processing mode into a structured query language;
And the processing subunit is used for processing the data in the target data source based on the structured query language, and storing the processed data in the target table.
In this embodiment, the method further includes:
A processing period determining unit, configured to determine whether to process data according to a configured processing period when data processing is performed on data in a target data source according to a data processing condition in the data processing manner;
And the processing unit is specifically used for processing the data in the target data source according to the configured processing period when the processing period determining unit determines that the result is yes.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.
2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
While the application has been described in terms of preferred embodiments, it is not intended to be limiting, but rather, it will be apparent to those skilled in the art that various changes and modifications can be made herein without departing from the spirit and scope of the application as defined by the appended claims.
Claims (28)
1. A method of creating a data processing means, comprising:
acquiring tag information, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relationship;
Outputting the label information to a configuration interface;
receiving data processing conditions based on a configuration interface of the existing tag information;
storing the data processing conditions and the tag information as call information in data processing;
the acquiring tag information includes:
selecting a data source to be processed from the provided selectable data sources;
And acquiring the tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source.
2. The method for creating a data processing pattern according to claim 1, further comprising:
Determining whether a data partition exists in the data source;
if yes, setting partition values of the data partitions;
the obtaining the tag information according to at least one of the entity information and the entity relationship information determined in the data source includes:
and acquiring the label information according to at least one of entity information, entity relation information and label custom function information which are included in the data partition corresponding to the partition value.
3. The method for creating a data processing pattern according to claim 1, further comprising:
and selecting a target data source according to the label information and the data processing conditions.
4. A method of creating a data processing system according to claim 3, wherein said selecting a target data source according to said tag information and said data processing condition comprises:
pre-selecting the label information which is expected to be acquired;
Detecting the pre-selected label information according to the data processing conditions;
And if the detection result is normal, selecting the pre-selected label information as a target data source.
5. The method for creating a data processing system according to claim 4, wherein the step of storing the data processing condition and the tag information as call information at the time of data processing includes:
Determining whether to configure the target entity and the target tag based on the selected target data source;
if yes, configuring a target entity and a target label for the target data source;
And storing the configured target data source and the data processing condition as call information in data processing.
6. The method for creating a data processing manner according to claim 1, wherein the receiving a data processing condition based on the configuration interface of the tag information includes:
According to the requirement of the target label to be generated, determining screening conditions based on the label information on the configuration interface;
and determining the screening condition as the received data processing condition.
7. The method for creating a data processing manner according to claim 6, wherein determining, on the configuration interface, a filtering condition based on the existing tag information according to a requirement of a target tag to be generated, includes:
and selecting or setting screening conditions based on the existing label information according to the requirements of the target label to be generated.
8. The method for creating a data processing manner according to claim 1, wherein outputting the tag information to a configuration interface includes:
And outputting the label information to a configuration interface through a movement operation of the label information, wherein the movement operation is a drag operation.
9. The method for creating a data processing pattern according to claim 1, further comprising:
Converting the data processing conditions and the tag information into tag query sentences;
The step of storing the data processing condition and the tag information as call information in data processing includes:
And saving the data processing conditions and the label inquiry statement corresponding to the label information as call information in data processing.
10. A data processing method creation apparatus, comprising:
The information processing device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring tag information, and the tag information is used for describing attribute information of an entity or attribute information of an entity relationship;
The output unit is used for outputting the label information to a configuration interface;
the receiving unit is used for receiving data processing conditions based on the configuration interface of the existing tag information;
A saving unit for saving the data processing condition and the tag information as call information in data processing;
The acquiring unit is specifically configured to select a data source to be processed from the provided selectable data sources when acquiring the tag information, and acquire the tag information according to at least one of entity information, entity relationship information and tag custom function information included in the data source.
11. A method of processing data information, comprising:
Creating calling information which is called when data processing is performed based on the tag information and the data processing conditions;
generating a target table for storing the processed data according to the submitted parameters recorded in the calling information;
calling the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table;
the creating call information when the creating performs data processing based on the tag information and the data processing condition includes:
selecting a data source to be processed from the provided selectable data sources;
Acquiring tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relation;
Outputting the label information to a configuration interface;
receiving data processing conditions based on a configuration interface of the existing tag information;
and storing the data processing conditions and the tag information as call information in data processing.
12. The method for processing data information according to claim 11, further comprising:
Determining whether a data partition exists in the data source;
if yes, setting partition values of the data partitions;
the obtaining the tag information according to at least one of the entity information and the entity relationship information determined in the data source includes:
and acquiring the label information according to at least one of entity information, entity relation information and label custom function information which are included in the data partition corresponding to the partition value.
13. The method for processing data information according to claim 11, further comprising:
and selecting a target data source according to the label information and the data processing conditions.
14. The method for processing data information according to claim 13, wherein selecting a target data source based on the tag information and the data processing condition comprises:
pre-selecting the label information which is expected to be acquired;
Detecting the pre-selected label information according to the data processing conditions;
And if the detection result is normal, selecting the pre-selected label information as a target data source.
15. The method according to claim 14, wherein the storing the data processing condition and the tag information as call information at the time of data processing, comprises:
Determining whether to configure the target entity and the target tag based on the selected target data source;
if yes, configuring a target entity and a target label for the target data source;
And storing the configured target data source and the data processing condition as call information in data processing.
16. The method according to claim 11, wherein the receiving data processing conditions based on the configuration interface of the tag information includes:
According to the requirement of the target label to be generated, determining screening conditions based on the label information on the configuration interface;
and determining the screening condition as the data processing condition.
17. The method for processing data information according to claim 16, wherein determining, on the configuration interface, a filtering condition based on existing tag information according to a requirement of a target tag to be generated, comprises:
and selecting or setting screening conditions based on the existing label information according to the requirements of the target label to be generated.
18. The method for processing data information according to claim 11, wherein the outputting the tag information to a configuration interface includes:
And outputting the label information to a configuration interface through a movement operation of the label information, wherein the movement operation is a drag operation.
19. The method for processing data information according to claim 11, further comprising:
Converting the data processing conditions and the tag information into tag query sentences;
The step of storing the data processing condition and the tag information as call information in data processing includes:
And saving the data processing conditions and the label inquiry statement corresponding to the label information as call information in data processing.
20. The method for processing data information according to claim 11, wherein the generating a target table storing processed data according to the parameters recorded in the submitted call information, further comprises:
Determining whether the communication path of the data source and the communication path of the target data source in the calling information are normal or not;
And if the data is normal, generating a target table for storing the processed data according to the parameters recorded in the calling information.
21. The method for processing data information according to claim 11, wherein the generating a target table storing processed data according to the parameters recorded in the submitted call information, further comprises:
Determining whether the target table is bound with the processed data;
If yes, binding the target table with at least one of the processed target entity information and the processed target entity relation information, and binding the list of the target table with the processed target label.
22. The method for processing data information according to claim 11, wherein the calling information, performing data processing on tag data to be processed, and storing the processed data in the target table, further comprises:
When the label data to be processed is processed, determining whether a data processing period is configured;
If yes, carrying out data processing on the label data to be processed according to the configured data processing period, and storing the processed data into the target table.
23. The method for processing data information according to claim 11, wherein the calling information, performing data processing on tag data to be processed, and storing the processed data in the target table, further comprises:
acquiring at least one of entity data and entity relation data to be processed, and tag data;
importing at least one of the entity data and the entity relation data, and the tag data to a target data source;
And according to the calling information, carrying out data processing on at least one of the entity data and the entity relation data imported in the target data source and the tag data, and storing the processed data in the target table.
24. The method for processing data information according to claim 23, further comprising:
Detecting the communication path of the common data source of the acquired at least one of the entity data and the entity relation data to be processed and the tag data;
And if the label data is detected to be normal, calling the calling information, carrying out data processing on the label data to be processed, and storing the processed data in the target table.
25. The method for processing data information according to claim 11, wherein the calling information, performing data processing on tag data to be processed, and storing the processed data in the target table, further comprises:
Outputting the target table or outputting the target label in the target table at the configuration interface according to the binding relation determined between the target table and the target label.
26. A data information processing apparatus, comprising:
the creating unit is used for creating calling information which is called when the data processing is performed based on the tag information and the data processing conditions;
The generating unit is used for generating a target table for storing processed data according to the submitted parameters recorded in the calling information;
the processing unit is used for calling the calling information, carrying out data processing on the tag data to be processed, and storing the processed data into the target table;
The creation unit is specifically configured to select a data source to be processed from the provided selectable data sources;
Acquiring tag information according to at least one of entity information, entity relation information and tag custom function information included in the data source, wherein the tag information is used for describing attribute information of an entity or attribute information of an entity relation;
Outputting the label information to a configuration interface;
receiving data processing conditions based on a configuration interface of the existing tag information;
and storing the data processing conditions and the tag information as call information in data processing.
27. A computer storage medium for storing network platform generated data and a program for processing the network platform generated data;
The program, when read and executed, performs the steps of the method for creating a data processing means according to any one of claims 1 to 9, or the steps of the method for processing data information according to any one of claims 11 to 25.
28. An electronic device, comprising:
a processor;
A memory for storing a program for processing data generated by a network platform, which program, when read by the processor, performs the steps of the method for creating a data processing means according to any one of claims 1 to 9, or the steps of the method for processing data information according to any one of claims 11 to 25.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910446967.5A CN112000699B (en) | 2019-05-27 | 2019-05-27 | Data processing method creation method and device, data information processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910446967.5A CN112000699B (en) | 2019-05-27 | 2019-05-27 | Data processing method creation method and device, data information processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112000699A CN112000699A (en) | 2020-11-27 |
| CN112000699B true CN112000699B (en) | 2024-12-13 |
Family
ID=73461847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910446967.5A Active CN112000699B (en) | 2019-05-27 | 2019-05-27 | Data processing method creation method and device, data information processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000699B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111994A (en) * | 2014-07-04 | 2014-10-22 | 北京思特奇信息技术股份有限公司 | Label data screening method and device based on mixed data source |
| CN105630475A (en) * | 2014-11-06 | 2016-06-01 | 阿里巴巴集团控股有限公司 | Data label organization system and organization method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7278103B1 (en) * | 2000-06-28 | 2007-10-02 | Microsoft Corporation | User interface to display and manage an entity and associated resources |
| CN109325213B (en) * | 2018-09-30 | 2023-11-28 | 北京字节跳动网络技术有限公司 | Method and device for labeling data |
-
2019
- 2019-05-27 CN CN201910446967.5A patent/CN112000699B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111994A (en) * | 2014-07-04 | 2014-10-22 | 北京思特奇信息技术股份有限公司 | Label data screening method and device based on mixed data source |
| CN105630475A (en) * | 2014-11-06 | 2016-06-01 | 阿里巴巴集团控股有限公司 | Data label organization system and organization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112000699A (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110442641B (en) | Link topology graph display method and device, storage medium and equipment | |
| JP6964706B2 (en) | User image processing method, user image processing device, server and storage medium | |
| US9336184B2 (en) | Representation of an interactive document as a graph of entities | |
| CN111125266B (en) | Data processing method, device, equipment and storage medium | |
| CN102915237B (en) | The method and system of rewrite data quality rule is required according to user application | |
| CN111126019B (en) | Report generation method and device based on mode customization and electronic equipment | |
| CN106610774B (en) | Webpage table editing method and device | |
| CN110674228A (en) | Data warehouse model construction and data query method, device and equipment | |
| WO2017189933A1 (en) | System, method, and apparatus for operating a unified document surface workspace | |
| CN106557486A (en) | A kind of storage method and device of data | |
| US9015608B2 (en) | Regenerating a user interface area | |
| CN109426604A (en) | The monitoring method and equipment of code development | |
| CN106933887A (en) | A kind of data visualization method and device | |
| US10672016B1 (en) | Pathing and attribution in marketing analytics | |
| US20190179638A1 (en) | Automatic creation of macro-services | |
| CN106528088A (en) | Method and device for addling control in online form | |
| CN113518187B (en) | Video editing method and device | |
| CN111143546A (en) | Method and device for obtaining recommendation language and electronic equipment | |
| CN116955386A (en) | Data processing method and device and data processing task generation method and device | |
| CN112000699B (en) | Data processing method creation method and device, data information processing method and device | |
| EP3454207B1 (en) | Dynamic preview generation in a product lifecycle management environment | |
| US20150170067A1 (en) | Determining analysis recommendations based on data analysis context | |
| KR100907709B1 (en) | Information extraction apparatus and method using block grouping | |
| US20150248477A1 (en) | Business content hierarchy | |
| CN114297330B (en) | Data map generation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |