CN114817149B - Metadata Management System - Google Patents
Metadata Management System Download PDFInfo
- Publication number
- CN114817149B CN114817149B CN202210287993.XA CN202210287993A CN114817149B CN 114817149 B CN114817149 B CN 114817149B CN 202210287993 A CN202210287993 A CN 202210287993A CN 114817149 B CN114817149 B CN 114817149B
- Authority
- CN
- China
- Prior art keywords
- data
- module
- feedback
- list
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明提供了一种元数据管理系统，包括：数据概况模块用于数据资产库中的数据进行校准、分类，并生成每个类别对应的数据列表；向用户展示资产库信息；当接收到搜索引擎模块发送的搜索信息时查找各个目标数据，构建各个所述目标数据对应的血缘关系；数据指标模块用于对每个数据列表中的各个数据进行指标划分；数据反馈模块用于接收用户发送的数据修正消息，并将数据修正消息发送至对应的业务端；获得业务端的反馈结果后，由数据概况模块或数据指标模块对反馈结果对应的数据进行修正；搜索引擎模块用于接收用户发送的搜索信息实现对数据的搜索。应用本发明提供的系统，可以实现对元数据的管理，并通过对元数据的管理实现数据的溯源。
The present invention provides a metadata management system, including: a data overview module for calibrating and classifying data in a data asset library, and generating a data list corresponding to each category; displaying asset library information to users; searching for each target data when receiving search information sent by a search engine module, and constructing the blood relationship corresponding to each target data; a data indicator module for dividing each data in each data list into indicators; a data feedback module for receiving a data correction message sent by a user, and sending the data correction message to a corresponding business end; after obtaining the feedback result of the business end, the data corresponding to the feedback result is corrected by the data overview module or the data indicator module; and the search engine module is used to receive the search information sent by the user to realize the search of data. The system provided by the present invention can realize the management of metadata, and realize the traceability of data through the management of metadata.
Description
Technical Field
The present invention relates to the field of metadata management technologies, and in particular, to a metadata management system.
Background
With the continuous increase of business product lines of companies, the bottom data record is more and more complex, and each business line has own data self-forming system and chimney type business development form, so that data island is easy to form. The construction of the company-level data warehouse is the optimal choice in the situation, and the data of the summarizing layer and the display layer can be used as the basis of BI analysis and can be directly packaged into a data interface. More and more business data and data warehouse data, how to more rapidly search data in the process of data analysis and business enabling, and select data according to business caliber of different requirements become important points of data management. Metadata is the data that is the core of recording such information as data. Metadata management and maintenance of the full life cycle of the data are the root of data management, and lay a foundation for ensuring stable and efficient energy-giving of company data.
In the prior art, metadata is easy to be lost in the process of managing data, full life cycle tracing of the data cannot be realized, and the difficulty of searching the data is increased in the process of applying the data in the follow-up process.
Disclosure of Invention
In view of the above, the present invention provides a metadata management system, by which metadata management can be implemented, and by which metadata management can be implemented to trace data.
A metadata management system, comprising:
The system comprises a data profile module, a data index module, a data feedback module and a search engine module;
the data profiling module is used for scanning the metadata information of the data warehouse in real time and calibrating the data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when a user login is detected, asset library information of the data asset library is displayed to the user, wherein the asset library information comprises the data lists; when search information sent by the search engine module is received, searching each target data list corresponding to the search information, and constructing a blood-edge relation corresponding to each target data according to each target data associated with each target data list;
The data index module is used for carrying out index division on each data in each data list based on a preset second classification rule to obtain a plurality of index class data indexes under each data list;
The data feedback module is used for receiving data correction information sent by a user, determining a feedback type corresponding to the data correction information, wherein the feedback type is a data type or an index type; the data correction message is sent to a service end corresponding to the feedback type; when receiving a feedback result corresponding to the data correction message fed back by the service end, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module so that the data profile module queries data corresponding to the search information from the data asset library.
In the above metadata management system, optionally, the data profile module is configured to determine, based on the metadata information, whether new data exists in the data warehouse, and whether updated data exists in the data warehouse; when the data warehouse has newly added data, adding the newly added data to a preset newly added list; when the data warehouse has updated data, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; and updating the data in the data asset library based on the updated data in the update list.
The metadata management system described above, optionally, the data profile module is configured to determine a user type of the user; when the user type of the user is a common user, displaying each data list of the data asset library and data details corresponding to each data list to the user; when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
In the above metadata management system, optionally, the data profile module is configured to obtain a table number included in the search information; determining a target data list matched with the table number in the data asset library; and searching each target data associated with the target data list based on the table number.
In the metadata management system, optionally, the data profile module is configured to associate each data matched with the id based on a blood-edge relationship between each target data associated with the target data list, obtain associated data, and send the associated data to the user.
The metadata management system, optionally, the data profile module is further configured to: and when receiving a discarding instruction corresponding to any data list in the data asset library, canceling displaying the data list corresponding to the discarding instruction.
In the metadata management system, optionally, the data profile module is further configured to restore and display a data list corresponding to the restore instruction when a restore instruction corresponding to the discarded data list is received.
In the metadata management system, optionally, the data feedback module is configured to obtain a feedback identifier included in the data correction message; when the feedback identifier is a first feedback identifier, determining a feedback type corresponding to the data correction message as a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
In the metadata management system, optionally, the data feedback module is configured to record the feedback result to a preset first repository when the feedback type corresponding to the data correction message is a data type, and call the data profile module to classify the feedback result according to the first classification rule; when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
In the metadata management system, optionally, the search engine module is configured to obtain a repository name and a data category included in the search information, and query data corresponding to the search information from the data asset library based on the repository name and the data category.
Compared with the prior art, the invention has the following advantages:
The present invention provides a metadata management system, comprising: a data profiling module for scanning metadata information of the data warehouse in real time and calibrating data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when a user login is detected, asset library information of the data asset library is displayed to the user, wherein the asset library information comprises the data lists; when search information sent by the search engine module is received, searching each target data list corresponding to the search information, and constructing a blood-edge relation corresponding to each target data according to each target data associated with each target data list; the data index module is used for carrying out index division on each data in each data list based on a preset second classification rule to obtain a plurality of index class data indexes under each data list; the data feedback module is used for receiving the data correction message sent by the user, and determining a feedback type corresponding to the data correction message, wherein the feedback type is a data type or an index type; the data correction message is sent to a service end corresponding to the feedback type; when receiving a feedback result corresponding to the data correction message fed back by the service end, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type; and the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module so that the data profile module queries data corresponding to the search information from the data asset library. By applying the system provided by the invention, the management of metadata can be realized, and the traceability of the data can be realized through the management of the metadata.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a system configuration diagram of a metadata management system according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an application of a data profile module in a metadata management system according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating another application example of a data profile module in a metadata management system according to an embodiment of the present invention;
Fig. 4 is an application example diagram of a data feedback module in a metadata management system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the present disclosure, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions, and the terms "comprise," "include," or any other variation thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor devices, distributed computing environments that include any of the above devices or devices, and the like.
The embodiment of the invention provides a metadata management system, wherein a system structure diagram of the metadata management system is shown in fig. 1, and specifically comprises:
A data profile module 100, a data metrics module 200, a data feedback module 300, and a search engine module 400;
The data profiling module 100 is used for scanning metadata information of the data warehouse in real time and calibrating data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when a user login is detected, asset library information of the data asset library is displayed to the user, wherein the asset library information comprises the data lists; when receiving the search information sent by the search engine module 400, searching each target data list corresponding to the search information, and constructing each blood-edge relation corresponding to each target data according to each target data associated with each target data list;
the data index module 200 is configured to perform index division on each data in each data list based on a preset second classification rule, so as to obtain a plurality of index class data indexes under each data list;
The data feedback module 300 is configured to receive a data correction message sent by a user, determine a feedback category corresponding to the data correction message, where the feedback category is a data type or an index type; the data correction message is sent to a service end corresponding to the feedback type; when receiving a feedback result corresponding to the data correction message fed back by the service end, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module 400 is configured to receive search information sent by a user, and send the search information to the data profile module, so that the data profile module queries data corresponding to the search information from the data asset library.
In the metadata management system provided by the embodiment of the invention, the data profile module mainly displays the information of the data list and displays the information differently according to different user types of logged-in users. The data asset library contains a plurality of data listings for storing individual data from the data warehouse. And (3) the information, classification and label of the data table are subjected to persistent storage record through scanning the metadata information design database table of the data warehouse. Compared with the prior manual maintenance mode, the method can design more auxiliary description fields without increasing the cost, and the recorded data is more complete and reliable. And generating a path of the blood edge relation data, and displaying a complete relation path through the front end. Compared with the prior single-stage upstream-downstream relation display, the whole dependence path can be seen more intuitively from the whole world, and the layer-by-layer self-query is not needed. The user can directly access through the daily office network environment without logging in the off-line production environment, and can access and know the data list structure and metadata information. Wherein, the data blood relationship is the basic concept of metadata management, and refers to a link generated by data. The method is used for finding the relation between related data in the process of tracing the data. For example, the field abc in the warehouse summary layer table B is the abc extracted from the patch source layer a, and then the data blood relationship exists between the table B and the table a. The first classification rule performs classification processing according to tables of different layers in the data warehouse architecture, the total amount of each layer of table and the distinction between major classes and minor classes of each table can be clearly displayed, and the detail data of each table can be checked by clicking each table. Each layer is a schema, and each schema corresponds to a plurality of tables.
The data index management module divides the data into finer granularity according to the rules, and a manager can classify the data according to the rules through the management module, so that the query efficiency of a user is improved. The second classification rule is to classify according to the service, so that the manager can find the index under the corresponding classification more quickly, and the index under each classification can be added, deleted, modified and inquired, thereby better realizing the management of the index by the manager.
The data feedback module is a module for providing data correction for users, and the users can select and submit correction on data, index or other aspects of the data in the whole system according to own requirements, so that the users can submit own suggestions, problems and other feedback conveniently. That is, the correction of the data is performed according to the feedback type.
The search engine module is used for facilitating the setting of a user aiming at the table query, and the module supports independent query and combined query, and further supports fuzzy search in the module. The combined query comprises a reservoir name, a major class and a minor class and the combined query of the search data, so that the retrieval efficiency is improved greatly.
By applying the method provided by the embodiment of the invention, the management of metadata can be realized, the content is supplemented, and the tracing of the data is realized through the management of the metadata.
In particular, for the data profiling module, data calibration, list generation and presentation, association and search of data blood edges, and the like can be realized.
In the embodiment of the invention, the data profile module is used for carrying out the processes of data calibration and list generation, wherein the processes are as follows: judging whether the data warehouse has newly added data or not based on the metadata information, and judging whether the data warehouse has updated data or not; when the data warehouse has newly added data, adding the newly added data to a preset newly added list; when the data warehouse has updated data, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; and updating the data in the data asset library based on the updated data in the update list.
Specifically, referring to fig. 2, fig. 2 is a process of performing data calibration in a data asset library, where a trigger end in fig. 2 is a data profile module, and a specific implementation process corresponding to fig. 2 is as follows:
(1) Data source_column_set of the quasi-production number bin and service is acquired,
(2) The undeleted data target column set in the data asset is obtained,
(3) Obtaining data to be added according to the data acquired in (1) and (2), wherein the data to be added is equal to all data in the current data warehouse, filtering out the data existing in the data asset,
(4) Obtaining data to be updated according to the data obtained in the steps (1) and (2), wherein the data to be updated is obtained by comparing the existing data in the current data asset with the data in the current data bin, storing the data with different contents,
(5) Judging whether the data to be newly added is empty, if not, integrating the data into a list to be inserted, finishing the integration, inserting the data into the data asset,
(6) And judging whether the data to be updated is empty or not, if not, integrating the data into the list to be updated, and updating the data of the data asset after integration is finished.
In the embodiment of the invention, the data profile module is used for carrying out the data display process as follows:
when the user type of the user is a common user, displaying each data list of the data asset library and data details corresponding to each data list to the user; when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
It can be understood that different display can be provided according to different logged-in users, and functions displayed by common users are as follows: the data list and the data details are added by an administrator user on the basis of a common user, so that the administrator can manage the data list conveniently. The data presented originates from a data warehouse and the latest information of the data table is synchronized to the data asset system at regular time through iteration. This module displays mainly the relevant information of the table viewed, including: basic information of the table (creation time, maintainer, etc.), classification to which the table belongs, field information in the table, blood relationship of the table, and use case.
In the embodiment of the invention, the process of the data profile module for carrying out association and searching of the data blood edges is as follows: acquiring a table number contained in the search information; determining a target data list matched with the table number in the data asset library; and searching each target data associated with the target data list based on the table number. And associating each data matched with the id based on the blood relationship among each target data associated with the target data list, obtaining associated data, and sending the associated data to the user.
Referring to fig. 3, fig. 3 is a schematic diagram of a connection process for searching data and blood relationship, and the corresponding specific process is:
(1) And acquiring the related data of the table according to the number of the input lookup table.
(2) Judging whether the acquired data in the step (1) is empty or not, and integrating the data if the acquired data is not empty.
(3) And (3) inquiring an upstream table and a downstream table of the table according to the number of the input table, integrating the data with the data in the step (2), and returning a result.
In the embodiment of the invention, the data profile module not only can realize the functions of calibrating, searching, displaying and the like of data, but also can realize the discarding and recovering processes of the list. Therefore, the data profile module is further used for restoring and displaying the data list corresponding to the restoring instruction when the restoring instruction corresponding to the abandoned data list is received.
After discarding the data list, the data profile module is further configured to obtain a feedback identifier included in the data correction message; when the feedback identifier is a first feedback identifier, determining a feedback type corresponding to the data correction message as a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
It should be noted that, the data profile module mainly manages the table, each iteration will synchronize new data to the data asset, after being checked strictly by the administrator, the whole data and related information of the table will be displayed in the data list, otherwise, the table will be in the checking state until the checking is completed. Thus, the data profiling module manages tables specifically including:
(1) Entering data table management: and obtaining the data of all tables according to the reverse order of the table time by default, and displaying the data in the list.
(2) Screening of the data sheet: the method can carry out combined inquiry according to the name of the library, the time for building the table, the state of the table and the fuzzy search, and improves the screening efficiency of the user.
(3) Auditing function: and the auditing function is arranged at the back of each table, the relevant information of the table needs to be filled after entering, and the auditing can be performed after filling.
(4) Discarding function: while checking, the table can be abandoned, after the table is abandoned, the information of the table is not displayed in the data profile, and if the table needs to be reused later, the table is also supported to be restored.
In the embodiment of the invention, the data feedback module is specifically configured to obtain a feedback identifier included in the data correction message; when the feedback identifier is a first feedback identifier, determining a feedback type corresponding to the data correction message as a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
It will be appreciated that the user, when sending the data correction message, defines the data to be corrected as a data type or an index type. For example, the first feedback is identified as 0 and the second feedback is identified as 1. When the feedback mark in the data correction message is 0, the feedback type is data type, and when the feedback mark is 1, the feedback type is index type.
Further, the data feedback module is specifically configured to record the feedback result to a preset first repository when the feedback type corresponding to the data correction message is a data type, and call the data profile module to classify the feedback result according to the first classification rule; when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
It should be noted that, the feedback result of the content supplement is newly added with updated data in the data warehouse, so that after the feedback result is obtained, the data profile module performs further management.
Referring to fig. 4, fig. 4 is a specific application process of the data feedback module, which specifically includes data feedback and index feedback. The data feedback process comprises the following steps:
(1) The user can select the basic situation of the data to be fed back according to the own problem, which comprises: feedback types, including: data quality, data loss; major and minor classes to which the data to be fed back belongs, a library name, an indication and a field; meanwhile, the description of the problem can be directly filled;
(2) Completing the data addition in the step (1), and submitting the data;
(3) The manager side processes the problem feedback, and finally backfills the processed result to inform the user of the problem processing condition;
(4) The user can know the result of the feedback of the problem through the feedback problem.
The index feedback process comprises the following steps:
(1) The user selects the index category and index name, fills out the question description, may submit,
(2) And the administrator receives the feedback of the index problem to process, and backfills the processing result, so that the user can check the feedback result through the login system.
It should be noted that, after receiving the feedback result, the data feedback module records and stores the feedback result and other feedback contents if other feedback contents exist.
In the embodiment of the invention, the search engine module is specifically configured to obtain a library name and a data category included in the search information, and query data corresponding to the search information from the data asset library based on the library name and the data category.
It will be appreciated that the search engine module is provided for better convenience to the user in terms of table queries, and that this module supports both individual queries and combined queries, and that fuzzy searches are supported in this module. The combined query comprises a reservoir name, a major class and a minor class and the combined query of the search data, so that the retrieval efficiency is improved greatly.
The library name query is to search out corresponding data according to the selected library name, and if the data exists in the fuzzy query, the data is carried to make a combined query. The major class query is based on the data corresponding to the major class query, if the library name or major class is not empty, the combination query is made, otherwise, the combination query is singly queried. The subclass query is that the subclass is constrained by the major class, if the major class is empty, the subclass has no data, otherwise, the corresponding subclass is displayed according to the corresponding major class, the combined query is made according to the selection, and the queried data is displayed.
The specific implementation process and derivative manner of the above embodiments are all within the protection scope of the present invention.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Those of skill would further appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both.
To clearly illustrate this interchangeability of hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A metadata management system, comprising:
The system comprises a data profile module, a data index module, a data feedback module and a search engine module;
The data profiling module is used for scanning the metadata information of the data warehouse in real time and calibrating the data in the data asset library in real time based on the metadata information in the data warehouse; classifying each data in the data asset library based on a preset first classification rule to obtain a data list corresponding to each category in the data asset library; when a user login is detected, based on the user type of the user, asset library information of the data asset library is displayed to the user, wherein the asset library information comprises each data list; when search information sent by the search engine module is received, searching each target data list corresponding to the search information, and constructing a blood-edge relation corresponding to each target data according to each target data associated with each target data list;
The data index module is used for carrying out index division on each data in each data list based on a preset second classification rule to obtain a plurality of index class data indexes under each data list;
The data feedback module is used for receiving data correction information sent by a user, determining a feedback type corresponding to the data correction information, wherein the feedback type is a data type or an index type; the data correction message is sent to a service end corresponding to the feedback type; when receiving a feedback result corresponding to the data correction message fed back by the service end, calling the data profile module or the data index module to classify result data corresponding to the feedback result based on the feedback type;
the search engine module is used for receiving search information sent by a user and sending the search information to the data profile module so that the data profile module queries data corresponding to the search information from the data asset library.
2. The metadata management system of claim 1, wherein the data profiling module is configured to determine whether new data exists in the data warehouse and whether updated data exists in the data warehouse based on the metadata information; when the data warehouse has newly added data, adding the newly added data to a preset newly added list; when the data warehouse has updated data, adding the updated data to a preset update list; storing the newly added data in the newly added list to the data asset library; and updating the data in the data asset library based on the updated data in the update list.
3. The metadata management system according to claim 1, wherein the data profile module is configured to present each data list of the data asset library and data details corresponding to each data list to the user when the user type of the user is a general user; when the user type of the user is a management user, displaying each data list of the data asset library, data details corresponding to each data list and a data management module corresponding to the data asset library to the user.
4. The metadata management system according to claim 1, wherein the data profile module is configured to acquire a table number included in the search information; determining a target data list matched with the table number in the data asset library; and searching each target data associated with the target data list based on the table number.
5. The metadata management system according to claim 1, wherein the data profile module is configured to associate each data matching the id based on a blood relationship between each target data associated with the target data list, obtain associated data, and send the associated data to the user.
6. The metadata management system according to any one of claims 1 to 5, wherein the data profiling module is further configured to: and when receiving a discarding instruction corresponding to any data list in the data asset library, canceling displaying the data list corresponding to the discarding instruction.
7. The metadata management system according to claim 6, wherein the data profile module is further configured to, when receiving a restore instruction corresponding to a discarded data list, restore and display the data list corresponding to the restore instruction.
8. The metadata management system according to claim 1, wherein the data feedback module is configured to obtain a feedback identifier included in the data correction message; when the feedback identifier is a first feedback identifier, determining a feedback type corresponding to the data correction message as a data type; and when the feedback identifier is a second feedback identifier, determining that the feedback type corresponding to the data correction message is an index type.
9. The metadata management system according to claim 1 or 8, wherein the data feedback module is configured to record the feedback result to a preset first repository and invoke the data profile module to classify the feedback result according to the first classification rule when the feedback type corresponding to the data correction message is a data type; when the feedback type corresponding to the data correction message is an index type, recording the feedback result to a preset second storage library, and calling the data index module to classify the feedback result according to the second classification rule.
10. The metadata management system of claim 1, wherein the search engine module is configured to obtain a repository name and a data category included in the search information, and query data corresponding to the search information from the data asset repository based on the repository name and the data category.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210287993.XA CN114817149B (en) | 2022-03-23 | 2022-03-23 | Metadata Management System |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210287993.XA CN114817149B (en) | 2022-03-23 | 2022-03-23 | Metadata Management System |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114817149A CN114817149A (en) | 2022-07-29 |
| CN114817149B true CN114817149B (en) | 2024-07-23 |
Family
ID=82531143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210287993.XA Active CN114817149B (en) | 2022-03-23 | 2022-03-23 | Metadata Management System |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114817149B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897836A (en) * | 2020-07-03 | 2020-11-06 | 中国建设银行股份有限公司 | A search system, method and storage medium |
| CN113468166A (en) * | 2020-03-31 | 2021-10-01 | 广州虎牙科技有限公司 | Metadata processing method and device, storage medium and server |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2545232A1 (en) * | 2005-07-29 | 2007-01-29 | Cognos Incorporated | Method and system for creating a taxonomy from business-oriented metadata content |
| US10791433B1 (en) * | 2019-03-29 | 2020-09-29 | Rovi Guides, Inc. | Systems and methods for personalizing messages based on a recipient relationship |
-
2022
- 2022-03-23 CN CN202210287993.XA patent/CN114817149B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113468166A (en) * | 2020-03-31 | 2021-10-01 | 广州虎牙科技有限公司 | Metadata processing method and device, storage medium and server |
| CN111897836A (en) * | 2020-07-03 | 2020-11-06 | 中国建设银行股份有限公司 | A search system, method and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114817149A (en) | 2022-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11727025B2 (en) | Method and system for implementing a log parser in a log analytics system | |
| US8326864B2 (en) | Method, system, and computer program product for implementing automated worklists | |
| JP4676199B2 (en) | Systems and methods for integrating, managing, and coordinating customer activities | |
| US8788525B2 (en) | Data model for machine data for semantic search | |
| US10452668B2 (en) | Smart defaults for data visualizations | |
| US9292623B2 (en) | System and method for regulatory intelligence | |
| US8145681B2 (en) | System and methods for generating manufacturing data objects | |
| US8296311B2 (en) | Solution search for software support | |
| US10216769B1 (en) | Asset catalog management methods and systems | |
| US20060265377A1 (en) | Personalizable information networks | |
| US20140330821A1 (en) | Recommending context based actions for data visualizations | |
| CN115374329B (en) | A method and system for managing enterprise business metadata and technical metadata | |
| US20140074801A1 (en) | Data de-duplication system | |
| US20150074103A1 (en) | Metadata-driven audit reporting system with dynamically created display names | |
| US8452773B1 (en) | Mapping terms between multiple taxonomies | |
| US20110010346A1 (en) | Processing related data from information sources | |
| WO2007127956A2 (en) | Apparatus and method for merging metadata within a repository | |
| US20090319559A1 (en) | Method And System of Using Social Networks and Communities to Ensure Data Quality of Configuration Items in a Configuration Management Database | |
| US20230368091A1 (en) | Systems and methods for efficiently distributing alert messages | |
| US7593957B2 (en) | Hybrid data provider | |
| US20100325101A1 (en) | Marketing asset exchange | |
| US12423275B2 (en) | Database metadata update validation | |
| US20090319537A1 (en) | Method And System of Using Structured Social Networks and Communities to Create And Maintain Relationships Between Configuration Items in a Configuration Management Database | |
| CN111414410A (en) | Data processing method, device, equipment and storage medium | |
| CN114817149B (en) | Metadata Management System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 100053 No. 28 Xuanwu Men Xi Street, Xicheng District, Beijing Patentee after: Beijing Yuehan Technology Co., Ltd. Country or region after: China Address before: 100055 Beijing City Xicheng District Langqin International A Building 17th Floor Patentee before: Jianxin Rongtong Co.,Ltd. Country or region before: China |