[go: up one dir, main page]

CN118964454A - Data exchange method, device and medium based on Connector - Google Patents

Data exchange method, device and medium based on Connector Download PDF

Info

Publication number
CN118964454A
CN118964454A CN202410944390.1A CN202410944390A CN118964454A CN 118964454 A CN118964454 A CN 118964454A CN 202410944390 A CN202410944390 A CN 202410944390A CN 118964454 A CN118964454 A CN 118964454A
Authority
CN
China
Prior art keywords
data
user
access
connector
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410944390.1A
Other languages
Chinese (zh)
Inventor
傅德谦
王雯静
孔瑞
刘子琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Linyi University
Original Assignee
Linyi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Linyi University filed Critical Linyi University
Priority to CN202410944390.1A priority Critical patent/CN118964454A/en
Publication of CN118964454A publication Critical patent/CN118964454A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24558—Binary matching operations
    • G06F16/2456—Join operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/604—Tools and structures for managing or administering access control systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/2141—Access rights, e.g. capability lists, access control lists, access tables, access matrices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Storage Device Security (AREA)

Abstract

本申请实施例公开了一种基于Connector的数据交换方法、设备及介质。属于数据处理技术领域，解决数据交换方式在数据传输过程中隐私性和完整性得不到保障的问题。包括，基于元构件配置或基于ODBC的数据访问配置，通过Connector连接器对所需数据进行获取；对获取到的所需数据进行预处理，并将预处理后的数据进行特定字段遮盖处理；获取用户对应的身份凭证，在身份凭证检测通过的情况下，确定用户具有访问Connector的权限；基于预置访问规则，确定具有访问Connector的权限的用户对应的数据访问范围；基于数据访问范围，对进行特定字段遮盖处理后的所需数据进行查询，以将符合要求的数据展示给用户。

The embodiment of the present application discloses a data exchange method, device and medium based on Connector. It belongs to the field of data processing technology, and solves the problem that the privacy and integrity of data exchange methods cannot be guaranteed during data transmission. It includes obtaining the required data through the Connector connector based on component configuration or ODBC-based data access configuration; preprocessing the obtained required data, and performing specific field masking on the preprocessed data; obtaining the identity credentials corresponding to the user, and determining that the user has the right to access the Connector if the identity credentials are detected; determining the data access scope corresponding to the user with the right to access the Connector based on preset access rules; and querying the required data after the specific field masking based on the data access scope to display the data that meets the requirements to the user.

Description

Connector-based data exchange method, device and medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a Connector-based data exchange method, device, and medium.
Background
In the digital age, data has become a core asset for enterprise operations and decisions, and efficient and secure exchange and sharing of data is critical to promoting business development and maintaining market competitiveness.
However, the conventional data exchange methods, such as API interfaces, file transfer protocols, webSockets, data exchange platforms, remote procedure calls, and ETL (Extract-Transform-Load) tools, expose many security risks while promoting data circulation. For example, many conventional data exchange methods lack sufficient encryption protection during data transmission, and are extremely easy to intercept, parse and even tamper by lawbreakers. This can not only lead to leakage of enterprise secrets, but can also pose a serious threat to customer privacy, such that the privacy and integrity of data is a serious threat.
Disclosure of Invention
The embodiment of the application provides a Connector-based data exchange method, device and medium, which are used for solving the following technical problems: many conventional data exchange schemes lack sufficient encryption protection during data transmission such that the privacy and integrity of the data pose a serious threat.
The embodiment of the application adopts the following technical scheme:
The embodiment of the application provides a data exchange method based on a Connector. The method comprises the steps of acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration; preprocessing the acquired required data, and masking the preprocessed data by a specific field; acquiring an identity credential corresponding to a user, and determining that the user has permission to access a Connector under the condition that the identity credential passes detection; determining a data access range corresponding to a user with the authority to access the Connector based on a preset access rule; the preset access rule is related to the user identity, the user role and the user authority level; and inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
The embodiment of the application realizes non-invasive data acquisition and retrieval through the mutual communication and cooperation of connectors, avoids direct operation and modification of the original system, and reduces secondary development of the system. And secondly, the integration of multi-source heterogeneous data is realized based on the data acquisition and retrieval of the components and the data access based on the ODBC. In addition, the embodiment of the application introduces a security authentication and access control mechanism, supports various identity verification modes, and ensures the security and data protection in the data exchange process. The non-invasive data exchange method based on the Connector provides key components such as efficient data collection, access control, security authentication, data processing and the like, ensures that the flow and interaction of data are seamless, enables a system to efficiently utilize the data, and has high efficiency and controllability.
In one implementation of the present application, the acquiring, through the Connector, the required data based on the meta-component configuration or the ODBC-based data access configuration specifically includes: when the required data is acquired through the component configuration, determining a corresponding component in a component pool based on the acquired metadata; determining a corresponding analysis method based on a data structure and a data format stored by a data source;
based on the components and the corresponding analysis method, the required data are acquired in the data storage system through the Connector.
In one implementation of the present application, the acquiring, through the Connector, the required data based on the meta-component configuration or the ODBC-based data access configuration specifically includes: when the required data is acquired through the ODBC-based data access configuration, establishing connection with a data source through configuration of ODBC data source information; based on the obtained query content, query type matching is carried out, and an SQL query request is sent to a data source through a Connector; determining query data in a data source based on the query type; optimizing the queried information to take the processed data as required data; the optimization processing at least comprises one of information selection, information filtering and information arrangement.
In one implementation manner of the present application, preprocessing is performed on acquired required data, and specific field covering processing is performed on the preprocessed data, which specifically includes: preprocessing required data; wherein the preprocessing at least comprises one of date formatting processing and data cleaning; determining a data source system and a target system corresponding to the preprocessed data, and performing format conversion on the preprocessed data based on data formats respectively corresponding to the data source system and the target system so as to map the preprocessed data to the target system; determining a data type corresponding to the preprocessed data, and determining a corresponding shielding field in a preset shielding content table based on the data type; and performing specific field covering processing on the preprocessed data based on the mask field.
In one implementation of the present application, specific field covering processing is performed on the preprocessed data based on the mask field, which specifically includes: determining a shielding type corresponding to the shielding field based on the field type corresponding to the shielding field; wherein the mask type comprises at least one of fixed character substitution, random character generation, partial character reservation, and character encryption; determining an encryption grade corresponding to the field type, and superposing the shielding type based on the encryption grade; and covering the mask field based on the overlapped mask type.
In one implementation manner of the application, an identity credential corresponding to a user is obtained, and when the identity credential passes detection, the user is determined to have the right to access the Connector, which specifically comprises the following steps: acquiring an identity credential corresponding to a user; the identity certificate is encrypted through a preset encryption algorithm; decrypting the identity credential based on a decryption algorithm corresponding to a preset encryption algorithm to obtain a reference identity credential and a reference timestamp; carrying out hash value calculation on the reference identity certificate, and comparing the calculated hash value with a prestored hash value; performing validity detection on the reference timestamp; and under the condition that the comparison results are consistent and the effectiveness detection is passed, carrying out data query permission allocation on the user based on the reference identity certificate, and determining the permission of the user to access the Connector based on the data query permission.
In one implementation of the present application, determining a data access range corresponding to a user having a right to access a Connector based on a preset access rule specifically includes: determining a reference data access range corresponding to a user based on a preset access rule, a user identity and a user role; determining abnormal access information and normal access information based on historical access information corresponding to a user, and determining initial confidence corresponding to the user based on a numerical value ratio between the abnormal access information and the normal access information; acquiring reference users with association relation with the users, and acquiring reference confidence degrees respectively corresponding to the reference users; and determining final confidence corresponding to the user based on the initial confidence and each reference confidence, and adjusting the reference data access range based on the final confidence to obtain the data access range.
In one implementation manner of the present application, based on a data access range, the method queries required data after specific field covering processing to display the required data to a user, specifically includes: acquiring query information sent by a user, and querying in a data access range based on the query information to obtain data to be displayed; based on user permission, determining a viewable field, matching the viewable field with the covered specific field, and displaying the covered specific field in the data to be displayed based on a matching result; acquiring historical query information corresponding to a user, and sorting the historical query information based on the query times; matching the data after the display processing with the historical query information to determine a display sequence corresponding to the data after the display processing; and displaying the data after the display processing to a user based on the display sequence.
The embodiment of the application provides a Connector-based data exchange device, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to: acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration; preprocessing the acquired required data, and masking the preprocessed data by a specific field; acquiring an identity credential corresponding to a user, and determining that the user has permission to access a Connector under the condition that the identity credential passes detection; determining a data access range corresponding to a user with the authority to access the Connector based on a preset access rule; the preset access rule is related to the user identity, the user role and the user authority level; and inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
The non-volatile computer storage medium provided by the embodiment of the application stores computer executable instructions, and the computer executable instructions are set as follows: acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration; preprocessing the acquired required data, and masking the preprocessed data by a specific field; acquiring an identity credential corresponding to a user, and determining that the user has permission to access a Connector under the condition that the identity credential passes detection; determining a data access range corresponding to a user with the authority to access the Connector based on a preset access rule; the preset access rule is related to the user identity, the user role and the user authority level; and inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
The above at least one technical scheme adopted by the embodiment of the application can achieve the following beneficial effects: the embodiment of the application realizes non-invasive data acquisition and retrieval through the mutual communication and cooperation of connectors, avoids direct operation and modification of the original system, and reduces secondary development of the system. And secondly, the integration of multi-source heterogeneous data is realized based on the data acquisition and retrieval of the components and the data access based on the ODBC. In addition, the embodiment of the application introduces a security authentication and access control mechanism, supports various identity verification modes, and ensures the security and data protection in the data exchange process. The non-invasive data exchange method based on the Connector provides key components such as efficient data collection, access control, security authentication, data processing and the like, ensures that the flow and interaction of data are seamless, enables a system to efficiently utilize the data, and has high efficiency and controllability.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art. In the drawings:
Fig. 1 is a flowchart of a data exchange method based on a Connector according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a relationship between a component pool, a meta-component, and a data element according to an embodiment of the present application;
Fig. 3 is a schematic structural diagram of a Connector-based data exchange device according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a Connector-based data exchange method, device and medium.
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
The following describes the technical scheme provided by the embodiment of the invention in detail through the attached drawings.
Fig. 1 is a flowchart of a data exchange method based on a Connector according to an embodiment of the present application.
As shown in fig. 1, the data exchange method includes the steps of:
S101, acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration.
In one embodiment of the application, when the required data is acquired through the meta-component configuration, the corresponding meta-component is determined in the component pool based on the acquired meta-data. Based on the data structure and data format stored by the data source, a corresponding parsing method is determined. Based on the components and the corresponding analysis method, the required data are acquired in the data storage system through the Connector.
In particular, metadata-based data collection and retrieval obtains data from a variety of different data storage modalities, including warehouse management systems, supplier databases, transportation management systems, files, and file systems, etc., through Connector connectors, reflecting the diversity and complexity of the data sources. In the process of data acquisition and retrieval, the data structure and format stored by the data sources are considered to ensure that the data can be correctly analyzed and processed, the characteristics of different data sources are deeply understood, and proper data processing and analysis methods are adopted. Data sources mainly refer to databases, files, web services, user inputs and the like. For relational databases, SQL (structured query language) is used for querying, inserting, updating and deleting data. For different text files, file manipulation functions or libraries in the programming language may be used to read and parse the file contents. For websites that do not provide an API interface, UIPath is utilized to crawl the web page content and extract the required data. And verifying the data input by the user through the form, and ensuring the accuracy and the integrity of the data.
Further, the metadata-based data collection and retrieval does not rely solely on a single data element, but rather is represented in terms of a higher level of metadata and reorganization of the data elements. The core idea is to access the data by reorganizing and utilizing the meta-members instead of directly accessing the original data elements. The meta-members are supported by various data elements, and typically contain a richer association between information and data elements, thereby providing a more comprehensive data representation and processing capability.
Further, the data elements are basic building blocks of the meta-component, they represent the minimum unit of data, and the data elements may be different types of data such as numbers, text, dates, etc. In the meta-component, one or more data elements are interrelated, together forming a larger data entity. During data access, the system utilizes meta-members corresponding to the data elements in the member pool to perform data access. The component pool is a central storage area for storing and managing the components, so that the data requirements can be responded quickly, and the efficiency and accuracy of data access can be improved. Fig. 2 is a schematic diagram of a relationship among a component pool, a component and a data element, where, as shown in fig. 2, the component is searched according to metadata, where a plurality of different components are stored in the component pool, and different components respectively include different data elements, and data extraction is performed according to the components, for example, data extraction may be performed in a logistics information system.
In one embodiment of the present application, when the required data is acquired through the ODBC-based data access configuration, a connection with the data source is established through configuration of ODBC data source information. And carrying out query type matching based on the acquired query content, and sending an SQL query request to a data source through a Connector. Query data is determined in the data source based on the query type. Optimizing the queried information to take the processed data as required data; the optimization processing at least comprises one of information selection, information filtering and information arrangement.
In particular, ODBC-based data access is another collection approach. The Connector establishes a connection with a data source by configuring ODBC data source information. The Connector can automatically identify the type, structure and parameters of the data source and generate corresponding connection configuration, so that intelligent configuration is realized. The Connector can intelligently identify the type and parameters of the data source and automatically complete the configuration of the connection. The user does not need to manually input complicated connection information, but enables the Connector to automatically complete connection setting according to the characteristics of the data source, so that time and energy are saved.
Further, once a connection is established with a data source, the Connector can send an SQL query request to obtain the required data. The Connector in the embodiment of the application can be optimized according to the query requirement of the user, and comprises the operations of selection, filtering, sorting and the like so as to ensure that only required data is retrieved. Therefore, the data retrieval efficiency can be improved, unnecessary data transmission and processing are reduced, and the data acquisition speed is further increased.
Further, the obtained data is returned in a structured mode after being processed by a Connector. The embodiment of the application also provides a data processing function, which can further process and convert the data so as to adapt to the data standard and format requirement of a target system. This includes data mapping, data cleansing, data conversion, etc., to ensure the quality and consistency of the data. Such data processing functionality enables a user to integrate data directly into a target system without additional data processing effort.
S102, preprocessing the acquired required data, and masking the preprocessed data by a specific field.
In one embodiment of the application, the required data is preprocessed; wherein the preprocessing includes at least one of date formatting and data cleansing. Determining a data source system and a target system corresponding to the preprocessed data, and performing format conversion on the preprocessed data based on data formats respectively corresponding to the data source system and the target system so as to map the preprocessed data to the target system. And determining a data type corresponding to the preprocessed data, and determining a corresponding mask field in a preset mask content table based on the data type. And performing specific field covering processing on the preprocessed data based on the mask field.
Specifically, preprocessing in embodiments of the present application refers to an important sequence of normalization, cleaning, and conversion steps performed on the collected data. In the process of logistics data, various operations need to be carried out on the data so as to ensure that the data are uniform in format, clear in content and meet the requirements of a target system. First, the date format needs to be standardized so that date data from different sources can be correctly identified and processed by the system.
Second, duplicate items or erroneous data in the data need to be cleared to ensure that analysis and decision-making is based on accurate information.
Further, the data conversion is to map the data to the data standard adopted by the target system, so that the data can be transferred and applied seamlessly between different systems. Thereby ensuring the accuracy and consistency of the data and providing a reliable basis for the subsequent data analysis and service application.
Further, in order to ensure the security and privacy of the data, the preprocessed data needs to be covered. In particular in the field of logistics, data processing is particularly important, as logistics data may relate to information that is not authorized to be accessed. Specifically, the data hiding is a blurring process for hiding part of the content of the data to prevent information leakage. And determining corresponding shielding fields in a preset shielding content table according to the types of the preprocessed data, wherein the preset shielding content table comprises a plurality of data types and shielding fields respectively corresponding to different data types. In combination, data processing plays a vital role in the logistics industry, so that not only can data security be protected, but also the availability and applicability of the data can be improved.
In one embodiment of the application, the mask type corresponding to the mask field is determined based on the field type corresponding to the mask field; wherein the mask type includes at least one of fixed character substitution, random character generation, partial character reservation, and character encryption. And determining an encryption grade corresponding to the field type, and superposing the shielding type based on the encryption grade. And covering the mask field based on the overlapped mask type.
In particular, common types of sensitive fields include personal identity information: such as name, identification card number, passport number, etc., financial information: such as bank card number, transaction amount, etc., contact means: such as telephone numbers, email boxes, etc. Each field type requires a different masking approach because of the different sensitivity and format of the information it contains. For example, replacing all or part of the characters in the field with a specific character (e.g., an asterisk), replacing sensitive information in the field with a randomly generated sequence of characters, retaining part of the characters in the field (e.g., the beginning and ending characters), replacing the remaining part with a specific character, or encrypting the field using an encryption algorithm to render the sensitive information unreadable ciphertext.
Further, an encryption level corresponding to the field is determined, wherein the encryption level reflects the data sensitivity and the degree of protection. Different field types may correspond to different encryption levels. For example, highly sensitive information such as identification numbers, bank card numbers, etc. may require a higher encryption level, while some relatively less sensitive fields (e.g., telephone numbers) may employ a lower encryption level. In some cases, a single mask type may not be sufficient to meet the data protection requirements. At this time, the superimposition processing may be performed on the mask type according to the encryption level. For example, for highly sensitive fields, character encryption may be first performed, and then the encrypted result may be subjected to fixed character substitution or partial character processing reserved to increase the level and difficulty of data protection. And carrying out actual covering treatment on the sensitive fields according to the determined shielding type and encryption level.
Furthermore, the application automatically covers the sensitive information by adopting the RPA technology, and only displays necessary information. For example UiPath may automatically mask sensitive information, only showing the necessary information. UiPath can automatically shield specific fields in files such as PDF, excel and the like, so that sensitive information is ensured to be visible only to personnel with specific rights. UiPath the automated process can obtain information from a variety of data sources and through a series of preprocessing steps, ensure data consistency and accuracy. After data acquisition and processing, the user with access rights can search the processed data through the Connector. Before access, the user needs to confirm and control the access through the security mechanism such as UiPath Orchestrator, so as to ensure the security of the data.
Specifically, users with different authorities can retrieve different information conforming to the roles and authorities of the users according to the set access rules. For example, a warehouse manager may access inventory data without being able to view real information. Data hiding is a widely used technique in data sharing and display, and its main purpose is to hide or obscure the restricted access portion in the data to limit the access range of the information. In the field of logistics, data hiding is often applied to processing information such as logistics orders and goods tracking. Through data coverage, only authorized users can view complete data, and other people can only see partial information or fuzzy information, so that the security and privacy of sensitive data are protected.
For example, assuming a logistics company needs to share cargo tracking information to customers, but does not want customers to be able to see a specific cargo location, a data mask may be used to display only obscured location information or to hide a portion of sensitive cargo information to customers. Therefore, even if the data is accessed by unauthorized persons, important information cannot be revealed, and the safety and privacy of the data are guaranteed.
S103, acquiring an identity credential corresponding to the user, and determining that the user has the authority to access the Connector under the condition that the identity credential passes detection.
In one embodiment of the application, an identity credential corresponding to a user is obtained; wherein the identity credential has been encrypted by a preset encryption algorithm. And decrypting the identity credential based on a decryption algorithm corresponding to a preset encryption algorithm to obtain a reference identity credential and a reference timestamp. And carrying out hash value calculation on the reference identity certificate, and comparing the calculated hash value with a prestored hash value. And performing validity detection on the reference timestamp. And under the condition that the comparison results are consistent and the effectiveness detection is passed, carrying out data query permission allocation on the user based on the reference identity certificate, and determining the permission of the user to access the Connector based on the data query permission.
Specifically, a user identity credential is obtained, where the identity credential in the embodiment of the present application is identity information, such as a user name, a password, a token, etc., that a user holds in order to access a system or perform some operation. The identity credentials are encrypted by a preset encryption algorithm to protect the security of the data during transmission or storage. The encrypted identity credentials are restored to the original information by a decryption algorithm corresponding to the preset encryption algorithm. The decrypted information typically contains the user's identity credential (e.g., a user name) and a timestamp. And carrying out hash value calculation on the decrypted reference identity certificate, and comparing the calculated hash value with a prestored hash value to verify whether the identity certificate submitted by the user is true and effective. And if the comparison results are consistent, indicating that the identity certificate submitted by the user is legal.
Further, by comparing the reference time stamp with the current time, it is determined whether the encryption operation is completed within a certain valid time range. If the time difference exceeds a preset threshold (e.g., minutes, hours, etc.), the identity credential is deemed to have expired and is no longer valid. And under the condition that the comparison results are consistent and the validity detection is passed, the system distributes corresponding data query authorities according to the reference identity credentials (actually, the user identity). These rights may include the range of data that the user can query, the level of detail of the query, etc.
Further, security authentication is the first line of defense to ensure Connector security and data protection. Only authenticated users or components may enter the system, thereby reducing unauthorized access and potential security vulnerabilities. It is the process in the Connector that confirms the validity of the user identity or component, which is typically done before access control is implemented. Security authentication in embodiments of the present application is supported by UiPath Orchestrator for managing access rights of users, robots, or other components connected to the Connector. UiPath Orchestrator also support various ways of identity verification including user name and password, single sign-on authentication, multi-factor authentication, etc., the user or component can perform identity verification by providing UiPath Orchestrator supported identity credentials. During the security authentication process UiPath Orchestrator may indicate that the user or component has been authenticated by issuing a temporary access token (Access Tokens), with the right to access the Connector. Second, the centralized identity management function of UiPath Orchestrator may be utilized to assign appropriate rights to a user or component. In the data exchange method based on the non-invasive data acquisition and retrieval of the Connector, the security authentication can also prevent unauthorized data access, ensure the security of the connection to different systems or data sources, and jointly construct a secure Connector architecture.
S104, determining a data access range corresponding to the user with the authority of accessing the Connector based on a preset access rule.
In one embodiment of the application, the reference data access range corresponding to the user is determined based on preset access rules, user identity and user role. Based on the historical access information corresponding to the user, abnormal access information and normal access information are determined, and based on the numerical value ratio between the abnormal access information and the normal access information, initial confidence corresponding to the user is determined. And obtaining reference users with association relation with the users, and obtaining the reference confidence degrees respectively corresponding to the reference users. And determining final confidence coefficient corresponding to the user by the initial confidence coefficient and each reference confidence coefficient, and adjusting the reference data access range based on the final confidence coefficient to obtain the data access range.
Specifically, the embodiment of the application presets a series of access rules which define the data ranges which can be accessed by different identities (such as staff and management layers) and roles (such as department manager and project manager). Based on the identity information (such as employee number, name) and role information (such as post, responsibility) of the user, the data access rights that the user should enjoy, i.e. the reference data access scope, can be preliminarily determined. The embodiment of the application records the past data access behaviors of the user, including the type, time, frequency and the like of the accessed data. By analyzing the historical access information, abnormal access behaviors of the user, such as frequent access to data in non-functional areas, access to sensitive data at irregular times, and normal access behaviors, are identified. By calculating the proportion between the abnormal access information and the normal access information, the compliance and risk of the user access behavior can be evaluated, and then the initial confidence of the user can be determined.
Further, reference users having direct or indirect associations with the user are identified, which associations may be based on work relationships (e.g., colleagues, superordinates), social network relationships, etc., and for each reference user, a reference confidence level is also calculated in a similar manner (based on their identity, roles, historical access behaviors, etc.), indicating the trustworthiness of the user's access behavior. And comprehensively evaluating the initial confidence coefficient of the user and the reference confidence coefficient of each reference user through a certain algorithm (such as weighted average, machine learning model and the like) to obtain the final confidence coefficient of the user. And dynamically adjusting the data access range of the user by the system according to the final confidence level of the user. If the final confidence is higher, the access behavior of the user is more compliant and credible, and the data access authority of the user can be possibly enlarged; conversely, if the final confidence is low, the system may restrict or reduce its data access rights to reduce potential security risks.
Furthermore, in the data exchange method based on the non-invasive data acquisition and retrieval of the Connector, the access control is a key component for ensuring the safety of system data and functions, and is also a second defense line of the Connector. The access control in the embodiments of the present application aims to define who can access the system, and the data and functional scope that can be accessed. The realization of the target depends on preset rules and the safety authentication passed by the user or the component, and meanwhile, a semantic-based access control reasoning method is adopted to ensure the accuracy and the high efficiency of the access control. Wherein the formulation of the access control rules involves consideration of a number of factors, all of which are determined by the data holder.
Specifically, the identity of the user, the role played, the level of authority, etc. are all important factors affecting the setting of access control rules. When rules are formulated, flexible adjustment can be performed according to different requirements and scenes so as to ensure that only authorized users can access the system. For example, a rule may define a particular type of data that a user can access or perform a particular operation, with its access rights being dynamically determined based on a particular attribute of the user or system. The rule can be deduced according to semantic information, so that access control can be effectively implemented under various situations, for example, a user is taken as an A warehouse manager, and has the authority to view information of goods in the A warehouse and manage the goods, and in this case, the manager has the authority to view information of fragile goods, dangerous goods and the like, and the access control is deduced through set semantics. In particular, these rules can be combined with the user's role and current context information, such as the system can dynamically determine whether the user has the right to perform an operation or access specific data by accessing information it obtains and information that is inferred later or attributes that the user has. This means that the data holder can update the rules to clarify the ownership of the data according to new traffic requirements or security considerations without modifying the code of the application. The flexible access control mechanism enables the system to be better suitable for different application scenes, like in a logistics scene, warehouse staff can access inventory data according to the self manager attribute, but cannot see order data or transportation data, and a carrier can check the transportation data of goods by taking the carrier as the attribute of an operator of a transportation means, so that the safety of the system is improved, and the convenience of use of a user is also improved.
S105, inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
In one embodiment of the application, query information sent by a user is acquired, and based on the query information, the query is performed within a data access range to obtain data to be displayed. Based on the user permission, determining a viewable field, matching the viewable field with the covered specific field, and displaying the covered specific field in the data to be displayed based on a matching result. And acquiring historical query information corresponding to the user, and sorting the historical query information based on the query times. And matching the data after the display processing with the historical query information to determine the display sequence corresponding to the data after the display processing. And displaying the data after the display processing to a user based on the display sequence.
Specifically, first it is necessary to receive query information entered by a user, typically through a Web form, API request, or other user interface element. The query information may include specific search terms, filter terms, data ranges, and the like. And executing the query operation within the allowed data access range according to the query information provided by the user. The user's rights are checked to determine which data fields the user is entitled to view. The user's rights may be based on their role, position, or other security policy. All specific fields (e.g., sensitive information, private data, etc.) that need to be obscured are identified and then matched to fields viewable by the user. For fields which the user has no authority to view, the system can adopt a mode of covering, hiding or replacing with placeholders and the like to process, so that the data displayed to the user is ensured to accord with the authority range of the user, and sensitive information is protected from being revealed.
Further, historical query information of the user is recorded and maintained, including query content, time stamps, and the like. These historical query information are ranked according to the number of queries or other ranking criteria (e.g., chronological order). And matching the data after the display processing with the historical query information of the user to determine the optimal display sequence. And displaying the processed data to a user in a user-friendly mode according to the determined display sequence.
Fig. 3 is a schematic structural diagram of a Connector-based data exchange device according to an embodiment of the present application. As shown in fig. 3, the Connector-based data exchange device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to: acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration; preprocessing the acquired required data, and masking the preprocessed data by a specific field; acquiring an identity credential corresponding to a user, and determining that the user has permission to access a Connector under the condition that the identity credential passes detection; determining a data access range corresponding to a user with the authority to access the Connector based on a preset access rule; the preset access rule is related to the user identity, the user role and the user authority level; and inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
The non-volatile computer storage medium provided by the embodiment of the application stores computer executable instructions, and the computer executable instructions are set as follows: acquiring required data through a Connector based on meta-component configuration or ODBC based data access configuration; preprocessing the acquired required data, and masking the preprocessed data by a specific field; acquiring an identity credential corresponding to a user, and determining that the user has permission to access a Connector under the condition that the identity credential passes detection; determining a data access range corresponding to a user with the authority to access the Connector based on a preset access rule; the preset access rule is related to the user identity, the user role and the user authority level; and inquiring the required data subjected to specific field covering processing based on the data access range so as to display the data meeting the requirements to a user.
The embodiments of the present application are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for apparatus, devices, non-volatile computer storage medium embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the section of the method embodiments being relevant.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the embodiments of the application by those skilled in the art. Such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (10)

1.一种基于Connector的数据交换方法，其特征在于，所述方法包括：1. A data exchange method based on Connector, characterized in that the method comprises: 基于元构件配置或基于ODBC的数据访问配置，通过Connector连接器对所需数据进行获取；Based on component configuration or ODBC-based data access configuration, the required data is obtained through the Connector. 对获取到的所述所需数据进行预处理，并将预处理后的数据进行特定字段遮盖处理；Preprocessing the acquired required data, and performing specific field masking processing on the preprocessed data; 获取用户对应的身份凭证，在所述身份凭证检测通过的情况下，确定所述用户具有访问Connector的权限；Obtain the identity credentials corresponding to the user, and if the identity credentials pass the detection, determine that the user has the authority to access the Connector; 基于预置访问规则，确定具有所述访问Connector的权限的用户对应的数据访问范围；其中，所述预置访问规则与用户身份、用户角色以及用户权限级别相关；Based on preset access rules, determine the data access scope corresponding to the user who has the permission to access the Connector; wherein the preset access rules are related to the user identity, user role and user permission level; 基于所述数据访问范围，对所述进行特定字段遮盖处理后的所述所需数据进行查询，以将符合要求的数据展示给用户。Based on the data access scope, the required data after the specific field masking process is queried to display the data that meets the requirements to the user. 2.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述基于元构件配置或基于ODBC的数据访问配置，通过Connector连接器对所需数据进行获取，具体包括：2. A Connector-based data exchange method according to claim 1, characterized in that the component-based configuration or ODBC-based data access configuration obtains the required data through the Connector connector, specifically comprising: 在通过所述元构件配置对所述所需数据进行获取时，基于获取到的元数据在构件池中确定出相应的元构件；When the required data is acquired through the meta-component configuration, a corresponding meta-component is determined in the component pool based on the acquired metadata; 基于数据源存储的数据结构与数据格式，确定出相应的解析方法；Determine the corresponding parsing method based on the data structure and format stored in the data source; 基于所述元构件与所述相应的解析方法，通过Connector连接器在数据存储系统中对所需数据进行获取。Based on the meta-component and the corresponding parsing method, the required data is acquired in the data storage system through the Connector. 3.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述基于元构件配置或基于ODBC的数据访问配置，通过Connector连接器对所需数据进行获取，具体包括：3. A Connector-based data exchange method according to claim 1, characterized in that the component-based configuration or ODBC-based data access configuration obtains the required data through the Connector connector, specifically comprising: 在通过所述基于ODBC的数据访问配置对所述所需数据进行获取时，通过配置ODBC数据源信息建立与数据源的连接；When acquiring the required data through the ODBC-based data access configuration, establishing a connection with the data source by configuring the ODBC data source information; 基于获取到的查询内容进行查询类型匹配，通过Connector向所述数据源发送SQL查询请求；Perform query type matching based on the acquired query content, and send an SQL query request to the data source through the Connector; 基于所述查询类型在所述数据源中确定出查询数据；Determining query data in the data source based on the query type; 对查询到的信息进行优化处理，以将处理后的数据作为所述所需数据；其中，所述优化处理至少包括信息选择、信息过滤以及信息排列中的一项。The queried information is optimized so as to use the processed data as the required data; wherein the optimization process includes at least one of information selection, information filtering and information arrangement. 4.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述对获取到的所述所需数据进行预处理，并将预处理后的数据进行特定字段遮盖处理，具体包括：4. According to claim 1, a data exchange method based on Connector is characterized in that the acquired required data is preprocessed and the preprocessed data is subjected to specific field masking processing, which specifically includes: 对所述所需数据进行预处理；其中，所述预处理至少包括日期格式化处理与数据清洗中的一项；Preprocessing the required data; wherein the preprocessing includes at least one of date formatting and data cleaning; 确定出预处理后的数据对应的数据来源系统与目标系统，并基于所述数据来源系统与目标系统分别对应的数据格式，对所述预处理后的数据进行格式转换，以将所述预处理后的数据映射至所述目标系统；Determine a data source system and a target system corresponding to the preprocessed data, and perform format conversion on the preprocessed data based on data formats corresponding to the data source system and the target system, respectively, so as to map the preprocessed data to the target system; 确定出所述预处理后的数据对应的数据类型，基于所述数据类型在预置屏蔽内容表中确定出相应的屏蔽字段；Determine a data type corresponding to the preprocessed data, and determine a corresponding masking field in a preset masking content table based on the data type; 基于所述屏蔽字段，对所述预处理后的数据进行特定字段遮盖处理。Based on the masked field, specific field masking processing is performed on the preprocessed data. 5.根据权利要求4所述的一种基于Connector的数据交换方法，其特征在于，所述基于所述屏蔽字段，对所述预处理后的数据进行特定字段遮盖处理，具体包括：5. A Connector-based data exchange method according to claim 4, characterized in that the specific field masking process is performed on the pre-processed data based on the masked field, specifically comprising: 基于所述屏蔽字段对应的字段类型，确定出所述屏蔽字段对应的屏蔽类型；其中，所述屏蔽类型至少包括固定字符替换、生成随机字符、保留部分字符以及字符加密中的一项；Based on the field type corresponding to the masked field, determining the mask type corresponding to the masked field; wherein the mask type includes at least one of fixed character replacement, random character generation, partial character retention, and character encryption; 确定出所述字段类型对应的加密等级，基于所述加密等级对所述屏蔽类型进行叠加；Determining an encryption level corresponding to the field type, and superimposing the shielding type based on the encryption level; 基于叠加后的屏蔽类型，对所述屏蔽字段进行遮盖处理。Based on the superimposed shielding type, the shielding field is masked. 6.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述获取用户对应的身份凭证，在所述身份凭证检测通过的情况下，确定所述用户具有访问Connector的权限，具体包括：6. A method for data exchange based on Connector according to claim 1, characterized in that the step of obtaining the identity credentials corresponding to the user and determining that the user has the right to access the Connector when the identity credentials are detected successfully comprises: 获取所述用户对应的身份凭证；其中，所述身份凭证已通过预置加密算法加密；Obtaining the identity credential corresponding to the user; wherein the identity credential has been encrypted using a preset encryption algorithm; 基于所述预置加密算法对应的解密算法对所述身份凭证进行解密，得到参考身份凭证与参考时间戳；Decrypting the identity credential based on a decryption algorithm corresponding to the preset encryption algorithm to obtain a reference identity credential and a reference timestamp; 对所述参考身份凭证进行哈希值计算，并将计算的哈希值与预存的哈希值进行比对；以及Calculating a hash value for the reference identity credential and comparing the calculated hash value with a pre-stored hash value; and 将所述参考时间戳进行实效性检测；Performing validity testing on the reference timestamp; 在比对结果一致且所述实效性检测通过的情况下，基于所述参考身份凭证对用户进行数据查询权限分配，基于所述数据查询权限确定出用户访问Connector的权限。When the comparison results are consistent and the validity test is passed, data query permissions are allocated to the user based on the reference identity credentials, and the user's access permissions to the Connector are determined based on the data query permissions. 7.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述基于预置访问规则，确定具有所述访问Connector的权限的用户对应的数据访问范围，具体包括：7. A Connector-based data exchange method according to claim 1, characterized in that the step of determining the data access scope corresponding to the user having the permission to access the Connector based on the preset access rules specifically comprises: 基于所述预置访问规则、所述用户身份与所述用户角色，确定出用户对应的参考数据访问范围；Determine the reference data access scope corresponding to the user based on the preset access rule, the user identity and the user role; 基于所述用户对应的历史访问信息，确定出异常访问信息与正常访问信息，基于所述异常访问信息与所述正常访问信息之间的数值比值，确定出所述用户对应的初始置信度；Based on the historical access information corresponding to the user, determine abnormal access information and normal access information, and based on the numerical ratio between the abnormal access information and the normal access information, determine the initial confidence corresponding to the user; 获取与所述用户存在关联关系的参考用户，获取各所述参考用户分别对应的参考置信度；Acquire reference users associated with the user, and acquire reference confidences corresponding to the reference users; 基于所述初始置信度与各所述参考置信度，确定出所述用户对应的最终置信度，并基于所述最终置信度对所述参考数据访问范围进行调整，以得到所述数据访问范围。Based on the initial confidence and each of the reference confidences, a final confidence corresponding to the user is determined, and based on the final confidence, the reference data access range is adjusted to obtain the data access range. 8.根据权利要求1所述的一种基于Connector的数据交换方法，其特征在于，所述基于所述数据访问范围，对所述进行特定字段遮盖处理后的所述所需数据进行查询，以将符合要求的数据展示给用户，具体包括：8. A Connector-based data exchange method according to claim 1, characterized in that the querying of the required data after the specific field masking processing is performed based on the data access scope to display the data that meets the requirements to the user specifically includes: 获取用户发送的查询信息，基于所述查询信息，在所述数据访问范围内进行查询，得到待展示数据；Acquire query information sent by the user, and perform a query within the data access scope based on the query information to obtain data to be displayed; 基于用户权限，确定出可查看字段，将所述可查看字段与遮盖的特定字段进行匹配，并基于匹配结果对所述待展示数据中遮盖的特定字段进行展示处理；Based on the user authority, the viewable fields are determined, the viewable fields are matched with the covered specific fields, and the covered specific fields in the data to be displayed are displayed based on the matching result; 获取所述用户对应的历史查询信息，并基于查询次数对所述历史查询信息进行排序；Obtaining historical query information corresponding to the user, and sorting the historical query information based on the number of queries; 将所述展示处理后的数据与所述历史查询信息进行匹配，以确定出所述展示处理后的数据对应的展示顺序；Matching the processed display data with the historical query information to determine a display order corresponding to the processed display data; 基于所述展示顺序，将展示处理后的数据展示给所述用户。Based on the display order, the data after display processing is displayed to the user. 9.一种基于Connector的数据交换设备，其特征在于，该设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发该设备执行权利要求1-8中的任一项所述的方法。9. A Connector-based data exchange device, characterized in that the device comprises a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute any one of the methods described in claims 1-8. 10.一种非易失性计算机存储介质，存储有计算机可执行指令，其特征在于，所述计算机可执行指令能够执行权利要求1-8任一项所述的方法。10. A non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions can execute the method according to any one of claims 1 to 8.
CN202410944390.1A 2024-07-15 2024-07-15 Data exchange method, device and medium based on Connector Pending CN118964454A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410944390.1A CN118964454A (en) 2024-07-15 2024-07-15 Data exchange method, device and medium based on Connector

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410944390.1A CN118964454A (en) 2024-07-15 2024-07-15 Data exchange method, device and medium based on Connector

Publications (1)

Publication Number Publication Date
CN118964454A true CN118964454A (en) 2024-11-15

Family

ID=93406022

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410944390.1A Pending CN118964454A (en) 2024-07-15 2024-07-15 Data exchange method, device and medium based on Connector

Country Status (1)

Country Link
CN (1) CN118964454A (en)

Similar Documents

Publication Publication Date Title
US12175002B2 (en) Cloud-based system for protecting sensitive information in shared content
US11520922B2 (en) Method for personal data administration in a multi-actor environment
US7363650B2 (en) System and method for incrementally distributing a security policy in a computer network
US7350226B2 (en) System and method for analyzing security policies in a distributed computer network
JP4443224B2 (en) Data management system and method
US20060200664A1 (en) System and method for securing information accessible using a plurality of software applications
US20140223578A1 (en) Secure data delivery system
US20120215809A1 (en) Search mediation system
CN112567716B (en) Secure data transmission system and method
US20240220652A1 (en) Data processing apparatus and methods
US11868339B2 (en) Blockchain based distributed file systems
CN117494163B (en) Data service method and device based on security rules
US20050102291A1 (en) Apparatus and method providing distributed access point authentication and access control with validation feedback
JP3843405B2 (en) Personal information recording method, personal information recording system and recording medium
CN115019920A (en) Medical archive management system based on intelligent identification technology
KR102771347B1 (en) Integrated log data security management system based on blockchain
CN118964454A (en) Data exchange method, device and medium based on Connector
KR20240160589A (en) Identity Verification and Related Platforms
JP5167795B2 (en) Database maintenance method and system
JP4805615B2 (en) Access control method
CN117195256B (en) Financial data processing method and system
Simske et al. APEX: Automated policy enforcement eXchange
CN120768649A (en) Data management method, device, electronic device and medium
HK1144480A (en) Search mediation system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination